Can a buffer update in the middle of two commands? - page 3

New comment
 
Jeepack #: That makes perfect sense and I'm glad you clarified it. I haven't figured out the details on how to implement it in the most effective way yet but I'll surely give it some thought. Things I'm wondering about is where would you want the timer event to be triggered, would it be on every new minute at 0 seconds if using M1 as the smallest timeframe? Probably. But then I'm wondering if that means OnTick becomes useless? I was thinking of using a timer instead of OnTick but I'm still wondering how they can both be used together and complement each other. Maybe the reason is to have the OnTimer event stop when the market is closed? I'm still trying to figure this out but I understand what you mean. It's true, EURUSD can sometimes go into lull periods during the day so a timer could help with that. Thanks!

The timing or frequency at which the OnTimer() event is called is up to you to decide and set wit the EventSetTimer function. Timing with OnTimer is not necessarily synchronised with the start of a minute and the can be delays or overlaps, as all triggered events are queued. So, you have to choose your frequency with that in mind.

There is also no need to separate the functionality between OnTick() and OnTimer(). Both can be made to call a common function when the events occur, and maybe use a flag parameter to distinguish the source. That function would carry out the functionality common to both and use the flag to carry out different functionality befitting the type of event.

When coding services, the only event it can handle is in fact the OnTimer event, so you can use a similar approach as well, of how to handle historical data, indicators and trading in Services.

 
Fernando Carreiro #:

The timing or frequency at which the OnTimer() event is called is up to you to decide and set wit the EventSetTimer function. Timing with OnTimer is not necessarily synchronised with the start of a minute and the can be delays or overlaps, as all triggered events are queued. So, you have to choose your frequency with that in mind.

There is also no need to separate the functionality between OnTick() and OnTimer(). Both can be made to call a common function when the events occur, and maybe use a flag parameter to distinguish the source. That function would carry out the functionality common to both and use the flag to carry out different functionality befitting the type of event.

When coding services, the only event it can handle is in fact the OnTimer event, so you can use a similar approach as well, of how to handle historical data, indicators and trading in Services.

Sounds good, I'll be trying that for sure. Thanks for the advice!

 
One way of using the OnTimer could be simply calling the OnTick from within the OnTimer..

Also you could use a loop which repeatedly calls OnTick, starting shortly before a period ends, and ending as it finished successfully.

It would discard all On tick events, probably, but you would be executed at the time in question anyways.

Would be interesting to see how it performs during news events.

Also, why could this all be relevant? I mean at all... What's the use case?

After all, periods are something arbitrary. More or less. Also you still have the network latency, the imprecision of the server time, the fact ForEx is a distributed market, different liquidity providers, not to mention different account types and brokers...

I guess, this whole approach will behave totally different with any other broker and/or account (type).

Concerning services, isn't it so that you have a simple infinite loop in OnStart? That's at least how I understood the docs... Or is it mandatory to work with OnTimer in a service?



 
Dominik Christian Egert #: One way of using the OnTimer could be simply calling the OnTick from within the OnTimer.

No, I don't agree! Event handlers should not be called from other event handlers. Event handlers should only be called by the terminal. The only reason MQL allows it is due to these bad practices being overlooked, and one never knows how it will work in the future.

The correct practice, is for those Event handlers to call a support function which can be common to both and use some extra parameters to differentiate things.

Event handlers should also carry out their tasks as quickly as possible, otherwise the waiting queue starts to build up, as only one event is handled at a time, be it OnTick, OnTimer, OnChartEvent, etc.

Dominik Christian Egert #: Concerning services, isn't it so that you have a simple infinite loop in OnStart? That's at least how I understood the docs... Or is it mandatory to work with OnTimer in a service?

You are correct — MT5 Services only support OnStart() and not OnTimer(). I am so used to coding EAs to function as "support services" (due to the fact that I usually code things to work on both MQL4 and MQL5), and rarely code exclusively for MQL5 services, that I mixed up the concepts.

So my apologies, for creating confusion. MT5 Services (not available on MT4), do in fact only support the OnStart() event handler.
 
Dominik Christian Egert #:
Also, why could this all be relevant? I mean at all... What's the use case?

Depends what you're asking about, but as a summary for the entire post:

A quicker reaction time with no risk of copying the wrong buffer values if new ticks are received during the computation process.

Especially useful for code that queries multiple buffers on multiple timeframes and symbols.

You could also say it's one of the quickest and most reliable ways to synchronize indicator data accross multiple timeframes and symbols (in my opinion). There are simpler ways to do this but I find them to be susceptible to significant lag if one or a few symbols are in a slow trading period and receiving fewer ticks.

Maybe your question was about something Fernando said, sorry if I'm confused.
 
Jeepack #: Depends what you're asking about, but as a summary for the entire post: A quicker reaction time with no risk of copying the wrong buffer values if new ticks are received during the computation process.

Especially useful for code that queries multiple buffers on multiple timeframes and symbols. You could also say it's one of the quickest and most reliable ways to synchronize indicator data accross multiple timeframes and symbols (in my opinion). There are simpler ways to do this but I find them to be susceptible to significant lag if one or a few symbols are in a slow trading period and receiving fewer ticks.

Maybe your question was about something Fernando said, sorry if I'm confused.

I would have to agree with Dominik as to why such precision is required to align the collection of collect data and at such high frequency. For the purposes that I believe you want it for, based on your descriptions, I think it is overkill. The approach can most probably be simplified.

However, since I am unsure exactly what it is you are developing, I can't exactly discourage you. All I can do is just pass on the knowledge and let you judge how you want to use it. The reason I have been able to give you all this advice on how to achieve it, is because I have used such methods before but for a completely different reason and purpose.

 
Fernando Carreiro #:

I would have to agree with Dominik as to why such precision is required to align the collection of collect data and at such high frequency. For the purposes that I believe you want it for, based on your descriptions, I think it is overkill. The approach can most probably be simplified.

However, since I am unsure exactly what it is you are developing, I can't exactly discourage you. All I can do is just pass on the knowledge and let you judge how you want to use it. The reason I have been able to give you all this advice on how to achieve it, is because I have used such methods before but for a completely different reason and purpose.

There's 3 reasons to build a multi-symbol and multi-timeframe EA: 1) limit the computer resources required to run a strategy on multiple charts, 2) testing a strategy on multiple symbols and timeframes at the same time in the MT5 tester, or 3) manage signals on a global scale where signals produced on each chart depend on data collected from all charts.

If you're doing it for reason 1 or 2, you don't have to process data from every chart before executing a signal. You can execute signals as soon as ticks come in to form new bars on any chart. But if you're doing it for reason 3, you want to collect data from all charts before processing any signals. This is the way my EA's work.

This leads to choosing how you define data as being ready to collect. Method 1: most EA's will wait for a new bar to form to consider the data for the previous bar as being available to collect. For reasons mentioned in this post, this helps avoid the issue of buffers updating while computations are taking place, so you can safely assume that index 1 on any buffer is for the previous close. Method 2: you could also want to consider that the data is ready to collect as soon as the time period for a bar expires, no matter if a new bar has already formed or not, which requires extra verifications to make sure your data collection isn't affected by new ticks coming in (see code examples in previous replies).

If using your EA to manage signals on a global scale, and your broker is like mine during slow trading periods, the difference between using method 1 or 2, can be as little as 2-3 seconds but it can also reach as high as 30-40 seconds sometimes.

If I'm potentially opening trades on a 15 minute timeframe, this lag can be very significant when it's at its worse.

The question is does this happen often enough to make method 2 significantly better than method 1 and are you trading on a small enough timeframe for it to make a difference in your results. Maybe. Maybe not. I'm not 100% sure to be honest.

At least now I know that if I use method 2, extra verifications are required to make sure the data is properly collected. So if I choose this method, there won't be any bugs in my code.

 
Jeepack #: There's 3 reasons to build a multi-symbol and multi-timeframe EA: 1) limit the computer resources required to run a strategy on multiple charts, 2) testing a strategy on multiple symbols and timeframes at the same time in the MT5 tester, or 3) manage signals on a global scale where signals produced on each chart depend on data collected from all charts.

You misunderstand. I'm not disputing the reason for creating a multi-symbol, multi-time-frame EAs. I have my own such EAs, and for all the reasons 1, 2 and 3. What I am questioning is the complexity that you are striving to implement.

Even for method 3, I have not needed so much complexity that you seem to want to implement. These are main steps I take.

  1. For the most part, in multi-symbol EAs, I don't use indicators at all. I replicate their calculations in the EA itself in an incremental way, on each new bar event (see next step). I effectively maintain my own buffers (using ring buffers, only for the data I need, which is way more efficient than indicators).
  2. On every new bar detection/pseudo-event (OnTick/OnTimer), I collect the bar data first (current and previous bar only, per symbol), then decide if the current bar is new and the previous one closed, or if it the current bar is stale and its duration lapsed, or even if the duration of several bars has lapsed with no updates. And based on this, I carry out the incremental calculations.

That is it. There is no more to consider. These conditions allow me to have synchronised data, in order to decide whether to enter or exit trades. After that, the management of positions is based on current position closing prices.

There are no worries. Even if there are changes between reading one symbol and the next symbol, it does not matter. Even if the data is not aligned when collected, does not matter. As long as I keep a state machine for each symbol, and do the incremental calculations, my data for each and every symbol will always be correctly aligned and synchronised.

 
Fernando Carreiro #:

You misunderstand. I'm not disputing the reason for creating a multi-symbol, multi-time-frame EAs. I have my own such EAs, and for all the reasons 1, 2 and 3. What I am questioning is the complexity that you are striving to implement.

Even for method 3, I have not needed so much complexity that you seem to want to implement. These are main steps I take.

  1. For the most part, in multi-symbol EAs, I don't use indicators at all. I replicate their calculations in the EA itself in an incremental way, on each new bar event (see next step). I effectively maintain my own buffers (using ring buffers, only for the data I need, which is way more efficient than indicators).
  2. On every new bar detection/pseudo-event (OnTick/OnTimer), I collect the bar data first (current and previous bar only, per symbol), then decide if the current bar is new and the previous one closed, or if it the current bar is stale and its duration lapsed, or even if the duration of several bars has lapsed with no updates. And based on this, I carry out the incremental calculations.

That is it. There is no more to consider. These conditions allow me to have synchronised data, in order to decide whether to enter or exit trades. After that, the management of positions is based on current position closing prices.

There are no worries. Even if there are changes between reading one symbol and the next symbol, it does not matter. Even if the data is not aligned when collected, does not matter. As longer as I keep a state machine for each symbol, and do the incremental calculations, my data for each and every symbol will always be correctly aligned and synchronised.

I'm impressed with your skills Fernando! Very advanced stuff for my level of knowledge at this point but you're right, from what I understand, your method seems simple and much more efficient. We probably end up with the same results but your code must run much faster and take less computer resources. I'll definitely have to try that. If I understand this correctly, it really needs to work without indicators if you want to do it your way. Working with your own buffers means they can't update unless you update them yourself, which is a smart way to fix the issue described in this post. Good job! I won't try it right away though because I want to do it when I have the time to learn how to do it properly, but it's on my project list for sure! Thanks!

------------------------

I'm running a few last tests on my code to make sure it works as intended... I'll post a final version of the data collection function when I'm sure it's right.

 
Jeepack #: I'm impressed with your skills Fernando! ... Thanks! ... I'll post a final version of the data collection function when I'm sure it's right.

Thank you 🙏 ... You are welcome ... I wish you good coding luck 👍.

1234
New comment