[SERVICE DESK] Error in getting the time of the senior TF in the timer! - page 7

[[Deleted]]

Ihor Herasko:

Yes, exactly. In OnInit() you simply call the required TFs without checking the result (you cannot rely on it there), and in OnCalculate call the function IsTFDataReady(). Once true is returned for all requested TFs, you can start to execute the indicator's algorithm.

Okay, we have sorted it out. But we must clearly add the documentation, otherwise the fast timer will cause a lot of questions to the developers.

[[Deleted]]

Ihor Herasko:

In general, what kind of problem is being solved for which the presence of a terminal connection is so crucial? As I understand it, the indicator is a tool for visualising data. The data that is available. When new data arrives, it will update the visualization. It should not be required to check the relevance of data. This is the task of the terminal.

The task is to get the data of the senior TF as soon as possible.Vitaly Gorbunov reminded me about IsConnected().

[Slava]

Alexey Kozitsyn:

Oh, man... We're already past that point. See your own log:

Sequence. First we check the connection. Once the connection is established, we get the time. Explain to me, please, why the hell is error 4066 returned first and then it's not returned!? What has changed in 20ms since the last call?

Error 4066 says no data, update request sent.

Once the request has been sent, another request is no longer sent, hence the 4066 error is not being triggered. This has already been discussed many times.

Why do you start a timer in the indicator? It's so small. You have to understand that in MT4 indicators run in an interface thread. The interface thread is where all the wind messages go

[[Deleted]]

Slava:

Error 4066 indicates that there is no data, an update request has been sent.

Once a request has been sent, another request is no longer sent, so error 4066 is not raised. This has already been discussed many times.

Why do you start a timer in an indicator? It's so small. You must understand that in MT4 indicators work in the interface thread. All of the wind messages go through the interface thread

Glad you joined the discussion.

It's not my first day on the forum + several people have commented here who are also not the first day on the forum. No one has said anything about:

Once a request has been sent, another request is no longer sent, so the 4066 error does not get triggered. This has already been discussed many times.

Thank you, we will know. Would really like to see this in the help. So, surely the error is "raised" only when OnTick()/OnCalculate() event occurs?

Why do you start the timer in the indicator? It's so small.

You need to get data from a few characters. As fast as possible. Unfortunately, neither MT4 nor MT5 have implemented receiving events of quotes arrival of any symbol (it's impossible to subscribe to such an update), therefore the only way out (as far as I know) is to interrogate the required symbols in the timer.

You should understand that in MT4 indicators work in the interface thread. All of the wind messages go through the interface thread

OK, they're coming, but then what? Can you elaborate on how this is bad/good/how it affects the timer operation?

[Slava]

It's been discussed many times. 12 pages per request "error 4066".

And you were correctly advised to send the request in OnInit and analyse it in OnCalculate.

What do you needa millisecond timer for? You are preventing the client terminal from starting up normally. It's not the wind messages that interfere with your timer, it's your timer that interferes with everyone. Once again: THE INDICATORS IN THE CLIENT'S MT4 TERMINAL WORK IN INTERFACE POTENTIAL.

[[Deleted]]

Slava:

It's been discussed many times. 12 pages on "error 4066".

Read it, thanks. Only with work in OnCalculate() or OnTick() there is no problem, the problem is in OnTimer(). And on request "error 4066 timer" I got results only from this branch :(

And you were correctly advised to send a request to OnInit and analyze it in OnCalculate.

I listened to the advice, but it doesn't change the peculiarities of timer handling which are not mentioned in documentation.

Why do you need a millisecond timer? You are preventing the client terminal from rising normally by your actions. It's not the wind messages that interfere with your timer, it's your timer that interferes with everyone else's. Once again: THE INDICATORS IN THE CLIENT'S MT4 TERMINAL ARE WORKING IN THE INTERFACE POCKET.

Once again, the millisecond timer is to get information from multiple symbols as fast as possible! I.e., the algorithm is as follows: the indicator is loaded, gets data of high TFs as soon as possible, then it monitors bits of the needed symbols in the millisecond timer. Is there any other way to solve the monitoring problem than using the timer?

Let's summarize everything that has been written here, correct me if I'm wrong:

1. Task: get quotes for several symbols through a timer:

Implementation: When working with a high-frequency timer, you have to wait for IsConnected() to be established with the server in OnCalculate() when loading the terminal, only then you can access the timer;

2. Objective: get data of the higher TFs as soon as possible after the indicator starts (the indicator uses fast timer);

Implementation: first we request the necessary data in OnInit(), then we wait for connection IsConnected() in OnCalculate(), then we get data of the higher TFs also in OnCalculate();

3. Does starting a high-frequency timer slow down the interface thread, and consequently the computer, and should it not be started at all? How to solve task #1 then?

4. Objective: to load actual data from older TFs.

Implementation: do not use high-frequency timer for this, since data retrieval functions are not designed to work in such a timer? We use only OnCalculate()?

5. If an error 4066 is received and subsequently reset, does OnCalculate() arm it on every tick?

6. Does OnTimer() in MT5 not work in interface thread?

@Slava, please answer also point by point;

[Slava]

1. Typically you will have the IsConnected state on the second call to OnCalculate. First call immediately after starting the terminal, second call on arrival of historical data

2. Do not use fast timer. First assess what timing is acceptable to you. It may be 100 milliseconds or 500. It's no accident that we originally introduced the seconds timer, SetMillisecondsTimer was introduced in five(!) only 3 or 4 years later. But the architecture is different in five.

3.1 Get a powerful computer which can handle instant message queues.

3.2 Don't launch the millisecond timer immediately, but at least after the first OnCalculate. Or rather: in the first OnCalculate start a second timer (what if there is no connection or the day off), so that you can analyse the environment. And then, when you are sure that all data is loaded, there is a connection and everything is OK, kill the second timer and start the millisecond timer. Then you will safely make it through the narrow front door. In the best case scenario (and there will be 99 per cent of those) you will lose 2 to 5 seconds at the start

4. A timer is possible. But not immediately (see 3.2). And I think 50 milliseconds is good enough. You're not providing HFT, are you?

5. 4066 only appears on the first request for data on someone else's period-character. On the next request for the same character-period 4066 will not get any more

6. In MT5 the indicators are counted in a separate symbol processing thread. So if you have more than one chart on this symbol (or there are other indicators on this symbol), you can slow them down. But still it is not an interface thread

[[Deleted]]

Slava:

1. Typically you will have the IsConnected state on the second call to OnCalculate. First call immediately after starting the terminal, second call on arrival of historical data

2. Do not use fast timer. First assess what timing is acceptable to you. It may be 100 milliseconds or 500. It's no accident that we originally introduced the seconds timer, SetMillisecondsTimer was introduced in five(!) only 3 or 4 years later. But the architecture is different in five.

3.1 Get a powerful computer which can handle instant message queues.

3.2 Don't launch the millisecond timer immediately, but at least after the first OnCalculate. Or rather: in the first OnCalculate start a second timer (what if there is no connection or the day off), so that you can analyse the environment. And then, when you are sure that all data is loaded, there is a connection and everything is OK, kill the second timer and start the millisecond timer. Then you will safely make it through the narrow front door. In the best case scenario (and there will be 99 per cent of those) you will lose 2 to 5 seconds at the start

4. A timer is possible. But not immediately (see 3.2). And I think 50 milliseconds is good enough. You're not providing HFT, are you?

5. 4066 only appears on the first request for data on someone else's period-character. On the next request for the same period character, you will not get 4066 again.

6. In MT5 the indicators are counted in a separate symbol processing thread. So if you have more than one chart on this symbol (or there are other indicators on this symbol) you can slow them down. But still it is not an interface thread

1. That's exactly how it works;

2. This is where the problem lies. The faster the better. And the evaluation has been made. The indicator is written for arbitrage (or rather research on arbitrage), i.e. every millisecond is important and the faster a quote is received - the better;

3.1. and now the system is quite powerful: 8600k CPU, SSD terminals, 16gb DDR4 RAM;

3.2. wow... ok, took note;

4. The arbitration task is most likely related to HFT;

5. Initially, this was what was stressing me out. If I had continued to get the error and knew the data wasn't ready yet, this thread wouldn't have happened;

6. I see.

Thanks for the elaborate response.

[Ihor Herasko]

Igor Makanu:

If it's not too much trouble, here's the topic of the topic - correct history loading from the older TF, here's the indicator: "I need to draw the MA" from the older TF on the bars of the younger TF, I did it within 5 minutes, it will work for 98% correctly, where in this code 2% "pitfalls" that will cause bugs?

Yes, just on the subject of the topic. And it's all sorted out here.

First, before any reference to timeseries of other TFs/symbols, be sure to check that the data is available (see IsTFDataReady() function above). In the above code you are only guided by the result from CopyClose. But it knows nothing about history load. So first - make sure the data is available and only then request it.

Secondly, calling one function as an argument of another function is not always justified. And in the above case, it is unacceptable in principle. After all, the result of calling iBars must also be checked. So, firstly iBars is called, the result is cached and checked and then only the received value is transferred to CopyClose().

Thirdly, after the call of CopyClose there is no check to get all the requested data. After all, the function can return 1 or 2 bars, and it was requested, for example, 10. I would consider such a result an error.

Fourthly, there is an error in the very idea of the approach. The loop assumes that it operates with bars of another TF but it mixes up calculations with rates_total variable whose value refers to the current TF. Here two approaches are possible which I use in this or that case:

1. To loop through bars of the current TF and to convert the index of the bar of the current TF to the bar index of another TF before data request (this approach is used in the code below).
2. Cycle through the bars of another TF. But then we need to include an additional loop for the case when the current TF is junior to the other one. Because one bar of the older TF will correspond to several bars of the current TF.

I am interested in correct code for MT4

In the light of these four points it should look like this (I did not check it, I did it by hand, but the sense should be clear):

   if (!IsTFDataReady(TimeFrame))
      return 0;

   int i,limit;

   static int nOldBars = 0;
   int nBars = iBars(_Symbol, TimeFrame);
   if (nBars == 0)
      return 0;
      
   if (nOldBars == 0 || nBars - nOldBars > 1)
   {
      if(nBars < MAPeriod)
      {
         Comment("Большой период МА!!!, в истории доступно ", nBars," баров");
         return 0;
      }
      
      limit = nBars - fmin(MAPeriod, nBars);
   }
   else
      limit = nBars - nOldBars;  // здесь всегда будет 0 или 1
   
   nOldBars = nBars;
   datetime dtTime = iTime(NULL, TimeFrame, limit);
   if (dtTime == 0)
      return 0;

   limit = iBarShift(NULL, PERIOD_CURRENT, dtTime);

// основной цикл расчета индикатора
   for(i = limit; i >= 0 && !IsStopped(); i--)
   {
      int nOtherTFBarIndex = iBarShift(NULL, TimeFrame, time[i]);
      if (nOtherTFBarIndex < 0 || nOtherTFBarIndex >= nBars)
         continue;
      
      BufMA[i] = iMA(_Symbol,TimeFrame,MAPeriod,0,MODE_SMA,PRICE_CLOSE,nOtherTFBarIndex);
   }
//---
   return rates_total;

By the way, I checked it:

[Igor Makanu]

Ihor Herasko:

Yes, that's the topic of the thread. And it's all sorted out here already.

First, before any reference to other TF/symbol timeseries, make sure the data is available (see IsTFDataReady() function above). In the above code you are only guided by the result from CopyClose. But it knows nothing about history load. So first - make sure the data is available and only then request it.

Secondly, calling one function as an argument of another function is not always justified. And in the above case, it is unacceptable in principle. After all, the result of calling iBars should also be checked. So, firstly iBars is called, the result is cached and checked and then only the received value is transferred to CopyClose().

Thirdly, after the call of CopyClose there is no check to get all the requested data. After all, the function can return 1 or 2 bars, and it was requested, for example, 10. I would consider such a result an error.

Fourthly, there is an error in the very idea of the approach. The loop assumes that it operates with bars of another TF but it mixes up calculations with rates_total variable whose value refers to the current TF. Here two approaches are possible which I use in this or that case:

1. To loop through bars of the current TF and to convert the bar index of the current TF to the bar index of another TF before requesting data (this approach is used in the code below).
2. Cycle through the bars of another TF. But then we need to include an additional loop for the case when the current TF is junior to the other one. After all, one bar of the older TF will correspond to several bars of the current TF.

In the light of these four points this is how it should be (I did not check it, I was doing it by hand, but the sense should be clear):

By the way, I checked it:

1. took it into account! Thank you!

2. golden words! I used to write so, but with time, reading other people's codes, which are chasing compactness... I pay more attention to "brevity is the sister of talent" .... and spend my time searching for the bugs I encountered.

4. "brevity is the sister of talent"... You are right!

3. interesting point of analysis, what returns CopyClose(), I checked it myself, if there is no .hst file for requested TF, CopyClose() never returns more than 2048 - i.e. this is maximum value that can be downloaded?