Multicurrency syncronization problem

[Rafael Caetano Pinto]

I Known that is a lot of information about the thema in this forum... some useful, some no so useful... but after some hours trying, I had to post my problem and my analisys here to better understand the problem and help so many poor souls like me...

I have a multicurrency EA that uses the iATR indicator. I load this indicator insite OnInit() and I use CopyBuffer() inside de OnTimer() to get the indicator values. On strategy tester, the CopyBuffer starts running with error, but after some days, the copybuffer starts to work and the EA finish with success.

My objective is to do some mechanism to force the preload and sincronization of all symbols BEFORE start the EA (inside OnInit). Does not matter if I'll lose some secs or minutes to do this load. I only want a deterministic behavior to my EA.  

I read several docs that the moderators shared with others. The docs that help me more are https://www.mql5.com/en/docs/series/timeseries_access#synchronized and https://www.mql5.com/en/forum/321459#comment_13079471. But no signal to solve this puzzle... Can anyone help me to undestand what I'm missing to do it works well?

Thanks by advance and sorry by the long text.

Analysis:

I copy/paste the CheckLoadHistory from metaquote article and used it in the OnInit(), with the CopyTime() and others validations. All seems working (at least on sight), but When I tried to recovery the iATR data, copyBuffer returns -1 with the error: 4806=Requested data not found. Notice that the log before this errors shows that the symbol loaded enough bars on the cache. 

Note: I run the test on WIN$N M1 but I'm getting data from PETR4 D1. This is part of the test, I want to handle distint symbols on distint timeframes.

[UPDATE]
I read the https://www.mql5.com/en/forum/322393 and tested some itens that the Alain said. 

- I put the creation of the indicator before all, run the CheckLoadHistory and after run the copyBuffer. Same error... :(

This sentence worry me:

2° For multi-symbols (and/or multi-timeframes), you can ALWAYS get an error 4806 at some point because MT5 doesn't wait the data to be calculated, so it happens that the data are not ready on other symbols/timeframes. You have to deal with it, the simpler is just return and wait next tick.

Seems like this can be the case here... But I'm not sure if I did some great failure in my code (I know that is wrong initiate the indicator and get the values in the same method, I put it in this example to simplificate).

Code:

//to simplify the example, I put only the relevant flow here. Forgive me about the poor quality, I did a lot of attempts and I'm very tired and exhausted with this problem :(
int OnInit(){
   string symbol = "PETR4";
   ResetLastError();
   if(!SymbolInfoInteger(symbol,SYMBOL_SELECT)){
      PrintFormat("ERROR: symbol not selected %s: %d", symbol, _LastError);
      SymbolSelect(symbol,true);
   }

   datetime first_date;
   SeriesInfoInteger(symbol,PERIOD_D1,SERIES_FIRSTDATE,first_date);
   
   int res1 = CheckLoadHistory(symbol, PERIOD_D1, first_date);
   PrintFormat("CheckLoadHistory res: %d", res1);

   SeriesInfoInteger(symbol,PERIOD_D1,SERIES_FIRSTDATE,first_date);
   int bars=Bars(symbol,PERIOD_D1);
   Print("First date ",first_date," - ",bars," bars");
   
   datetime times[];
   int countTime1=CopyTime(symbol,PERIOD_D1,TimeCurrent(),1000,times);
   PrintFormat("Downloaded %d %s history bars for %s", countTime1, EnumToString(PERIOD_D1), symbol);
   int hATR = iATR(symbol, PERIOD_D1, 14);
   if(INVALID_HANDLE == hATR){
      PrintFormat("ERROR: Unable to load ATR for %s: %d", symbol, _LastError);
   }
   
   ResetLastError();
   double values[3];
   int res = CopyBuffer(hATR,0,0,ArraySize(values),values);
   if(res != ArraySize(values)){
      PrintFormat("%s: ERROR: Can't copy indicator buffer properly! handler=%d res=%d ERROR=%d", __FUNCTION__, hATR, res, _LastError);
   }
   ArrayPrint(values);
}




//Copy paste from metaquotes article...
int CheckLoadHistory(string symbol,ENUM_TIMEFRAMES period,datetime start_date)
  {
   datetime first_date=0;
   datetime times[100];
//--- check symbol & period
   if(symbol==NULL || symbol=="") symbol=Symbol();
   if(period==PERIOD_CURRENT)     period=Period();
//--- check if symbol is selected in the Market Watch
   if(!SymbolInfoInteger(symbol,SYMBOL_SELECT))
     {
      if(GetLastError()==ERR_MARKET_UNKNOWN_SYMBOL) return(-1);
      SymbolSelect(symbol,true);
     }
//--- check if data is present
   SeriesInfoInteger(symbol,period,SERIES_FIRSTDATE,first_date);
   if(first_date>0 && first_date<=start_date) return(1);
//--- don't ask for load of its own data if it is an indicator
   if(MQL5InfoInteger(MQL5_PROGRAM_TYPE)==PROGRAM_INDICATOR && Period()==period && Symbol()==symbol)
      return(-4);
//--- second attempt
   if(SeriesInfoInteger(symbol,PERIOD_M1,SERIES_TERMINAL_FIRSTDATE,first_date))
     {
      //--- there is loaded data to build timeseries
      if(first_date>0)
        {
         //--- force timeseries build
         CopyTime(symbol,period,first_date+PeriodSeconds(period),1,times);
         //--- check date
         if(SeriesInfoInteger(symbol,period,SERIES_FIRSTDATE,first_date))
            if(first_date>0 && first_date<=start_date) return(2);
        }
     }
//--- max bars in chart from terminal options
   int max_bars=TerminalInfoInteger(TERMINAL_MAXBARS);
//--- load symbol history info
   datetime first_server_date=0;
   while(!SeriesInfoInteger(symbol,PERIOD_M1,SERIES_SERVER_FIRSTDATE,first_server_date) && !IsStopped())
      Sleep(5);
//--- fix start date for loading
   if(first_server_date>start_date) start_date=first_server_date;
   if(first_date>0 && first_date<first_server_date)
      Print("Warning: first server date ",first_server_date," for ",symbol,
            " does not match to first series date ",first_date);
//--- load data step by step
   int fail_cnt=0;
   while(!IsStopped())
     {
      //--- wait for timeseries build
      while(!SeriesInfoInteger(symbol,period,SERIES_SYNCHRONIZED) && !IsStopped())
         Sleep(5);
      //--- ask for built bars
      int bars=Bars(symbol,period);
      if(bars>0)
        {
         if(bars>=max_bars) return(-2);
         //--- ask for first date
         if(SeriesInfoInteger(symbol,period,SERIES_FIRSTDATE,first_date))
            if(first_date>0 && first_date<=start_date) return(0);
        }
      //--- copying of next part forces data loading
      int copied=CopyTime(symbol,period,bars,100,times);
      if(copied>0)
        {
         //--- check for data
         if(times[0]<=start_date)  return(0);
         if(bars+copied>=max_bars) return(-2);
         fail_cnt=0;
        }
      else
        {
         //--- no more than 100 failed attempts
         fail_cnt++;
         if(fail_cnt>=100) return(-5);
         Sleep(10);
        }
     }
//--- stopped
   return(-3);
  }

The strategy tester log:

Test configuration:

[Rafael Caetano Pinto]

This code works.

I follow these steps:

1. I have to put the iATR first.
2. force the CheckLoadHistory (in this case, the history was preloaded, but I like to be redundant)
3. I did a while loop for the first attempts with a very long sleep between each iteraction.
4. After some attempts, the data was loaded.

I think that now I can do some improvements, like use only one sleep after try recovery data from all symbols, use sleep with ms instead of seconds, and remove the excessive code to turn this solution more elegant... 

I think that is it. If anyone of you think about an improvement or a easy way, you are welcome to comment.

Code:

   string symbol = "PETR4";
   ResetLastError();
   if(!SymbolInfoInteger(symbol,SYMBOL_SELECT)){
      PrintFormat("ERROR: symbol not selected %s: %d", symbol, _LastError);
      SymbolSelect(symbol,true);
   }

   int hATR = iATR(symbol, PERIOD_D1, 14);
   if(INVALID_HANDLE == hATR){
      PrintFormat("ERROR: Unable to load ATR for %s: %d", symbol, _LastError);
   }

   datetime first_date;
   SeriesInfoInteger(symbol,PERIOD_D1,SERIES_FIRSTDATE,first_date);
   
   int res1 = CheckLoadHistory(symbol, PERIOD_D1, first_date);
   PrintFormat("CheckLoadHistory res: %d", res1);

   SeriesInfoInteger(symbol,PERIOD_D1,SERIES_FIRSTDATE,first_date);
   int bars=Bars(symbol,PERIOD_D1);
   Print("First date ",first_date," - ",bars," bars");
   
   datetime times[];
   int countTime1=CopyTime(symbol,PERIOD_D1,TimeCurrent(),1000,times);
   PrintFormat("Downloaded %d %s history bars for %s", countTime1, EnumToString(PERIOD_D1), symbol);
   
   double values[3];
   int attempts = 0;
   
   while(attempts < 10){
      ResetLastError();
      int res = CopyBuffer(hATR,0,0,3,values);
      if(res != ArraySize(values)){
         PrintFormat("%s: ERROR: Can't copy indicator buffer properly! handler=%d res=%d ERROR=%d", __FUNCTION__, hATR, res, _LastError);
         attempts++;
      } else {
         PrintFormat("Loaded ATR with %d values after %d attempts", res, attempts);
         attempts = 10;
         ArrayPrint(values);
      }
      Sleep(2000);
   }

Log: