Help: array out of range, though in do...while loop I have tried to check error and exit - page 4

 
Alain Verleyen #:

I mean ArrayCopyRates. It's always the slowest method (which is logical as it gets all data of a given symbol/period).

This is retrieving 50000 bars on 28 symbols, 1 timeframe, on MT4.

What variant of the function are you talking about?

I'm talking about this function variant:

int  ArrayCopyRates(
   MqlRates&  rates_array[],   // MqlRates array, passed by reference
   string     symbol=NULL,     // symbol
   int        timeframe=0      // timeframe
   );

#
In the first variant of the function it performs the virtual data copying to the array of MqlRates type. It means that if timeseries data has been updated, rates_array[] array always will refer to the actual data.


Judging by your screenshot, you most likely measured the time of the very first function call. The first call always takes a relatively long time. But I neglect this (the very first call), for me it’s like initialization.

Moreover, after the very first call, an error will most likely be generated (unless this chart is opened in another window). After the error, I start waiting 30 seconds for the quotes to load, continuing to call ArrayCopyRates with each tick. During these 30 seconds the advisor will not use quotes.

That's why I neglect the time spent on the first calls. The time spent will become much less after a couple of dozen ticks.

 
Alain Verleyen #:
2024.02.02 02:00:21.425 Expert advisor_test2 EURUSD,Monthly: removed
2024.02.02 02:00:21.412 advisor_test2 EURUSD,Monthly: uninit reason 1
2024.02.02 02:00:19.575 advisor_test2 EURUSD,Monthly: Tick# 51, Microseconds taken: 14
2024.02.02 02:00:19.169 advisor_test2 EURUSD,Monthly: Tick# 50, Microseconds taken: 17
2024.02.02 02:00:18.568 advisor_test2 EURUSD,Monthly: Tick# 49, Microseconds taken: 16
2024.02.02 02:00:17.849 advisor_test2 EURUSD,Monthly: Tick# 48, Microseconds taken: 15
2024.02.02 02:00:17.177 advisor_test2 EURUSD,Monthly: Tick# 47, Microseconds taken: 15
2024.02.02 02:00:16.232 advisor_test2 EURUSD,Monthly: Tick# 46, Microseconds taken: 13
2024.02.02 02:00:15.653 advisor_test2 EURUSD,Monthly: Tick# 45, Microseconds taken: 13
2024.02.02 02:00:14.841 advisor_test2 EURUSD,Monthly: Tick# 44, Microseconds taken: 13
2024.02.02 02:00:14.169 advisor_test2 EURUSD,Monthly: Tick# 43, Microseconds taken: 16
2024.02.02 02:00:13.606 advisor_test2 EURUSD,Monthly: Tick# 42, Microseconds taken: 15
2024.02.02 02:00:13.044 advisor_test2 EURUSD,Monthly: Tick# 41, Microseconds taken: 17
2024.02.02 02:00:12.888 advisor_test2 EURUSD,Monthly: Tick# 40, Microseconds taken: 14
2024.02.02 02:00:12.216 advisor_test2 EURUSD,Monthly: Tick# 39, Microseconds taken: 12
2024.02.02 02:00:11.809 advisor_test2 EURUSD,Monthly: Tick# 38, Microseconds taken: 12
2024.02.02 02:00:11.325 advisor_test2 EURUSD,Monthly: Tick# 37, Microseconds taken: 15
2024.02.02 02:00:10.997 advisor_test2 EURUSD,Monthly: Tick# 36, Microseconds taken: 12
2024.02.02 02:00:10.216 advisor_test2 EURUSD,Monthly: Tick# 35, Microseconds taken: 14
2024.02.02 02:00:09.887 advisor_test2 EURUSD,Monthly: Tick# 34, Microseconds taken: 15
2024.02.02 02:00:09.450 advisor_test2 EURUSD,Monthly: Tick# 33, Microseconds taken: 13
2024.02.02 02:00:08.606 advisor_test2 EURUSD,Monthly: Tick# 32, Microseconds taken: 13
2024.02.02 02:00:08.090 advisor_test2 EURUSD,Monthly: Tick# 31, Microseconds taken: 15
2024.02.02 02:00:07.575 advisor_test2 EURUSD,Monthly: Tick# 30, Microseconds taken: 14
2024.02.02 02:00:06.543 advisor_test2 EURUSD,Monthly: Tick# 29, Microseconds taken: 14
2024.02.02 02:00:05.809 advisor_test2 EURUSD,Monthly: Tick# 28, Microseconds taken: 14
2024.02.02 02:00:05.246 advisor_test2 EURUSD,Monthly: Tick# 27, Microseconds taken: 13
2024.02.02 02:00:04.824 advisor_test2 EURUSD,Monthly: Tick# 26, Microseconds taken: 14
2024.02.02 02:00:04.621 advisor_test2 EURUSD,Monthly: Tick# 25, Microseconds taken: 12
2024.02.02 02:00:04.168 advisor_test2 EURUSD,Monthly: Tick# 24, Microseconds taken: 13
2024.02.02 02:00:03.324 advisor_test2 EURUSD,Monthly: Tick# 23, Microseconds taken: 13
2024.02.02 02:00:03.012 advisor_test2 EURUSD,Monthly: Tick# 22, Microseconds taken: 15
2024.02.02 02:00:02.340 advisor_test2 EURUSD,Monthly: Tick# 21, Microseconds taken: 15
2024.02.02 02:00:01.965 advisor_test2 EURUSD,Monthly: Tick# 20, Microseconds taken: 12
2024.02.02 02:00:01.683 advisor_test2 EURUSD,Monthly: Tick# 19, Microseconds taken: 13
2024.02.02 02:00:01.246 advisor_test2 EURUSD,Monthly: Tick# 18, Microseconds taken: 12
2024.02.02 02:00:00.605 advisor_test2 EURUSD,Monthly: Tick# 17, Microseconds taken: 14
2024.02.02 02:00:00.214 advisor_test2 EURUSD,Monthly: Tick# 16, Microseconds taken: 32
2024.02.02 02:00:00.214 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 1 seconds, 0 ticks
2024.02.02 01:59:59.652 advisor_test2 EURUSD,Monthly: Tick# 15, Microseconds taken: 42
2024.02.02 01:59:59.652 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 1 seconds, 0 ticks
2024.02.02 01:59:59.011 advisor_test2 EURUSD,Monthly: Tick# 14, Microseconds taken: 37
2024.02.02 01:59:59.011 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 2 seconds, 0 ticks
2024.02.02 01:59:58.574 advisor_test2 EURUSD,Monthly: Tick# 13, Microseconds taken: 43
2024.02.02 01:59:58.574 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 2 seconds, 0 ticks
2024.02.02 01:59:57.886 advisor_test2 EURUSD,Monthly: Tick# 12, Microseconds taken: 35
2024.02.02 01:59:57.886 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 3 seconds, 0 ticks
2024.02.02 01:59:57.246 advisor_test2 EURUSD,Monthly: Tick# 11, Microseconds taken: 36
2024.02.02 01:59:57.246 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 4 seconds, 1 ticks
2024.02.02 01:59:56.761 advisor_test2 EURUSD,Monthly: Tick# 10, Microseconds taken: 36
2024.02.02 01:59:56.761 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 4 seconds, 2 ticks
2024.02.02 01:59:51.042 advisor_test2 EURUSD,Monthly: Tick# 9, Microseconds taken: 33
2024.02.02 01:59:51.042 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 9 seconds, 3 ticks
2024.02.02 01:59:50.808 advisor_test2 EURUSD,Monthly: Tick# 8, Microseconds taken: 48
2024.02.02 01:59:50.808 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 10 seconds, 4 ticks
2024.02.02 01:59:49.402 advisor_test2 EURUSD,Monthly: Tick# 7, Microseconds taken: 39
2024.02.02 01:59:49.402 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 11 seconds, 5 ticks
2024.02.02 01:59:47.495 advisor_test2 EURUSD,Monthly: Tick# 6, Microseconds taken: 38
2024.02.02 01:59:47.495 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 13 seconds, 6 ticks
2024.02.02 01:59:46.846 advisor_test2 EURUSD,Monthly: Tick# 5, Microseconds taken: 47
2024.02.02 01:59:46.846 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 14 seconds, 7 ticks
2024.02.02 01:59:43.933 advisor_test2 EURUSD,Monthly: Tick# 4, Microseconds taken: 47
2024.02.02 01:59:43.933 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 17 seconds, 8 ticks
2024.02.02 01:59:43.376 advisor_test2 EURUSD,Monthly: Tick# 3, Microseconds taken: 40
2024.02.02 01:59:43.376 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 17 seconds, 9 ticks
2024.02.02 01:59:37.243 advisor_test2 EURUSD,Monthly: Tick# 2, Microseconds taken: 60
2024.02.02 01:59:37.243 advisor_test2 EURUSD,Monthly: Waiting for rates to be loaded. Remaining: 23 seconds, 10 ticks
2024.02.02 01:59:30.702 advisor_test2 EURUSD,Monthly: Tick# 1, Microseconds taken: 60031
2024.02.02 01:59:30.702 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 50000, timeframe M1
2024.02.02 01:59:30.702 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 50000, timeframe M5
2024.02.02 01:59:30.687 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 50000, timeframe M15
2024.02.02 01:59:30.687 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 50000, timeframe M30
2024.02.02 01:59:30.671 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 50000, timeframe H1
2024.02.02 01:59:30.655 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 45892, timeframe H4
2024.02.02 01:59:30.655 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 13347, timeframe D1
2024.02.02 01:59:30.655 advisor_test2 EURUSD,Monthly: ArrayCopyRates error 4066. Copied 2714, timeframe W1
2024.02.02 01:59:10.339 advisor_test2 EURUSD,Monthly: initialized
2024.02.02 01:59:10.339 advisor_test2 EURUSD,Monthly: Rates manager initialized. Timeframes (9): M1 M5 M15 M30 H1 H4 D1 W1 MN1
2024.02.02 01:59:08.246 Expert advisor_test2 EURUSD,Monthly: loaded successfully

Pay attention to the very first call and to calls starting from tick 17.

The first 17 ticks (in this test), that is, approximately the first 30-45 seconds, are preparation for work. Starting from tick 17, 9 calls to ArrayCopyRates + one call to RefreshRates takes an average of 15 microseconds.

The code looks like this:

void OnTick()
  {
   ulong temp = GetMicrosecondCount();
   ratesMgr.update(TimeCurrent());
   ulong temp2 = GetMicrosecondCount();
   Print("Tick# ", ++tick, ", Microseconds taken: ", temp2 - temp);
  }

Inside ratesMgr.update() there are 9 calls to ArrayCopyRates + a call to RefreshRates (because there are 9 charts in this test). You can see the number of bars on the charts in the error messages before tick #1.


I am writing this because you most likely measured this:

2024.02.02 01:59:30.702 advisor_test2 EURUSD,Monthly: Tick# 1, Microseconds taken: 60031

Although the actual time taken will be on average 15 microseconds, not 60000

 
Anil Varma #:
Will try it and revert.

did you used it?!

as i see your code's problem is running yet!

 
Mahdi Ebrahimzadeh #:

did you used it?!

as i see your code's problem is running yet!

Hi @Mahdi Ebrahimzadeh

Sorry for my delayed response as was stuck with alternate suggestions on the forum.

I have tried your suggestion now and wooola it seems to working fine. THANKS A TON :)

I would therefore inform the forum to treat this thread as 'resolved'.

I have posted below the code used, which may be useful for other members. 

//+-----------------------------------------------------------------------------------------------------------------------------+
//| METHOD:       getBarShift()
//+-----------------------------------------------------------------------------------------------------------------------------+
int CAccessTimeseries::getBarShift(ENUM_TIMEFRAMES TF,datetime mytime) {
        
                int ret = iBarShift(mSymbol,TF,mytime,false)-1;

                if(ret < 0)                                                                                                     ret = 0;
                if(ret > iBars(mSymbol,TF))                                                                             ret = iBars(mSymbol,TF);

                for(int j = ret; j < iBars(mSymbol,TF); j++) 
                        if(iTime(mSymbol,TF,j) < mytime)                                                                return((j-1 >= 0) ? j-1 : 0);

                return(-1);
} // End of method getBarShift()
//+-----------------------------------------------------------------------------------------------------------------------------+
//| METHOD:       copySeriesTHL()
//| APPLICATION:  To get synchronized time series Time, High, and Low data
//+-----------------------------------------------------------------------------------------------------------------------------+
int CAccessTimeseries::copySeriesTHL(ENUM_TIMEFRAMES pTimeFrame,datetime pStartTime,datetime pStopTime,
                                                         datetime &pTime[],double &pHigh[],double &pLow[]) {

                ulong rates_mask = COPY_RATES_TIME|COPY_RATES_HIGH|COPY_RATES_LOW;

                int copiedBars = 0;
                int start_pos    = getBarShift(pTimeFrame,pStartTime);          // Index of Bar on pTimeFrame and at pStartTime
                int stop_pos     = getBarShift(pTimeFrame,pStopTime);           // Index of Bar on pTimeFrame and at pStartTime
                int count                = start_pos - stop_pos + 1;                            // Include Index[0] bar in Count
                copiedBars = CopySeries(mSymbol,pTimeFrame,start_pos,count,rates_mask,pTime,pHigh,pLow);

                return(copiedBars);

} // End of method copySeriesTHL()
2024.02.02 13:09:45.799 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.01.30 16:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][3754] barsCopied[3754] Time[3754] High[3754] Low[3754]
2024.02.02 13:09:45.812 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.01.31 00:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][3275] barsCopied[3275] Time[3275] High[3275] Low[3275]
2024.02.02 13:09:45.824 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.01.31 08:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][2856] barsCopied[2856] Time[2856] High[2856] Low[2856]
2024.02.02 13:09:45.837 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.01.31 16:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][2376] barsCopied[2376] Time[2376] High[2376] Low[2376]
2024.02.02 13:09:45.850 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.02.01 00:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][1897] barsCopied[1897] Time[1897] High[1897] Low[1897]
2024.02.02 13:09:45.863 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.02.01 08:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][1478] barsCopied[1478] Time[1478] High[1478] Low[1478]
2024.02.02 13:09:45.875 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.02.01 16:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][998] barsCopied[998] Time[998] High[998] Low[998]
2024.02.02 13:09:45.888 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.02.02 00:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][519] barsCopied[519] Time[519] High[519] Low[519]
2024.02.02 13:09:45.902 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.02.02 08:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][100] barsCopied[100] Time[100] High[100] Low[100]
2024.02.02 13:09:45.916 iSessionVP (XAUUSD,M15) [XAUUSD Chart:PERIOD_M15] CSessionVPHisto::calculationPOCs SessionStart-Stop[2024.02.02 16:00]-[2024.02.02 09:39] barsInSession[PERIOD_M1][0] barsCopied[1] Time[1] High[1] Low[1]

 
Anil Varma #:
THANKS A TON :)

:) u r welcome, 

 

Hi

I have an update on previously mentioned copySeriesTHL() method. It does not gives correct bars within given startTime and StopTime.

I have revised it to following code, and it seems to be working correctly.

//+-----------------------------------------------------------------------------------------------------------------------------+
//| METHOD:       copySeriesTHL()
//| APPLICATION:  To get synchronized time series Time, High, and Low data
//|     Instead of using CopySeries() which uses idxStart and idxStop, we shall used CopyType() with timeStart and timeStop
//+-----------------------------------------------------------------------------------------------------------------------------+
bool CAccessTimeseries::copySeriesTHL(ENUM_TIMEFRAMES pTimeFrame,datetime pStartTime,datetime pStopTime,
                		      datetime &pTime[],double &pHigh[],double &pLow[],int &pBarsCopied) {

                string vMethod = "[" + mSymbol + "," + EnumToString(pTimeFrame) + "] " + __FUNCTION__;

                int count       = Bars(mSymbol,pTimeFrame,pStartTime,pStopTime);                // No of bars between pStartTime and pStopTime

                ArrayResize(pTime,count);                                                       ArrayResize(pHigh,count);
                ArrayResize(pLow,count);

                if(count <= 0 ||
                        CopyTime(mSymbol,pTimeFrame,pStartTime,pStopTime,pTime)                                 != count ||
                        CopyHigh(mSymbol,pTimeFrame,pStartTime,pStopTime,pHigh)                                 != count ||
                        CopyLow(mSymbol,pTimeFrame,pStartTime,pStopTime,pLow)                                   != count) {
                        PrintFormat("%s: Error[%i] Copying Time,High and Low data",vMethod,GetLastError());                      
                        pBarsCopied = count;
                        return(false);
                }

                pBarsCopied = count;
                return(true);

} // End of method copySeriesTHL()

We can now track, if bars copied (pBarsCopied) are in sync or bars not copied correctly as method returns true/false.

 
Anil Varma #:

Hi

I have an update on previously mentioned copySeriesTHL() method. It does not gives correct bars within given startTime and StopTime.

I have revised it to following code, and it seems to be working correctly.

We can now track, if bars copied (pBarsCopied) are in sync or bars not copied correctly as method returns true/false.

You should check _LastError for != ERR_SUCCESS as well.

Also you don't need the ArrayResize calls.
 
Dominik Egert #:
You should check _LastError for != ERR_SUCCESS as well.

Also you don't need the ArrayResize calls.

Thanks Dominik.

These are useful suggestions. I will incorporate them.

ArrayResize I have included for the reason "To copy a predetermined amount of data, it is recommended to use a statistically allocated buffer to avoid unnecessary memory reallocation."