Features of the mql5 language, subtleties and tricks - page 255

New comment
 
amrali #:

I checked days_from_civil() and epoch_days_fast() from the links you sent.

They are not the fastest (maybe it is the MQL5 compiler).

There's interest in the explanations there.

 
fxsaber #:


x64:

this option does not work. The checksum is not correct. It's limited to one month. You must have forgotten.
Different processors. I have a 13th generation Intel


 
Nikolai Semko #:

this option doesn't work. The checksum is not correct.

Sorry, I forgot.

There solved a different problem - search for the day of the month. I have not tried the GetMonth task.

 
fxsaber #:

It was solving a different problem - finding the day of the month. I haven't tried the GetMonth task.

Finding the month is the most difficult task. To get the day of the month, you need to add a couple of lines of code to this function.
But it would be interesting to implement it without an array
 

Updated Benchmarks

Fastest functions to construct datetime values from date components (year, month and day).

//+------------------------------------------------------------------+
//| Create a datetime value from the given year, month and day.      |
//| https://www.mql5.com/en/forum/393227/page251#comment_53067868    |
//+------------------------------------------------------------------+
datetime CreateDateTime4(const int year, const int month, const int day)
  {
   static const int dm[] = {0, 0, 31, 59, 90, 120, 151, 181, 212,243, 273, 304, 334};
   int leap_days = (year - 1968) / 4 - (year % 4 == 0 && month < 3);
   return datetime((year - 1970) * 365 + leap_days + dm[month] + day - 1) * 86400;
  }
//+------------------------------------------------------------------+
//| https://www.mql5.com/en/forum/393227/page254#comment_53104384    |
//+------------------------------------------------------------------+
datetime CreateDateTime5(const int Year, const int Month, const int Day)
  {
   static const uint Months[] = {0, 11512676, 11512180, 11511728, 11511232, 11510750, 11510256,
                                    11509774, 11509280, 11508781, 11508302, 11507806, 11507326
                                };

   return (((Year * 5844 - Months[Month]) >> 4) + Day - 1) * 86400;
  }
//+------------------------------------------------------------------+
//| Create a datetime value from the given year, month and day.      |
//+------------------------------------------------------------------+
datetime CreateDateTime6(const int year, const int month, const int day)
  {
   datetime jd = (365 * (year - 1)) + (year - 1) / 4 +
                 (367 * month - 362) / 12 +
                 (month <= 2 ? 0 : (year % 4 == 0 ? -1 : -2)) +
                 (day - 719178);

   return (jd * (60 * 60 * 24));
  }

Benchmark:

void OnStart()
  {
   int k = 5000;
   ulong sum1 = 0;
   ulong sum2 = 0;
   ulong sum3 = 0;
   ulong sum4 = 0;
   ulong start = GetMicrosecondCount();
   for(int i = 0; i < k; i++)
      for(int y = 1970; y <= 2099; y++)
         for(int m = 1; m <= 12; m++)
            for(int d = 1; d <= 28; d++)
              {
               MqlDateTime st = {y, m, d};
               sum1 += StructToTime(st);  // <-------------------
              }
   PrintFormat("MQL StructToTime() -> %.1f msec", (GetMicrosecondCount()-start)/1000.);
   start = GetMicrosecondCount();
   for(int i = 0; i < k; i++)
      for(int y = 1970; y <= 2099; y++)
         for(int m = 1; m <= 12; m++)
            for(int d = 1; d <= 28; d++)
              {
               sum2 += CreateDateTime4(y, m, d);  // <-------------------
              }
   PrintFormat("CreateDateTime4()  -> %.1f msec", (GetMicrosecondCount()-start)/1000.);
   start = GetMicrosecondCount();
   for(int i = 0; i < k; i++)
      for(int y = 1970; y <= 2099; y++)
         for(int m = 1; m <= 12; m++)
            for(int d = 1; d <= 28; d++)
              {
               sum3 += CreateDateTime5(y, m, d);  // <-------------------
              }
   PrintFormat("CreateDateTime5()  -> %.1f msec", (GetMicrosecondCount()-start)/1000.);
   start = GetMicrosecondCount();
   for(int i = 0; i < k; i++)
      for(int y = 1970; y <= 2099; y++)
         for(int m = 1; m <= 12; m++)
            for(int d = 1; d <= 28; d++)
              {
               sum4 += CreateDateTime6(y, m, d);  // <-------------------
              }
   PrintFormat("CreateDateTime6()  -> %.1f msec", (GetMicrosecondCount()-start)/1000.);
//--- Display results.
   Print("sum1: ", sum1);
   Print("sum2: ", sum2);
   Print("sum3: ", sum3);
   Print("sum4: ", sum4);
  }

Results:

X64:
2024.04.19 10:57:28.649 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   MQL StructToTime() -> 6542.2 msec
2024.04.19 10:57:28.709 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   CreateDateTime4()  -> 59.5 msec
2024.04.19 10:57:28.737 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   CreateDateTime5()  -> 28.3 msec
2024.04.19 10:57:28.739 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   CreateDateTime6()  -> 2.2 msec
2024.04.19 10:57:28.739 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum1: 447943668480000000
2024.04.19 10:57:28.739 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum2: 447943668480000000
2024.04.19 10:57:28.739 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum3: 447943668480000000
2024.04.19 10:57:28.739 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum4: 447943668480000000

AVX2:

2024.04.19 11:00:49.809 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   MQL StructToTime() -> 6377.5 msec
2024.04.19 11:00:49.830 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   CreateDateTime4()  -> 20.5 msec
2024.04.19 11:00:49.843 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   CreateDateTime5()  -> 13.6 msec
2024.04.19 11:00:49.844 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   CreateDateTime6()  -> 0.8 msec
2024.04.19 11:00:49.844 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum1: 447943668480000000
2024.04.19 11:00:49.844 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum2: 447943668480000000
2024.04.19 11:00:49.844 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum3: 447943668480000000
2024.04.19 11:00:49.844 benchmark_CreateDateTime (EURUSD_Custom+3,H1)   sum4: 447943668480000000

Impressive speedup! more than 3000x times faster than the native way (StructToTime).

Also faster than all the variations posted before.

MQL compiler is doing some magic!

previous benchmark here

Documentation on MQL5: Constants, Enumerations and Structures / Data Structures / Date Type Structure
Documentation on MQL5: Constants, Enumerations and Structures / Data Structures / Date Type Structure
  • www.mql5.com
The date type structure contains eight fields of the int type: Note The day number of the year day_of_year for the leap year, since March, will...
 
amrali #:

Impressive acceleration! More than 3000 times faster than the native method (StructToTime).

Also faster than all variants posted earlier.

The MQL compiler works magic!

previous benchmark here

If you disable compiler optimisation.

CreateDateTime4()  -> 2896.8 msec
CreateDateTime5()  -> 1238.9 msec
CreateDateTime6()  -> 2697.4 msec


The results of the optimiser are too fantastic. This is most likely the reason why Nikolay does rand-sampling.

 
Nikolai Semko #:
The checksum with iBarShift will not match , because iBarShift works with real bars. The checksum will match only on MN1 and W1 timeframes, because there are no holes in the history of such bars.

I have no match on GBPUSD, I changed only the first print.

2024.04.19 23:50:58.534 timeToStartMonth (GBPUSD,H1)    ======== LOOP = 10000 ======== STEPS = 100000 seconds ======== GBPUSD ======== PERIOD_MN1 ========
2024.04.19 23:50:58.535 timeToStartMonth (GBPUSD,H1)    контрольная сумма - 12123055555200, время выполнения 1 итерации = 44.90 наносекунд - Расчет через структуру MqlDateTime
2024.04.19 23:50:58.535 timeToStartMonth (GBPUSD,H1)    контрольная сумма - 12123055555200, время выполнения 1 итерации = 13.80 наносекунд - Быстрый расчет
2024.04.19 23:50:58.538 timeToStartMonth (GBPUSD,H1)    контрольная сумма - 12125912112000, время выполнения 1 итерации = 322.20 наносекунд - Расчет через iBarShift
2024.04.19 23:50:58.538 timeToStartMonth (GBPUSD,H1)    ========================================================================
 
Andrei Iakovlev #:

I don't have a match on GBPUSD, only changed the first print.

Well, if the checksum is the same with the calculation through the structure, then MN1 has history holes.

Maybe the history hasn't loaded yet, or the validation range is larger than the historical data



 

We can summarise
A set of fast structure-less functions to get date and time parameters up to 2100 by a single input parameter datetime:

char GetSeconds(datetime time) { // return 0..59
   return char(time%60);
}
//+------------------------------------------------------------------+
char GetMinutes(datetime time) { // return 0..59
   return char((time/60)%60);
}
//+------------------------------------------------------------------+
char GetHours(datetime time) { // return 0..23
   return char((time/(60*60))%24);
}
//+------------------------------------------------------------------+
char GetDayOfMonth(datetime time) { // return 1...31
   static const int dm[12] = {0,31,61,92,122,153,184, 214, 245, 275, 306, 337};
   static int last_days = 0;
   static char last_result = 0;
   int days = int(time/(24*60*60));
   if (last_days!=days) {
      last_days = days;
      int d1 = (days+306+365)%1461;
      int y = d1/365;
      char m = 0;
      if (d1==1460) {
         last_result = 29;
         return 29;
      }
      int d = d1-y*365+1;
      if (d!=31) if (d==276) m=9;
         else m = char (d/30.68);
      last_result = char(d-dm[m]);
   }
   return last_result;
}
//+------------------------------------------------------------------+
char GetDayOfWeek(datetime time) {  // return 0 - Sunday, 1 - Monday,  .... 6 - Saturday
   return char(((time/(60*60*24))+4)%7);
}
//+------------------------------------------------------------------+
char GetMonth(datetime time) { // return 1...12
   static int last_days = 0;
   static char last_result = 0;
   int days = int(time/(24*60*60));
   if (last_days!=days) {
      last_days = days;
      int d1 = (days+306+365)%1461;
      char m = 3;
      if (d1==1460) {
         last_result = 2;
         return 2;
      }
      int d = d1-(d1/365)*365+1;
      if (d!=31) if (d==276) {
            last_result = 12;
            return 12;
         } else m = char (d/30.68)+3;
      if (m>12) m-=12;
      last_result = m;
   }
   return last_result;
}
//+------------------------------------------------------------------+
int GetYear(const datetime time) { // return 1970 ... 2099
   return((int)((time / (3600*24) * 4 + 2) / 1461) + 1970);
}
//+------------------------------------------------------------------+
 

With the help from codes by fxsaber here and here and some codes from this webpage source code, I was able to devise a very fast function to get year, month and day of year.

I updated the previous test done by Nokali before.

#define  DAY (24 * 3600)
#define  SIZE 10000000
//+------------------------------------------------------------------+
void OnStart() {
   ulong sum = 0;
   datetime t[];
   ArrayResize(t, SIZE);
   for(int i = 0;i<SIZE;i++) {
      t[i] = datetime(randUlong(ulong(365)*128*3600*24));
   }
   ulong tt = GetMicrosecondCount();
   for(int i=0; i<SIZE; i++) {
      int mm =GetMonth3(t[i]);
      sum+=mm;
   }
   tt = GetMicrosecondCount()-tt;
   Print("1 - " + DoubleToString(tt*1000.0/SIZE,2) + " ns, контрольная сумма = " + string(sum)+"   // fxsaber");
   ///////////
   sum = 0;
   tt = GetMicrosecondCount();
   for(int i=0; i<SIZE; i++) {
      uchar mm =GetMonth5(t[i]);
      sum+=mm;
   }
   tt = GetMicrosecondCount()-tt;
   Print("2 - " + DoubleToString(tt*1000.0/SIZE,2) + " ns, контрольная сумма = " + string(sum)+"   // C++ lib");
   ///////////
   sum = 0;
   tt = GetMicrosecondCount();
   for(int i=0; i<SIZE; i++) {
      uchar mm =GetMonth(t[i]);
      sum+=mm;
   }
   tt = GetMicrosecondCount()-tt;
   Print("3 - " + DoubleToString(tt*1000.0/SIZE,2) + " ns, контрольная сумма = " + string(sum)+"   // nikolai");
   ///////////
   sum = 0;
   tt = GetMicrosecondCount();
   for(int i=0; i<SIZE; i++) {
      int mm =GetMonthFast(t[i]);
      sum+=mm;
   }
   tt = GetMicrosecondCount()-tt;
   Print("4 - " + DoubleToString(tt*1000.0/SIZE,2) + " ns, контрольная сумма = " + string(sum)+"   // amrali");
   ///////////
   sum = 0;
   tt = GetMicrosecondCount();
   for(int i=0; i<SIZE; i++) {
      MqlDateTime dt;
      TimeToStruct(t[i], dt);
      int mm =dt.mon;
      sum+=mm;
   }
   tt = GetMicrosecondCount()-tt;
   Print("5 - " + DoubleToString(tt*1000.0/SIZE,2) + " ns, контрольная сумма = " + string(sum)+"  // TimeToStruct()");

}
//+------------------------------------------------------------------+
ulong randUlong(ulong max=ULONG_MAX) {
   return(((ulong)rand()<<60)|((ulong)rand()<<45)|((ulong)rand()<<30)|((ulong)rand()<<15)|(ulong)rand())%max;
}
//+------------------------------------------------------------------+
int GetMonth3( const datetime time ) {
   int time2 = (int)(time / DAY);
   const int Year = ((time2 << 2) + 2) / (365 * 4 + 1); // to 2100

   const bool Flag = ((Year & 3) == 2);
   time2 -= Year * 365 + ((Year + 2) >> 2);

   return (time2 < 59) ? (time2 + Flag) / 31 + 1
          : ((time2 >= 90) ? (time2 * 500 + 1532 * 11) / (1532 * 10) : 3);
}
//+------------------------------------------------------------------+
uchar GetMonth5(const datetime time) {
   const int dy4 =  365 * 4 + 1;
   int total_days = (int)(time / DAY);// + 306+365;
   const int td = (total_days << 2) + 2;

   const int Year = td / dy4; // to 2100
   const bool LeapYear = ((Year & 3) == 2);
   int days = (td % dy4)/4;
   const uchar JAN_AND_FEB_DAY_LEAP_YEAR = 59;
   const uchar TABLE_DAY_OF_YEAR[] = {
      1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,  // 31 январь
      2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,        // 28 февраль
      3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,  // 31 март
      4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,    // 30 апрель
      5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,5,
      6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,6,
      7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,
      8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,
      9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,
      10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,
      11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,11,
      12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,12,
   };
   return (LeapYear && days >= JAN_AND_FEB_DAY_LEAP_YEAR) ? TABLE_DAY_OF_YEAR[days - 1] : TABLE_DAY_OF_YEAR[days];
}
//+------------------------------------------------------------------+
char GetMonth(datetime time) { // return 1...12
   int days = int(time/(24*60*60));
   int d1 = (days+306+365)%1461;
   char m = 3;
   if (d1==1460) {
      return 2;
   }
   int d = d1-(d1/365)*365+1;
   if (d!=31) if (d==276) return 12;
      else m = char (d/30.68)+3;
   if (m>12) m-=12;
   return m;
}
//+------------------------------------------------------------------+
int GetMonthFast(datetime time)   // return 1...12
  {
   int days    = (int)(time / (60 * 60 * 24));
   int year    = ((days * 4 + 2) / 1461) + 1970;
   int yearday = days - ((year * 5844 - 11512676) >> 4);
   int isleap  = (year % 4 == 0);
   int leapadj = ((yearday < (59 + isleap)) ? 0 : (2 - isleap));
   int month   = ((((yearday + leapadj) * 12) + 373) / 367);
   return (month);
  }
//+------------------------------------------------------------------+

Results:

2024.04.20 03:30:56.494 TestDate2 (EURUSD,H1)   1 - 4.05 ns, контрольная сумма = 65197478   // fxsaber
2024.04.20 03:30:56.544 TestDate2 (EURUSD,H1)   2 - 4.91 ns, контрольная сумма = 65197478   // C++ lib
2024.04.20 03:30:56.576 TestDate2 (EURUSD,H1)   3 - 3.21 ns, контрольная сумма = 65197478   // nikolai
2024.04.20 03:30:56.598 TestDate2 (EURUSD,H1)   4 - 2.19 ns, контрольная сумма = 65197478   // amrali
2024.04.20 03:30:56.779 TestDate2 (EURUSD,H1)   5 - 18.13 ns, контрольная сумма = 65197478  // TimeToStruct()
2024.04.20 03:30:59.462 TestDate2 (EURUSD,H1)   1 - 4.00 ns, контрольная сумма = 65189469   // fxsaber
2024.04.20 03:30:59.511 TestDate2 (EURUSD,H1)   2 - 4.90 ns, контрольная сумма = 65189469   // C++ lib
2024.04.20 03:30:59.544 TestDate2 (EURUSD,H1)   3 - 3.23 ns, контрольная сумма = 65189469   // nikolai
2024.04.20 03:30:59.566 TestDate2 (EURUSD,H1)   4 - 2.20 ns, контрольная сумма = 65189469   // amrali
2024.04.20 03:30:59.747 TestDate2 (EURUSD,H1)   5 - 18.12 ns, контрольная сумма = 65189469  // TimeToStruct()

Edit:

GetMonthFast() uses branchless code, no if statements, and I think memoisation is not required because it takes only 2 nanosec to execute.

May be the comparisons with TimeToStruct() is not fair enough (it has to calculate all the other members of the struct), but it was included here to verify checksums.
Files:
TestDate2.mq5  11 kb
1...248249250251252253254255256257258259260261262...272
New comment