MT5 and speed in action - page 2

 
fxsaber:

Please give a rudimentary diagram-code for this idea. At first glance it sounds like a complete misunderstanding.

During execution of OnMain function there is no way to know state of current real queue. The only workaround to do this is a spyware program, as given in the link.

In its simplest form:

OnXXX( параметры )
{
        запомнить параметры (поместить в очередь)
        OnMain();
}
On2XX( параметры )
{
/*вычисления*/
        поменять приоритет обработки событий (если нужно)
/*вычисления*/
        промежуточный return (если нужно) 
/*вычисления*/
}
void OnMain()
{
        прочитать приоритет обработки событий
        для каждой группы событий
                извлечь параметры из очереди
                OnTradeXXX( параметры )
                удалить событие из очереди
}

You just need to change the approach to the calculations themselves (do intermediate return as often as the task requires). But if it is difficult, consider at the 1-step that OnMain does not exist for you (you move the main code not in OnMain, but in On2XX), respectively, in OnMain you don't need to learn anything

 
A100:

Crutches are being made in the wrong direction:

Leave the OnXXX functions only copying events (parameters) in the queue and calling the main OnMain function. Move all their code to duplicate On2XX functions. And call these duplicating On2XX functions from the OnMain in the sequence in which you need, passing them in turn the data from the queues as parameters.

Then run measurements and show the speed gain, and then you can suggest to complement MQL with appropriate functionality. But why add, if you have already done everything yourself here and now?

That's not what you're writing about.
The problem is that the functionality of filling fields in input variables of "OntaredeTransaction" function does not correspond to the description given in Help, for the worse:
i.e. many fields are not filled in calls and even in final call of TRADE_TRANSACTION_HISTORY_ADD.
Consequently, all traders are agonizing here to somehow determine these missing parameters at the moment of arrival.
There were many discussions here, just do a search on the forum - poor "OntaredeTransaction" functionality causes additional expenses both in time and "loads" traders with extra time spent on development of adequately working code (i.e. huge time losses and colossal ineffectiveness at community level).
The current replies from developers like "If you have a Market execution symbol, it will have zero price value; it should be like this" (Link).(Link) - this is UNNORMAL, once again -
lack of exhaustive values in the last function call leads to a lot of extra work.

Новая версия платформы MetaTrader 5 build 2450: Сервис "Подписки", улучшения в интерфейсе и удобные функции в MetaEditor
Новая версия платформы MetaTrader 5 build 2450: Сервис "Подписки", улучшения в интерфейсе и удобные функции в MetaEditor
  • 2020.05.18
  • www.mql5.com
В пятницу 22 мая 2020 года будет выпущена обновленная версия платформы MetaTrader 5...
 
fxsaber:

HistorySelect.


This is an insanely expensive feature. And unfortunately, no amount of caching can make its speed acceptable now.


For example, in battlefield EAs it is important to work with actual data. In particular, that the ticks from Market Watch and CopyTicks are not outdated if possible.

To do this, there are not very good, but forced mechanisms to check the relevance of current price data. One part of this mechanism is to detect situations where a trade on a symbol has passed later than the last tick. This does not happen rarely, so it is important to know if the current tick is still up to date or not.


For this detection, you need to be able to access the history of recent deals. It is done using the HistorySelect in the following schematic way.


Unfortunately, such a call of HistorySelect takes 5-30 milliseconds (I measured it myself in Release-EX5). When the Expert Advisor makes several such updates in OnTick (it should be done after any pause, for example, after each OrderSend), then everything becomes insanely expensive/long. HistorySelect can collectively eat up several seconds in a single OnTick.


Dear developers, why is it so expensive? This function can execute instantly with proper implementation.

Do you have evidence about "such a call to HistorySelect lasts 5-30 milliseconds"?

If I understood your idea correctly, then the test code should look like this:

void OnStart()
  {
   MqlTick Tick;
   SymbolInfoTick(_Symbol, Tick);
   ulong start=GetMicrosecondCount();
   for(int i=0; i<100000; i++)
     {
      HistorySelect(Tick.time, INT_MAX);
     }
   ulong end=GetMicrosecondCount()-start;
   Print("100000 HistorySelect = ",DoubleToString(end/1000.0,2)," ms");
  }

This is how 100000 queries work:

100000 HistorySelect = 141.63 ms
 
Anton:

Do you have evidence about "such a call to HistorySelect lasts 5-30 milliseconds"?

If I understood your point correctly, then the test code should look like this:

This is how 100,000 queries work:

And one query? I wouldn't be surprised if the compiler, when optimising, did just that)))
 
Anton:

Do you have evidence about "such a call to HistorySelect lasts 5-30 milliseconds"?

If I understood your point correctly, then the test code should look like this:

This is how 100,000 queries work:

Running your script.

        100000 HistorySelect = 19493.96 ms


Launch another script.

#define  PRINT(A) Print(#A + " = " + (string)(A))

void OnStart()
{
  if (HistorySelect(0, INT_MAX))
  {
    PRINT(HistoryDealsTotal());
    PRINT(HistoryOrdersTotal());
  }
}


HistoryDealsTotal() = 17828
HistoryOrdersTotal() = 22142


Normal battle count. Not HFT.


By the way, your script showed that HistorySelect recalculates everything every time. And the input parameters didn't change, the history table wasn't updated, other HistorySelect functions weren't called.

 
fxsaber:

Running your script.

Then details are needed.

My test was running on not the fastest "Intel Xeon E5-2630 v4 @ 2.20GHz" with a lot of other processes in the background.

The test account had 31315 orders in its history more or less evenly since 2009, including 8 orders for today.

Perhaps, you had some script or Expert Advisor running at the same time causing a drastic loading of the terminal databases. Try it on a "clean" terminal.

More informative test:

void OnStart()
  {
   MqlTick Tick;
   SymbolInfoTick(_Symbol, Tick);
//---   
   ulong start=GetMicrosecondCount();
   int count=100000;
   for(int i=0; i<count; i++)
     {
      HistorySelect(Tick.time, INT_MAX);
     }
   ulong end=GetMicrosecondCount()-start;

   Print(count," HistorySelect = ",DoubleToString(end/1000.0,2)," ms");
   Print("Selected = ",HistoryOrdersTotal());
//---   
   Tick.time=(Tick.time/86400)*86400;
   
   start=GetMicrosecondCount();
   count=1000;
   for(int i=0; i<count; i++)
     {
      HistorySelect(Tick.time, INT_MAX);
     }
   end=GetMicrosecondCount()-start;

   Print(count," HistorySelect = ",DoubleToString(end/1000.0,2)," ms");
   Print("Selected = ",HistoryOrdersTotal());
//---   
   HistorySelect(0, INT_MAX);
   Print("Selected = ",HistoryOrdersTotal());
  }

Three runs:

2020.05.29 13:53:14.281 TestHistorySelect (EURUSD,H1)   100000 HistorySelect = 141.76 ms
2020.05.29 13:53:14.281 TestHistorySelect (EURUSD,H1)   Selected = 0
2020.05.29 13:53:14.284 TestHistorySelect (EURUSD,H1)   1000 HistorySelect = 2.82 ms
2020.05.29 13:53:14.284 TestHistorySelect (EURUSD,H1)   Selected = 8
2020.05.29 13:53:15.566 TestHistorySelect (EURUSD,H1)   Selected = 31315
2020.05.29 13:53:16.930 TestHistorySelect (EURUSD,H1)   100000 HistorySelect = 138.30 ms
2020.05.29 13:53:16.930 TestHistorySelect (EURUSD,H1)   Selected = 0
2020.05.29 13:53:16.933 TestHistorySelect (EURUSD,H1)   1000 HistorySelect = 2.61 ms
2020.05.29 13:53:16.933 TestHistorySelect (EURUSD,H1)   Selected = 8
2020.05.29 13:53:18.176 TestHistorySelect (EURUSD,H1)   Selected = 31315
2020.05.29 13:53:22.217 TestHistorySelect (EURUSD,H1)   100000 HistorySelect = 142.94 ms
2020.05.29 13:53:22.217 TestHistorySelect (EURUSD,H1)   Selected = 0
2020.05.29 13:53:22.220 TestHistorySelect (EURUSD,H1)   1000 HistorySelect = 2.57 ms
2020.05.29 13:53:22.220 TestHistorySelect (EURUSD,H1)   Selected = 8
2020.05.29 13:53:23.498 TestHistorySelect (EURUSD,H1)   Selected = 31315
 
Anton:

Then details are needed.

My test was running on not the fastest "Intel Xeon E5-2630 v4 @ 2.20GHz" with lots of other processes in the background.

The test account had 31315 orders in its history more or less evenly since 2009, including 8 orders for today.

Perhaps, you had some script or Expert Advisor running at the same time which has drastically loaded the terminal databases. Try it on a "clean" terminal.

Empty Terminal on the same account and machine.

        100000 HistorySelect = 19367.01 ms


On other accounts and Terminals the same picture. Configuration.

2020.05.29 13:53:44.766 Terminal        ICMarkets - MetaTrader 5 x64 build 2453 started for International Capital Markets Pty Ltd.
2020.05.29 13:53:44.766 Terminal        Windows 7 Service Pack 1 build 7601, Intel Core i7-2700 K  @ 3.50 GHz, 6 / 15 Gb memory, 4 / 29 Gb disk, IE 11, Admin, GMT+2


I ask readers to give their results of this script.

void OnStart()
{
  // https://www.mql5.com/ru/forum/342090/page2#comment_16607997
#define  PRINT(A) Print(#A + " = " + (string)(A))
  
  if (HistorySelect(0, INT_MAX))
  {
    PRINT(HistoryDealsTotal());
    PRINT(HistoryOrdersTotal());
  }

   // https://www.mql5.com/ru/forum/342090/page2#comment_16607922
   MqlTick Tick;
   SymbolInfoTick(_Symbol, Tick);
   ulong start=GetMicrosecondCount();
   for(int i=0; i<100000; i++)
     {
      HistorySelect(Tick.time, INT_MAX);
     }
   ulong end=GetMicrosecondCount()-start;
   Print("100000 HistorySelect = ",DoubleToString(end/1000.0,2)," ms");
}
 
Anton:

A more informative test:

Three starts:

        100000 HistorySelect = 18344.01 ms
        Selected = 0
        1000 HistorySelect = 602.77 ms
        Selected = 1169
        Selected = 22142
        100000 HistorySelect = 18331.15 ms
        Selected = 0
        1000 HistorySelect = 446.45 ms
        Selected = 1169
        Selected = 22142
        100000 HistorySelect = 18495.35 ms
        Selected = 0
        1000 HistorySelect = 549.40 ms
        Selected = 1169
        Selected = 22142

Empty Terminal 2460.


ZS Run on empty account.

        100000 HistorySelect = 28.30 ms
        Selected = 0
        1000 HistorySelect = 0.28 ms
        Selected = 0
        Selected = 0


The speed seems to be strongly influenced by the number of trades, not orders.

 
fxsaber:

Empty Terminal on the same account and machine.


On other accounts and Terminals the same pattern. Configuration.


I ask readers to quote their results of this script.

It doesn't prove anything, but the fact that with every new run (on a different symbol), the time increases - is alarming...

2020.05.29 13:59:36.625 test1 (USDJPY,H1)       HistoryDealsTotal() = 1
2020.05.29 13:59:36.626 test1 (USDJPY,H1)       HistoryOrdersTotal() = 0
2020.05.29 13:59:36.642 test1 (USDJPY,H1)       100000 HistorySelect = 16.00 ms
2020.05.29 13:59:53.522 test1 (EURUSD,H1)       HistoryDealsTotal() = 1
2020.05.29 13:59:53.522 test1 (EURUSD,H1)       HistoryOrdersTotal() = 0
2020.05.29 13:59:53.546 test1 (EURUSD,H1)       100000 HistorySelect = 24.36 ms
2020.05.29 14:00:03.640 test1 (USDCHF,H1)       HistoryDealsTotal() = 1
2020.05.29 14:00:03.640 test1 (USDCHF,H1)       HistoryOrdersTotal() = 0
2020.05.29 14:00:03.669 test1 (USDCHF,H1)       100000 HistorySelect = 29.25 ms
2020.05.29 14:00:14.994 test1 (GBPUSD,H1)       HistoryDealsTotal() = 1
2020.05.29 14:00:14.994 test1 (GBPUSD,H1)       HistoryOrdersTotal() = 0
2020.05.29 14:00:15.026 test1 (GBPUSD,H1)       100000 HistorySelect = 31.76 ms
 
Сергей Таболин:

Need to run on an account with a long trading history.