MT5 and speed in action - page 16

 
fxsaber:

It would be interesting to compare the same script with other trading platforms.

MT4 b1280.

// Мониторинг длительных пиков выполнения важных функций для торговли.
#include <fxsaber\Benchmark.mqh> // https://c.mql5.com/3/321/Benchmark.mqh

input int inCycle = 10;    // Циклов проверки в одном OnTick
input int inAlertTime = 1; // Нижний порог в миллисекундах

#define _B2(A) _B(A, AlertTime)

void Check( const string Symb, const int AlertTime = 1 )
{
  MqlTick Tick;
  
  if (_B2(SymbolInfoTick(Symb, Tick)))
  {
    _B2(OrdersHistoryTotal());
    _B2(OrdersTotal());
    _B2(OrderSelect(0, SELECT_BY_POS));
    _B2(OrderSelect(0, SELECT_BY_POS, MODE_HISTORY));
    _B2(OrderSelect(0, SELECT_BY_TICKET));
        
    _B2(GetLastError());
    _B2(IsStopped());
    
    _B2(SymbolInfoDouble(Symb, SYMBOL_ASK));
    _B2(SymbolInfoDouble(Symb, SYMBOL_TRADE_TICK_VALUE));
    _B2(SymbolInfoDouble(Symb, SYMBOL_POINT));
    _B2(SymbolInfoInteger(Symb, SYMBOL_DIGITS));

    _B2(TimeCurrent());
    _B2(TimeLocal());
    
    _B2(OrderGetDouble(ORDER_PRICE_CURRENT));
    _B2(OrderGetInteger(ORDER_MAGIC));
    _B2(OrderGetString(ORDER_SYMBOL));
                
    _B2(AccountInfoDouble(ACCOUNT_EQUITY));
        
    _B2(MQLInfoInteger(MQL_TRADE_ALLOWED));
    _B2(AccountInfoInteger(ACCOUNT_TRADE_EXPERT));
    _B2(AccountInfoInteger(ACCOUNT_TRADE_ALLOWED));
    _B2(TerminalInfoInteger(TERMINAL_TRADE_ALLOWED));
    
    _B2(SymbolsTotal(true));
    _B2(SymbolName(0, true));
    _B2(Symbol());
    
    _B2(GlobalVariableCheck(NULL));
    _B2(GlobalVariableGet(NULL));
    
    _B2(ResourceFree(NULL));
  }
}

void OnTick()
{
  for (int i = 0; i < inCycle; i++)
    Check(_Symbol, inAlertTime);      
}


2020.08.27 13:09:46.799 Test6 EURUSD,M1: Alert: Time[Test6.mq4 39: AccountInfoInteger(ACCOUNT_TRADE_EXPERT)] = 7 ms.
2020.08.27 13:09:46.790 Test6 EURUSD,M1: Alert: Time[Test6.mq4 18: OrderSelect(0,SELECT_BY_POS,MODE_HISTORY)] = 2 ms.
2020.08.27 13:09:46.784 Test6 EURUSD,H1: Alert: Time[Test6.mq4 25: SymbolInfoDouble(Symb,SYMBOL_TRADE_TICK_VALUE)] = 1 ms.

Only three slips and then they very rarely popped up. Probably hard to create a brake as there is no HistorySelect and CopyTicks.

 
Fast235:

so they are both Haswell, xeon has much lower operating frequency, there will be performance degradation in performance and single tests, only in multi-threaded optimization will be an advantage. The i3 of the latest models should be much faster to run

ask the developers about cache level effect on speed and in general the speed of Zen2 and latest intel


add

Ryzen 3700x I have, you can do tests with Intel

for example with this MQL5\Scripts\UnitTests\Stat\TestStatBenchmark.mq5 script

loop it several times with a timer

We are not talking about tests here, but about delays in order execution. This delay is there and it is floating. And it bothers both the TS and me quite a lot.

 
fxsaber:

Only three things popped up and then they very rarely popped up. Must be hard to create a brake as there are no HistorySelect and CopyTicks.

Waited on MT4 as well.

2020.08.27 13:18:00.306 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 26: SymbolInfoDouble(Symb,SYMBOL_POINT)] = 1 ms.
2020.08.27 13:17:39.820 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 30: TimeLocal()] = 2 ms.
2020.08.27 13:17:32.598 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 30: TimeLocal()] = 36 ms.
2020.08.27 13:17:29.676 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 15: OrdersHistoryTotal()] = 1 ms.
2020.08.27 13:17:26.468 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 27: SymbolInfoInteger(Symb,SYMBOL_DIGITS)] = 5 ms.
2020.08.27 13:17:26.460 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 13: SymbolInfoTick(Symb,Tick)] = 1 ms.
2020.08.27 13:17:14.528 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 13: SymbolInfoTick(Symb,Tick)] = 3 ms.
2020.08.27 13:16:52.309 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 15: OrdersHistoryTotal()] = 2 ms.
2020.08.27 13:16:52.308 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 15: OrdersHistoryTotal()] = 2 ms.
2020.08.27 13:16:52.307 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 36: AccountInfoDouble(ACCOUNT_EQUITY)] = 1 ms.
2020.08.27 13:16:42.448 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 17: OrderSelect(0,SELECT_BY_POS)] = 2 ms.
2020.08.27 13:16:32.480 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 36: AccountInfoDouble(ACCOUNT_EQUITY)] = 3 ms.
2020.08.27 13:16:32.479 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 15: OrdersHistoryTotal()] = 3 ms.
2020.08.27 13:16:32.477 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 15: OrdersHistoryTotal()] = 1 ms.
2020.08.27 13:16:29.096 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 26: SymbolInfoDouble(Symb,SYMBOL_POINT)] = 1 ms.
2020.08.27 13:16:14.593 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 15: OrdersHistoryTotal()] = 23 ms.
2020.08.27 13:14:55.398 Test6 GBPCAD.rann,H1: Alert: Time[Test6.mq4 27: SymbolInfoInteger(Symb,SYMBOL_DIGITS)] = 1 ms.
2020.08.27 13:13:34.891 Test6 GBPCAD.rann,M1: Alert: Time[Test6.mq4 25: SymbolInfoDouble(Symb,SYMBOL_TRADE_TICK_VALUE)] = 2 ms.

TimeLocal in 36 milliseconds. Chose a symbol with a larger tick volume.

 

To anyone interested, here are the instructions for playback.

Who thinks it won't be touched.

2020.08.27 13:17:56.139 Test6 (EURUSD,H1)       Alert: Time[Test6.mq5 48: OrdersTotal()] = 2 ms.
2020.08.27 13:17:56.167 Test6 (EURUSD,M1)       Alert: Time[Test6.mq5 48: OrdersTotal()] = 39 ms.
2020.08.27 13:17:56.198 Test6 (EURUSD,H1)       Alert: Time[Test6.mq5 48: OrdersTotal()] = 54 ms.
2020.08.27 13:18:11.512 Test6 (EURUSD,M1)       Alert: Time[Test6.mq5 48: OrdersTotal()] = 3 ms.
 
fxsaber:

Picked a symbol with a bigger tick volume.

Not even going to check it. Imagine the most popular FORTS symbol with a tick subscription. Instead of OnTick logic in OnBookEvent. The lags must be terrible.

Need official advice on what to do to minimise lags.

 
fxsaber:

To reproduce the brakes, you need to run the script on multiple chars of the ONE character - get OnTick to be called at the same time. Then, the alerts will be pinged on every tick.

The CPU load graph shows that terminal64.exe loads up to 30% of eight logical cores. That's only four EURUSD charts with the script running. You can clearly see how much each chart is loaded at a time.

Where do so many resources go?

This question is easy to answer.

Here you copy a lot of data:

    HistorySelect(MathRand(), INT_MAX);

You're actually giving a command to take all available trading history from the terminal database into the EA environment, for later use. You are purposely trying to run down a possible caching algorithm of identical requests.

But you do not use all this data, but immediately reset it in the next line:

    _B2(HistorySelect(Tick.time, INT_MAX));

Obviously the terminal base here is a shared resource with synchronized access. And you have deliberately created thousands of orders and deals in it.

All this meaningless action is repeated 10 times with every tick from several threads at a time. And you are deliberately making these actions happen simultaneously from several threads.

So you know very well what you are doing and why, where resources are spent, and at the same time you claim that "delay is caused by excessive CPU load on the part of MT5".

Having said that, you clearly have a problem with your computer. I.e., yes, you are actively moving significant amounts of memory, but it should not affect the execution time of functions, especially not related to HistorySelect() in any way.

In our tests b2582, even with 1000 times per tick and 5 EAs on one character charts, i.e. orders of magnitude greater than your default conditions, not a single Alert was observed.

Our test system: Windows 10 build 18363, Intel Xeon E5-2630 v4 @ 2.20GHz

 
Anton:

This question is easy to answer.

This is where you copy a lot of data:

You are actually giving a command to take all available trading history from the terminal database into the EA environment, for later use. Purposely randomly trying to disrupt the possible caching algorithm of identical requests.

But you do not use all this data, but immediately reset it in the next line:

Obviously the terminal base here is a shared resource with synchronized access. And you have deliberately created thousands of orders and deals in it.

All this meaningless action is repeated 10 times with every tick from several threads at a time. And you are deliberately making these actions happen simultaneously from several threads.

So you know very well what you are doing and why, where resources are spent, and at the same time you claim that "delay is caused by excessive CPU load on the part of MT5".

Having said that, you clearly have a problem with your computer. I.e., yes, you are actively moving significant amounts of memory, but it should not affect the execution time of functions, especially not related to HistorySelect() in any way.

In our tests b2582, even with 1000 times per tick and 5 EAs on one character charts, i.e. orders of magnitude greater than your default conditions, not a single Alert was observed.

Our test system: Windows 10 build 18363, Intel Xeon E5-2630 v4 @ 2.20GHz


Colleagues,

it's time for you to come out of the aircraft modeling circle.

Here are the battle conditions: 4 terminals, about 300 Expert Advisors, about 30 instruments. One third of them are subscribed to tumblers. All this on FORTS. Simulate under such conditions.

 
Dmi3:


Colleagues,

it's time for you to get out of the aircraft modelling circle.

Here are the combat conditions: 4 terminals, around 300 EAs, around 30 instruments. One third of the EAs are subscribed to tumblers. All this on FORTS. Simulate under such conditions.

"Here you go" is accepted as a zip file, plus a detailed description of the problem. Otherwise it's an empty conversation.

What is being discussed here is the code of the EA submitted and the effectiveness of its execution. Based on the problems identified, work has been done to optimise the terminal code.

 
Anton:

"Here you go" is accepted as a zip file, plus a detailed description of the problem. Otherwise it is an empty conversation.

In this case, the discussion is about the code submitted by the EA and the efficiency of its execution. The terminal code has been optimised for the problems identified.

I don't have any problems, there is nothing to send.

fxsaber has problems. He already wrote 16 pages here.

And Mikhail has had the same problems since 2014, he has already written 149 pages: https://www.mql5.com/ru/forum/38456/page149.

They are both qualified enough to give you all the information you need.

ФОРТС. Вопросы по исполнению
ФОРТС. Вопросы по исполнению
  • 2020.08.20
  • www.mql5.com
С большими проблемами удалось это сделать (начальник отдела по работе с профессиональными клиентами ДЦ Открытие Евгений Сергеевич,.
 
Anton:

This question is easy to answer.

This is where you copy a lot of data:

You are actually giving a command to take all available trading history from the terminal database into the EA environment, for later use. Purposely randomising trying to disrupt the possible caching algorithm of identical requests.

You have not followed the chronology of development of this thread, so you allow yourself accusatory notes in your statements.

I removed the MathRand line. Here's a brief log.

2020.08.27 22:38:57.920 Alert: Time[Test6.mq5 40: SymbolInfoDouble(Symb,SYMBOL_TRADE_TICK_VALUE)] = 3 ms.
2020.08.27 22:38:57.923 Alert: Time[Test6.mq5 19: CopyTicksRange(Symb,Ticks,COPY_TICKS_ALL,Tick.time_msc)] = 1 ms.
2020.08.27 22:38:57.926 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 3 ms.
2020.08.27 22:38:57.928 Alert: Time[Test6.mq5 61: AccountInfoDouble(ACCOUNT_EQUITY)] = 1 ms.
2020.08.27 22:38:57.940 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 11 ms.
2020.08.27 22:39:02.227 Alert: Time[Test6.mq5 17: CopyTicks(Symb,Ticks,COPY_TICKS_ALL,0,1)] = 1 ms.
2020.08.27 22:39:02.229 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:39:02.238 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 7 ms.
2020.08.27 22:39:02.241 Alert: Time[Test6.mq5 28: HistoryDealSelect(0)] = 1 ms.
2020.08.27 22:40:34.575 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:40:44.407 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 1 ms.
2020.08.27 22:40:44.409 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 9 ms.
2020.08.27 22:40:46.819 Alert: Time[Test6.mq5 40: SymbolInfoDouble(Symb,SYMBOL_TRADE_TICK_VALUE)] = 1 ms.
2020.08.27 22:40:52.760 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 3 ms.
2020.08.27 22:40:52.764 Alert: Time[Test6.mq5 28: HistoryDealSelect(0)] = 3 ms.
2020.08.27 22:40:54.907 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 3 ms.
2020.08.27 22:41:11.415 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 5 ms.
2020.08.27 22:41:13.517 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 1 ms.
2020.08.27 22:41:14.689 Alert: Time[Test6.mq5 18: CopyTicks(Symb,Ticks,COPY_TICKS_ALL,Tick.time_msc)] = 1 ms.
2020.08.27 22:41:45.343 Alert: Time[Test6.mq5 19: CopyTicksRange(Symb,Ticks,COPY_TICKS_ALL,Tick.time_msc)] = 1 ms.
2020.08.27 22:42:12.156 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 3 ms.
2020.08.27 22:42:19.654 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:42:32.581 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:42:48.662 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 1 ms.
2020.08.27 22:42:49.754 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:42:49.756 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:42:50.803 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 1 ms.
2020.08.27 22:42:50.804 Alert: Time[Test6.mq5 32: HistoryOrderGetInteger(0,ORDER_MAGIC)] = 1 ms.
2020.08.27 22:42:50.805 Alert: Time[Test6.mq5 44: TimeCurrent()] = 1 ms.
2020.08.27 22:43:42.143 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 6 ms.
2020.08.27 22:43:42.148 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 4 ms.
2020.08.27 22:43:54.985 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.
2020.08.27 22:44:01.402 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 1 ms.
2020.08.27 22:44:01.405 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 2 ms.


But you don't use all this data, you immediately dump it in the next line:

Obviously the terminal base here is a shared resource with synchronised access. And you have deliberately created thousands of orders and deals in it.

I test it on real accounts where orders over 10K are the norm. These are not fake orders as >70% of them were executed.

On the screenshot, by the way, 9331+576 != 12529.

All this nonsensical action is repeated 10 times on every tick from several threads at once. And you are deliberately making these actions from multiple threads occur simultaneously.

I'm having problems on different characters. A single symbol is suggested to reproduce the problem more quickly.

Repeating 10 times on each tick is a vital necessity. Since it is normal for one EA to contain a dozen TCs with different majors.

So you know very well what you are doing and why, and where the resources go, and yet you claim that "The latency is due to excessive load of CPU on the part of MT5".

Having said that, you clearly have a problem with your computer. I mean, yes, you are actively moving significant amounts of memory, but it should not affect the execution time of functions, especially not related to HistorySelect() in any way.

I cannot accuse you of incompetence, but what you have written, to put it mildly, causes bewilderment. HistorySelect is the finding of four indexes (beginning/end for the table of orders and beginning/end for the table of deals). The tables are sorted by time, so there is (should be) a binary search at worst. For 10K orders it is instantaneous (calculate the binary logarithm). What memory volume movement! Nobody here is talking about the dreaded HistorySelectByPosition. The elementary HistorySelect is affected.

On our tests b2582, even with 1000 times per tick and 5 EAs on one character charts, i.e. orders of magnitude greater than your default conditions, not a single Alert is observed.

Our test system: Windows 10 build 18363, Intel Xeon E5-2630 v4 @ 2.20GHz

Please provide login details for the trading account on which the tests were conducted here.