Transactional asynchrony is a problem in the terminal - General

Stanislav Korotky 2020.06.14 10:54 #27771

Alain Verleyen:

Another Terminal, but 20 charts (different symbols), started at the same time.

I checked on AUDUSD where the peak comes in:

2020.06.13 09: 16: 35.151 342152 (USDJPY, H1) Quantity = 240
2020.06.13 09: 16: 35.151 342152 (USDJPY, H1) Min = 38
2020.06.13 09: 16: 35.151 342152 (USDJPY, H1) Max = 2235
2020.06.13 09: 16: 35.151 342152 (USDJPY, H1) Avg = 472
2020.06.13 09: 16: 35.151 342152 (CADCHF, H1) Avg = 240
2020.06.13 09: 16: 35.151 342152 (CADCHF, H1) Min = 48
2020.06.13 09: 16: 35.151 342152 (CADCHF, H1) Max = 2323
2020.06.06.13 09: 16: 35.152 342152 (CADCHF, H1) Avg = 606
2020.06.13 09: 16: 35.152 342152 (EURAUD, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (EURCHF, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (GBPCHF, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (EURCHF, H1) Min = 141
2020.06.13 09: 16: 35.152 342152 (EURGBP, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (CHFJPY, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (AUDJPY, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (AUDJPY, H1) Min = 47
2020.06.13 09: 16: 35.152 342152 (EURUSD, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (EURUSD, H1) Min. = 23
2020.06.13 09: 16: 35.152 342152 (EURUSD, H1) Max = 2368
2020.06.13 09: 16: 35.152 342152 (AUDUSD, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (EURUSD, H1) Avg = 607
2020.06.13 09: 16: 35.152 342152 (AUDUSD, H1) Min = 74
2020.06.13 09: 16: 35.152 342152 (GBPJPY, H1) Avg = 240
2020.06.13 09: 16: 35.152 342152 (AUDNZD, H1) Avg = 240
2020.06.13 09: 16: 35.152 342152 (USDCHF, H1) Avg = 240
2020.06.06.13 09: 16: 35.152 342152 (AUDNZD, H1) Min = 122
2020.06.13 09: 16: 35.152 342152 (AUDCHF, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (AUDNZD, H1) Max = 2402
2020.06.13 09: 16: 35.152 342152 (AUDNZD, H1) Avg = 660
2020.06.06.13 09: 16: 35.152 342152 (AUDCHF, H1) Min = 157
2020.06.13 09: 16: 35.152 342152 (AUDCHF, H1) Max = 2354
2020.06.13 09: 16: 35.152 342152 (AUDCHF, H1) Avg = 663
2020.06.13 09: 16: 35.152 342152 (GBPCHF, H1) Min = 32
2020.06.06.13 09: 16: 35.152 342152 (GBPCHF, H1) Max = 2380
2020.06.13 09: 16: 35.152 342152 (GBPCHF, H1) Avg = 616
2020.06.13 09: 16: 35.152 342152 (EURCHF, H1) Max = 2327
2020.06.13 09: 16: 35.152 342152 (EURCHF, H1) Avg = 618
2020.06.13 09: 16: 35.152 342152 (EURGBP, H1) Min = 36
2020.06.13 09: 16: 35.152 342152 (EURGBP, H1) Max = 2233
2020.06.13 09: 16: 35.152 342152 (EURGBP, H1) Avg = 503
2020.06.13 09: 16: 35.152 342152 (CHFJPY, H1) Min = 11
2020.06.13 09: 16: 35.152 342152 (EURGBP, H1)13 09: 16: 35.152 342152 (CHFJPY, H1) Max = 2387
2020.06.13 09: 16: 35.152 342152 (CHFJPY, H1) Avg = 657
2020.06.13 09: 16: 35.152 342152 (AUDJPY, H1) Max = 2292
2020.06.13 09: 16: 35.152 342152 (AUDJPY, H1) Avg = 612
2020.06.13 09: 16: 35.152 342152 (EURNZD, H1) Quantity = 240
2020.06.13 09: 16: 35.152 342152 (EURNZD, H1) Min = 54
2020.06.13 09: 16: 35.152 342152 (EURNZD, H1) Max = 2361
2020.06.13 09: 16: 35.152 342152 (EURNZD, H1) Avg = 612
2020.06.06.13 09: 16: 35.152 342152 (AUDCAD, H1) Avg = 240
2020.06.13 09: 16: 35.152 342152 (AUDCAD, H1) Min = 121
2020.06.13 09: 16: 35.152 342152 (EURCAD, H1) Avg = 240
2020.06.13 09: 16: 35.152 342152 (AUDCAD, H1) Max = 2328
2020.06.13 09: 16: 35.152 342152 (AUDCAD, H1) Avg = 605
2020.06.13 09: 16: 35.152 342152 (AUDUSD, H1) Max = 2361
2020.06.13 09: 16: 35.152 342152 (AUDUSD, H1) Avg = 611
2020.06.13 09: 16: 35.152 342152 (GBPJPY, H1) Min = 13
2020.06.13 09: 16: 35.152 342152 (GBPJPY, H1) Max = 2411
2020.06.13 09: 16: 35.152 342152 (GBPJPY, H1) Avg = 661
2020.06.13 09: 16: 35.153 342152 (GBPUSD, H1) Avg = 240
2020.06.13 09: 16: 35.13 09: 16: 35.153 342152 (GBPUSD, H1) Min = 113
2020.06.13 09: 16: 35.153 342152 (GBPUSD, H1) Max = 2362
2020.06.13 09: 16: 35.153 342152 (GBPUSD, H1) Avg = 667
2020.06.13 09: 16: 35.153 342152 (USDCAD, H1) Quantity = 240
2020.06.06.13 09: 16: 35.153 342152 (USDCAD, H1) Min = 151
2020.06.13 09: 16: 35.153 342152 (USDCAD, H1) Max = 2386
2020.06.13 09: 16: 35.153 342152 (EURAUD, H1) Min = 35
2020.06.13 09: 16: 35.153 342152 (USDCAD, H1) Avg = 671
2020.06.13 09: 16: 35.153 342152 (EURAUD, H1) Max = 2233
2020.06.13 09: 16: 35.153 342152 (EURAUD, H1) Avg = 478
2020.06.13 09: 16: 35.153 342152 (EURJPY, H1) Quantity = 240
2020.06.13 09: 16: 35.153 342152 (EURJPY, H1) Min = 10
2020.06.06.13 09: 16: 35.153 342152 (EURCAD, H1) Min = 123
2020.06.13 09: 16: 35.153 342152 (EURCAD, H1) Max = 2384
2020.06.13 09: 16: 35.153 342152 (EURCAD, H1) Avg = 658
2020.06.13 09: 16: 35.153 342152 (USDCHF, H1) Min = 32
2020.06.06.13 09: 16: 35.153 342152 (USDCHF, H1) Max = 2369
2020.06.13 09: 16: 35.153 342152 (USDCHF, H1) Avg = 614
2020.06.13 09: 16: 35.153 342152 (EURJPY, H1) Max = 2344
2020.06.13 09: 16: 35.153 342152 (EURJPY, H1) Avg = 658

And this is when the tops come in:

2020.06.13 09: 17: 35.151 342152 (USDJPY, H1) Avg = 480
2020.06.13 09: 17: 35.151 342152 (USDJPY, H1) Min = 38
2020.06.13 09: 17: 35.151 342152 (USDJPY, H1) Max = 17421
2020.06.06.13 09: 17: 35.151 342152 (USDJPY, H1) Avg = 533
2020.06.13 09: 17: 35.151 342152 (EURAUD, H1) Quantity = 480
2020.06.13 09: 17: 35.151 342152 (EURAUD, H1) Min = 35
2020.06.06.13 09: 17: 35.151 342152 (EURAUD, H1) Max = 17406
2020.06.13 09: 17: 35.151 342152 (EURAUD, H1) Avg = 518
2020.06.13 09: 17: 35.151 342152 (EURGBP, H1) Quantity = 480
2020.06.13 09: 17: 35.151 342152 (EURGBP, H1) Min = 36
2020.06.13 09: 17: 35.151 342152 (EURGBP, H1) Max = 17425
2020.06.13 09: 17: 35.151 342152 (EURGBP, H1) Avg = 515
2020.06.13 09: 17: 35.151 342152 (EURUSD, H1) Quantity = 480
2020.06.13 09: 17: 35.151 342152 (EURUSD, H1) Min = 18
2020.06.13 09: 17: 35.151 342152 (EURUSD, H1) Max = 17366
2020.06.13 09: 17: 35.151 342152 (EURCHF, H1) Max = 480
2020.06.13 09: 17: 35.151 342152 (EURUSD, H1) Avg = 618
2020.06.13 09: 17: 35.151 342152 (AUDCHF, H1) Max = 480
2020.06.13 09: 17: 35.151 342152 (USDCHF, H1) Quantity = 480
2020.06.13 09: 17: 35.151 342152 (AUDCHF, H1) Min = 126
2020.06.13 09: 17: 35.151 342152 (CADCHF, H1) Quantity = 480
2020.06.13 09: 17: 35.151 342152 (AUDCHF, H1) Max = 10477
2020.06.13 09: 17: 35.151 342152 (EURCAD, H1) Quantity = 480
2020.06.13 09: 17: 35.151 342152 (GBPUSD, H1) Quantity = 480
2020.06.13 09: 17: 35.152 342152 (EURCAD, H1) Min = 123
2020.06.13 09: 17: 35.152 342152 (GBPUSD, H1) Min = 112
2020.06.13 09: 17: 35.152 342152 (EURCAD, H1) Max = 10485
2020.06.13 09: 17: 35.152 342152 (EURCAD, H1) Avg = 663
2020.06.13 09: 17: 35.152 342152 (GBPUSD, H1) Max = 10435
2020.06.13 09: 17: 35.152 342152 (GBPUSD, H1) Avg = 661
2020.06.13 09: 17: 35.152 342152 (EURCHF, H1) Min = 71
2020.06.06.13 09: 17: 35.152 342152 (EURJPY, H1) Avg = 480
2020.06.13 09: 17: 35.152 342152 (USDCAD, H1) Avg = 480
2020.06.13 09: 17: 35.152 342152 (USDCAD, H1) Min = 54
2020.06.13 09: 17: 35.152 342152 (EURJPY, H1) Min = 10
2020.06.13 09: 17: 35.152 342152 (EURJPY, H1) Max = 10487
2020.06.13 09: 17: 35.152 342152 (EURJPY, H1) Avg = 664
2020.06.13 09: 17: 35.152 342152 (EURJPY, H1) Avg = 664.13 09: 17: 35.152 342152 (EURNZD, H1) Quantity = 480
2020.06.13 09: 17: 35.152 342152 (EURNZD, H1) Min = 54
2020.06.13 09: 17: 35.152 342152 (EURNZD, H1) Max = 17426
2020.06.06.13 09: 17: 35.152 342152 (EURNZD, H1) Avg = 620
2020.06.13 09: 17: 35.152 342152 (USDCHF, H1) Min = 32
2020.06.13 09: 17: 35.152 342152 (USDCHF, H1) Max = 17427
2020.06.13 09: 17: 35.152 342152 (USDCHF, H1) Avg = 649
2020.06.13 09: 17: 35.152 342152 (GBPCHF, H1) Max = 480
2020.06.13 09: 17: 35.152 342152 (GBPCHF, H1) Min = 32
2020.06.13 09: 17: 35.152 342152 (GBPCHF, H1) Max = 17433
2020.06.13 09: 17: 35.152 342152 (GBPCHF, H1) Avg = 647
2020.06.13 09: 17: 35.152 342152 (AUDJPY, H1) Min = 480
2020.06.13 09: 17: 35.152 342152 (AUDJPY, H1) Min. = 47
2020.06.13 09: 17: 35.152 342152 (AUDJPY, H1) Max = 17415
2020.06.13 09: 17: 35.152 342152 (AUDJPY, H1) Avg = 640
2020.06.13 09: 17: 35.152 342152 (CADCHF, H1) Min = 48
2020.06.13 09: 17: 35.152 342152 (CADCHF, H1) Max = 17435
2020.06.13 09: 17: 35.152 342152 (CADCHF, H1) Avg = 637
2020.06.13 09: 17: 35.152 342152 (AUDCHF, H1) Avg = 662
2020.06.13 09: 17: 35.152 342152 (AUDUSD, H1) Quantity = 480
2020.06.13 09: 17: 35.152 342152 (AUDUSD, H1) Min = 43
2020.06.13 09: 17: 35.152 342152 (AUDUSD, H1) Max = 17416
2020.06.13 09: 17: 35.152 342152 (AUDUSD, H1) Avg = 620
2020.06.13 09: 17: 35.152 342152 (GBPJPY, H1) Quantity = 480
2020.06.13 09: 17: 35.13 09: 17: 35.152 342152 (GBPJPY, H1) Min = 13
2020.06.13 09: 17: 35.152 342152 (GBPJPY, H1) Max = 10468
2020.06.13 09: 17: 35.152 342152 (GBPJPY, H1) Avg = 660
2020.06.13 09: 17: 35.152 342152 (CHFJPY, H1) Quantity = 480
2020.06.06.13 09: 17: 35.152 342152 (CHFJPY, H1) Min = 11
2020.06.13 09: 17: 35.152 342152 (CHFJPY, H1) Max = 10474
2020.06.13 09: 17: 35.152 342152 (AUDNZD, H1) Quantity = 480
2020.06.13 09: 17: 35.152 342152 (AUDNZD, H1) Min = 61
2020.06.13 09: 17: 35.152 342152 (AUDNZD, H1) Max = 10491
2020.06.13 09: 17: 35.152 342152 (AUDNZD, H1) Avg = 663
2020.06.13 09: 17: 35.152 342152 (EURCHF, H1) Max = 17414
2020.06.13 09: 17: 35.152 342152 (EURCHF, H1) Avg = 646
2020.06.13 09: 17: 35.152 342152 (AUDCAD, H1) Max = 480
2020.06.13 09: 17: 35.152 342152 (AUDCAD, H1) Min = 100
2020.06.13 09: 17: 35.152 342152 (AUDCAD, H1) Max = 17421
2020.06.06.13 09: 17: 35.152 342152 (AUDCAD, H1) Avg = 638
2020.06.13 09: 17: 35.152 342152 (USDCAD, H1) Max = 10488
2020.06.13 09: 17: 35.152 342152 (USDCAD, H1) Avg = 666
2020.06.13 09: 17: 35.152 342152 (CHFJPY, H1) Avg = 656

So it all happened at the same time. But this could still be an MT5 problem. I will check with other simultaneous terminals.

Judging by the logs, the deceleration in one did not coincide with the deceleration in the other, i.e. not simultaneously. So the problem is in the terminal itself.

Renat Fatkhullin 2020.06.14 11:40 #27772

MQL5 programs work (Get/Set methods) with a chart via a transaction queue.

This allows to release the GUI and the terminal itself from the inevitable blockages that would be a problem for MQL5 programs.

Transactional asynchrony allows for quick writing or reading in separated modes and enables the synchronization mode immediately when mixing Set and Get methods.

That is, it's better to do asynchronous Set 1000 times and then Get 1000 times instead of doing alternate Get & Set, turning the queue into a synchronous process. Because you have to make sure that the previous asynchronous Set was exactly overlapped and now you can read it.

You need to use system functions carefully and cache them whenever possible.

Error number 6 Question about the OnTradeTransaction Any questions from newcomers

Sergey Dzyublik 2020.06.14 11:59 #27773

Renat Fatkhullin:

You need to use system functions carefully and cache them where possible.

Afternoon, the problem is a bit different - the ChartGetInteger and similar functions are VERY slow to execute.
When switching from build 2009 to build 2485, ChartGetInteger runtime increased from 5ms to 200-250ms, this problem is especially noticeable when 50+ charts are open.
System: Terminal Windows 10 build 18363, Intel Core i7-7700HQ @ 2.80GHz, 19 / 31 GB memory, 262 / 640 GB disk, 4K monitor, NVidia 1050Ti
Thecode from the problem description was used:https://www.mql5.com/en/forum/342152

Possible causes of the problem:

Bugs, bugs, issues

Sergey Dzyublik, 2020.06.13 19:20

I have compared the implementation of ChartGetInteger function for two versions of MT5 2009 and MT5 2485, perhaps the problem is as follows:
1. In 2485 for reading "atomic" fields of a chart object we use rather slow operations:
mfence; lock mov eax,[rax+2C];
While in 2009 build this is done using: lock xadd [rcx+2C],eax

2. It also seems that the logic and possible dwell time in ntdll_RtlEnterCriticalSection have significantly changed.
Previously, in 2009, a critical section would only check a pair of received values, without any atomic operations.
And in 2485, the linked-list objects of the chart could additionally be enumerated.

Presumably, the problem could happen when crash was fixed when working with chart functions as part of migration to the new compiler (about 2-3 months ago).
Assembler code for ChartGetInteger call in MT5 (build 2485) is attached.

New MetaTrader 5 Platform New MetaTrader 5 platform Key obstacle affecting the

Renat Fatkhullin 2020.06.14 12:05 #27774

Let's check it out.

Sergey Dzyublik 2020.06.14 12:59 #27775

Renat Fatkhullin:
Check it out.

Steps for playback:

long chart_id;

int OnInit(){
   for(int i = 0; i < 95; ++i){
      ChartOpen(_Symbol, _Period);
   }

   chart_id = ChartID();
   EventSetMillisecondTimer(250);
   return(INIT_SUCCEEDED);
}

void OnTimer(){
  ulong t=GetMicrosecondCount();
  long autoscroll = ChartGetInteger(chart_id, CHART_AUTOSCROLL);
  ulong delay =(GetMicrosecondCount()-t)/1000;
  if (delay>0){
    Print("Execution delay: ",delay, " ms");
  }
}

1. Take a clean MT terminal, open one chart on it, run the compiled EA above on that chart.
2. After opening all 95 new charts, expand the chart window to the full width of the chart space in MT, if this has not been done before.
3. To move from a chart tab to another and record from the log the ChartGetInteger runtime values.
To close all open charts, you can press and hold CTRL+W

Result of MT5 (build 2009):

2020.06.14 15:33:43.173 Test123456 (EURUSD,H1)  Execution delay: 2 ms
2020.06.14 15:33:43.674 Test123456 (EURUSD,H1)  Execution delay: 3 ms
2020.06.14 15:33:44.177 Test123456 (EURUSD,H1)  Execution delay: 5 ms
2020.06.14 15:33:44.422 Test123456 (EURUSD,H1)  Execution delay: 4 ms
2020.06.14 15:33:44.673 Test123456 (EURUSD,H1)  Execution delay: 2 ms
2020.06.14 15:33:44.923 Test123456 (EURUSD,H1)  Execution delay: 3 ms
2020.06.14 15:33:45.173 Test123456 (EURUSD,H1)  Execution delay: 2 ms
2020.06.14 15:33:45.423 Test123456 (EURUSD,H1)  Execution delay: 2 ms
2020.06.14 15:33:45.672 Test123456 (EURUSD,H1)  Execution delay: 2 ms

MT5 result (buidl 2485):

2020.06.14 15:33:12.947 Test123456 (EURUSD,H1)  Execution delay: 163 ms
2020.06.14 15:33:13.198 Test123456 (EURUSD,H1)  Execution delay: 1 ms
2020.06.14 15:33:13.293 Test123456 (EURUSD,H1)  Execution delay: 1 ms
2020.06.14 15:33:13.831 Test123456 (EURUSD,H1)  Execution delay: 287 ms
2020.06.14 15:33:14.702 Test123456 (EURUSD,H1)  Execution delay: 136 ms
2020.06.14 15:33:15.316 Test123456 (EURUSD,H1)  Execution delay: 3 ms
2020.06.14 15:33:15.677 Test123456 (EURUSD,H1)  Execution delay: 110 ms
2020.06.14 15:33:17.041 Test123456 (EURUSD,H1)  Execution delay: 221 ms

Comparison of results and conclusions:
1. The number of records which are displayed in 2009 build is much more than records in 2485 build.
The ChartGetInteger function under "normal conditions" got faster after switching to build 2485.

2. The peak run time for 2009 build - 15 ms, and for 2485 build - 310 ms.
The ChartGetInteger function in "unfavorable conditions" becomes up to 20 times slower after switching to build 2485.

3. the same way you can estimate opening speed of 95 charts by eye.
For both builds, the "exponential" complexity of the number of charts opened earlier is noticeable, as well as significantly faster execution in 2009 build.

Execution speed faster up New MetaTrader 4 Platform MT5 and speed in

Alain Verleyen 2020.06.14 13:03 #27776

Stanislav Korotky :

Judging from the logs, the deceleration in one did not coincide with the deceleration in the other, i.e. not simultaneously. So the problem is in the terminal itself.

The log is only printed every minute (translation messed up the time stamp !!), I could check more accurately, but it's not worth it.

I've tried it on several terminals and it clearly shows that the peaks are NOT simultaneous. This is clearly an MT5 thing.

Questions from a "dummy" How can I protect Extracting Double Value from

Alain Verleyen 2020.06.14 13:05 #27777

Renat Fatkhullin :

MQL5 programs work (Get/Set methods) with a chart via a transaction queue.

This allows to release the GUI and the terminal itself from the inevitable blockages that would be a problem for MQL5 programs.

Transactional asynchrony allows for quick writing or reading in separated modes and enables the synchronization mode immediately when mixing Set and Get methods.

That is, it is better to do asynchronous Set 1000 times and then Get 1000 times instead of doing alternate Get & Set, turning the queue into a synchronous process. Because you have to make sure that the previous asynchronous Set was exactly overlapped and now you can read it.

You need to use system functions carefully and cache them whenever possible.

It's an understandable and good system. But I think there is a glitch somewhere, as @Sergey Dzyubli also demonstrated.

Sergey Dzyublik

www.mql5.com

Добавил тему Вход через MQL5.community (OAuth) В настройках профиля появилась новая вкладка Приложения (https://www.mql5.com/ru/users/USER_NAME/apps): Эта функция позволяет людям авторизоваться на вашем сайте или в приложении, используя аккаунт MQL5.community. Технология проста и безопасна. Она Добавил...

Nikolai Semko 2020.06.14 13:09 #27778

Alexey Navoykov:
As far as I know ChartRedraw call doesn't cause immediate redrawing of chart. Redrawing only happens when some Get-method is called.

And ChartRedraw is essentially the same asynchronous method. So your BuildChart measurement is not reliable.

try to swap these two lines

then there won't be anything asynchronous in the measurement and see what happens. It will be even faster.

Andrei Trukhanovich 2020.06.14 13:11 #27779

fxsaber:
Been dumb for a long time, not understanding what the compiler doesn't like about this line.

Forgot to write if. Thought it would be a good idea to chew up the message for such dumbasses.

Asked me to make a description and example for all compiler errors and warnings about 5 years ago, maybe more.

Maybe you can do better.

Nikolai Semko 2020.06.14 13:52 #27780

Renat Fatkhullin:

MQL5 programs work (Get/Set methods) with a chart via a transaction queue.

This allows to release the GUI and the terminal itself from the inevitable blockages that would be a problem for MQL5 programs.

Transactional asynchrony allows for quick writing or reading in separated modes and enables the synchronization mode immediately when mixing Set and Get methods.

That is, it is better to do asynchronous Set 1000 times and then Get 1000 times instead of doing alternate Get & Set, turning the queue into a synchronous process. Because you need to make sure that the previous asynchronous Set was exactly overlapped and now you can read it.

You need to use system functions carefully and cache them whenever possible.

I understand correctly that not only Set methods are asynchronous but Get as well?
Ilyas was wrong here , wasn't he?
And Slava was right here when he said that ChartXYToTimePrice method is asynchronous? After all, the ChartXYToTimePrice method most likely refers to the Get methods.

The documentation speaks only about asynchronicity of Set methods.

My EA does a Fractals, fractal structures, their Not the Grail, just

Errors, bugs, questions - page 2778