MT5 and speed in action - page 11

New comment
 
Once upon a time I wrote a primitive lag indicator. It does a great job of showing a massive torrent pump, for example.
Ping
Ping
  • www.mql5.com
Для торговли важным параметром является актуальность текущей цены. На него влияет множество факторов, самый популярный из которых - сетевой пинг между терминалом и торговым сервером. Но часто из виду упускается другой параметр: так называемый "внутренний пинг терминала" - дополнительный лаг котировок внутри самого терминала (платформы) . Даже...
 
fxsaber:

Misunderstood. We need to understand if MT is jamming or not. We were parsing the server logs, when the pauses were modifying for dozens of seconds. Everything is perfect on the server.

So the problem is somewhere else.

I have seen that I very often hit past prices. I.e. serious lag in Terminal. There is a question how to identify it.

Here I have bought VPS. How do I know that it fits perfectly? I mean, there has to be some kind of a numerical indicator. As it turns out, retransmits can't be trusted. At midnight - tens of percent.


Maybe I can deploy a virtual machine on the same physical machine as the trading server. And I would be able to run the Terminal from there. I guess that would reduce the number of deadlocks.

It seems to me that tracing, pinging from the console or special software won't solve the problem.
It is necessary to register MT involvement somehow. It seems to me that we should take a stand and ask the developers to make their own counter forTERMINAL_RETRANSMISSION

 
fxsaber:

Misunderstood. We need to understand if MT is jamming or not. We were parsing the server logs, when the pauses were modifying for dozens of seconds. Everything is perfect on the server.
So the problem is somewhere else.
I have seen that I very often hit past prices. I.e. serious lag in Terminal. The question arose as to how to identify it.

Somehow it's not very clear what exactly "everything is perfect on the server" means:
1) No modification requests came to the server, or all requests within 10 seconds had an outdated price?
2) And how many seconds is the difference between the "current" and "obsolete" price?
3) Who determines that the price is out of date: the MT server or a third party, maybe some exchange?


If the problem is frequent, you could analyse order modification traffic using Wireshark paired with Process Monitor (Network Activity) from Sysinternals.

 
Sergey Dzyublik:

Somehow it is not very clear what exactly "everything is perfect on the server" means:
1) No modification requests came to the server, or all requests within 10 seconds had an out-of-date price?
2) And how many seconds is the difference between the "current" and "obsolete" price?
3) Who determines that the price is out of date: the MT server or a third party, maybe some exchange?

On the server, all requests were processed in units of milliseconds. About the outdated prices, you need to check with the admin.

If the problem is recurring, you could analyze the order modification traffic using Wireshark paired with Process Monitor (Network Activity) from Sysinternals.

I am a complete novice at this.

 

It was possible to reproduce HistorySelect-functions brakes.

  1. Go to a MetaQuotes-Demo account with a relatively long trading history.
    1. If your demo account has little trading history, enable auto-trading and run the OrderSend-Test2.ex5 script in parallel (see the source code in the attachment).
  2. Open two EURUSD charts and run the Expert Advisor below on each(yellow in the source code shows additional changes).

    Forum on trading, automated trading systems and testing trading strategies

    MT5 and Speed in Action

    fxsaber, 2020.06.04 14:38

    I drew such an EA.

    // Мониторинг длительных пиков выполнения важных функций для торговли.
#include <fxsaber\Benchmark.mqh> // https://c.mql5.com/3/321/Benchmark.mqh

input int inCycle = 10;    // Циклов проверки в одном OnTick
input int inAlertTime = 1; // Нижний порог в миллисекундах

#define _B2(A) _B(A, AlertTime)

void Check( const string Symb, const int AlertTime = 1 )
{
  MqlTick Tick;
  
  if (_B2(SymbolInfoTick(Symb, Tick)))
  {
    MqlTick Ticks[];
    
    _B2(CopyTicks(Symb, Ticks, COPY_TICKS_ALL, 0, 1));
    _B2(CopyTicks(Symb, Ticks, COPY_TICKS_ALL, Tick.time_msc));
    _B2(CopyTicksRange(Symb, Ticks, COPY_TICKS_ALL, Tick.time_msc));

    HistorySelect(MathRand(), INT_MAX);
    _B2(HistorySelect(Tick.time, INT_MAX));
    
    _B2(HistoryDealsTotal());
    _B2(HistoryDealGetTicket(0));
    _B2(HistoryDealGetInteger(0, DEAL_MAGIC));
    _B2(HistoryDealGetDouble(0, DEAL_PRICE));
    _B2(HistoryDealSelect(0));

    _B2(HistoryOrdersTotal());
    _B2(HistoryOrderGetTicket(0));
    _B2(HistoryOrderGetInteger(0, ORDER_MAGIC));
    _B2(HistoryOrderGetDouble(0, ORDER_PRICE_CURRENT));
    _B2(HistoryOrderSelect(0));
    
    _B2(GetLastError());
    _B2(IsStopped());
    
    _B2(SymbolInfoDouble(Symb, SYMBOL_ASK));
    _B2(SymbolInfoDouble(Symb, SYMBOL_TRADE_TICK_VALUE));
    _B2(SymbolInfoDouble(Symb, SYMBOL_POINT));
    _B2(SymbolInfoInteger(Symb, SYMBOL_DIGITS));

    _B2(TimeCurrent());
    _B2(TimeLocal());
    _B2(TimeTradeServer());
    
    _B2(OrdersTotal());
    _B2(OrderSelect(0));
    _B2(OrderGetDouble(ORDER_PRICE_CURRENT));
    _B2(OrderGetInteger(ORDER_MAGIC));
    _B2(OrderGetString(ORDER_SYMBOL));
    
    _B2(PositionsTotal());
    _B2(PositionSelect(Symb));
    _B2(PositionSelectByTicket(0));
    _B2(PositionGetDouble(POSITION_PRICE_CURRENT));
    _B2(PositionGetInteger(POSITION_MAGIC));
    _B2(PositionGetString(POSITION_SYMBOL));
    
    _B2(AccountInfoDouble(ACCOUNT_EQUITY));
    _B2(AccountInfoInteger(ACCOUNT_MARGIN_MODE));
    
    MqlTradeRequest Request = {0};
    MqlTradeCheckResult CheckResult;

    _B2(OrderCheck(Request, CheckResult));
    
    _B2(MQLInfoInteger(MQL_TRADE_ALLOWED));
    _B2(AccountInfoInteger(ACCOUNT_TRADE_EXPERT));
    _B2(AccountInfoInteger(ACCOUNT_TRADE_ALLOWED));
    _B2(TerminalInfoInteger(TERMINAL_TRADE_ALLOWED));
    
    _B2(SymbolsTotal(true));
    _B2(SymbolName(0, true));
    _B2(Symbol());
    
    _B2(GlobalVariableCheck(NULL));
    _B2(GlobalVariableGet(NULL));
    
    _B2(ResourceFree(NULL));
  }
}

void OnTick()
{
  for (int i = 0; i < inCycle; i++)
    Check(_Symbol, inAlertTime);      
}

MT5 b2572 is sending such messages. 

2020.08.13 05:28:00.143 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 6 ms.
2020.08.13 05:28:00.148 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 3 ms.
2020.08.13 05:28:00.153 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 4 ms.
2020.08.13 05:28:00.162 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 5 ms.
2020.08.13 05:28:00.167 Alert: Time[Test6.mq5 34: HistoryOrderSelect(0)] = 2 ms.
2020.08.13 05:28:00.174 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 4 ms.
2020.08.13 05:28:00.180 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 5 ms.
2020.08.13 05:28:00.186 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 4 ms.
2020.08.13 05:28:00.194 Alert: Time[Test6.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 6 ms.


Please write about your results, who will try it. What about the lags?


ZS Long tried to figure out why there are brakes on live accounts. I think I have found the problem area.

Files:
OrderSend-Test2.ex5  289 kb
 
fxsaber:

ZS Long tried to work out why the battle accounts were slow. Seems to have found the problem area.

This is how depressing the combat situation looks.

2020.08.13 10:20:09.074 Expert (EURAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 59 ms.
2020.08.13 10:20:09.074 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 59 ms.
2020.08.13 10:20:09.074 Expert (GBPNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 59 ms.
2020.08.13 10:20:09.074 Expert (GBPAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 59 ms.
...
2020.08.13 11:31:58.559 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 34 ms.
2020.08.13 11:31:58.559 Expert (GBPNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 35 ms.
2020.08.13 11:31:58.624 Expert (EURAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 99 ms.
2020.08.13 11:32:12.483 Expert (GBPNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 7 ms.
2020.08.13 11:33:44.877 Expert (EURAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 27 ms.
2020.08.13 11:33:44.877 Expert (GBPNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 21 ms.
2020.08.13 11:33:44.879 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 27 ms.
2020.08.13 11:33:47.911 Expert (GBPAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 18 ms.
2020.08.13 11:33:47.911 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 18 ms.
2020.08.13 11:33:47.912 Expert (GBPNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 29 ms.

2020.08.13 11:33:49.312 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 22 ms.
2020.08.13 11:33:49.312 Expert (EURAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 22 ms.

2020.08.13 11:34:02.612 Expert (EURAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 63 ms.
2020.08.13 11:34:02.613 Expert (GBPNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 63 ms.

2020.08.13 11:34:02.616 Expert (GBPAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 56 ms.
2020.08.13 11:34:12.057 Expert (GBPAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 12 ms.

2020.08.13 11:34:30.643 Expert (EURAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 32 ms.
2020.08.13 11:34:30.643 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 32 ms.

2020.08.13 11:34:30.643 Expert (GBPAUD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 8 ms.
2020.08.13 11:37:17.059 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 24 ms.
2020.08.13 11:38:30.360 Expert (EURNZD,M1)      Alert: Time[NewTicks.mqh 116: ::HistorySelect(TimeMsc/1000,INT_MAX)] = 13 ms.

If you look closely, you can see that EAs independent of each other are slowing down in the same way (lag time and duration). That is, when several Expert Advisors make HistorySelect call, then MT5 stops execution of all Expert Advisors for some time, and then returns its results to the Expert Advisors.


What is the reason for such a strange (in comparison with the impending) API for working with history? Where are the pitfalls in this approach?

Forum on trading, automated trading systems and strategy testing

MT5 and Speed in Action

fxsaber, 2020.05.28 15:27

HistorySelect.

This is an insanely expensive feature. And unfortunately no amount of caching can make its speed acceptable now.


Please consider introducing such history features.

HistoryDealsSelect( const int Index, const int Count = WHOLE_ARRAY );  // Из внутренней таблицы сделок взять сделки, начиная с заданного индекса в таблице.
HistoryOrdersSelect( const int Index, const int Count = WHOLE_ARRAY ); // Из внутренней таблицы ордеров взять ордера, начиная с заданного индекса в таблице.

They would completely close the HistorySelect brakes. Because it would solve the problem of getting the latest deals very cheaply. Right now it's one torment in combat execution.


It's not always possible to control the last deals via OnTradeTransaction. That's why a quick HistorySelect is relevant.


It's probably not normal for MT4 performance to outperform MT5 when dealing with trading history.

Please remove the brakes from the trading terminal!

 
Is 1,000 transactions in history enough?
 
Rorschach:
Is 1000 trades in history enough?

Tested with these readings.

        : HistoryDealsTotal() = 11045
        : HistoryOrdersTotal() = 11518
 

Orders 20990

Deals 10277

Log in 20 minutes

Files:
20200813.log  4817 kb
 
Rorschach:

Orders 20990

Deals 10277

Log in 20 mins.

Super, Thank you! Finally, a complete confirmation of the sad lags of MT5. And that's without any trading functions. Problems almost everywhere.

KD      0       16:00:33.382    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 34: HistoryOrderSelect(0)] = 25 ms.
PE      0       16:00:44.913    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 17: CopyTicks(Symb,Ticks,COPY_TICKS_ALL,0,1)] = 24 ms.
DP      0       16:00:44.888    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 22: HistorySelect(Tick.time,INT_MAX)] = 46 ms.
FI      0       16:00:49.579    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 28: HistoryDealSelect(0)] = 22 ms.
EE      0       16:01:03.287    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 33: HistoryOrderGetDouble(0,ORDER_PRICE_CURRENT)] = 1 ms.
KE      0       16:01:07.013    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 50: OrderGetDouble(ORDER_PRICE_CURRENT)] = 1 ms.
JM      0       16:01:12.189    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 44: TimeCurrent()] = 39 ms.
MD      0       16:01:13.067    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 81: ResourceFree(NULL)] = 1 ms.
RS      0       16:01:13.572    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 41: SymbolInfoDouble(Symb,SYMBOL_POINT)] = 7 ms.
GL      0       16:01:27.816    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 79: GlobalVariableGet(NULL)] = 22 ms.
PD      0       16:01:33.892    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 58: PositionGetInteger(POSITION_MAGIC)] = 1 ms.
KP      0       16:01:39.864    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 67: OrderCheck(Request,CheckResult)] = 3 ms.
ML      0       16:01:39.970    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 62: AccountInfoInteger(ACCOUNT_MARGIN_MODE)] = 1 ms.
KM      0       16:01:41.045    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 55: PositionSelect(Symb)] = 2 ms.
NS      0       16:01:46.832    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 78: GlobalVariableCheck(NULL)] = 1 ms.
JP      0       16:01:49.211    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 75: SymbolName(0,true)] = 1 ms.
EL      0       16:01:59.101    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 19: CopyTicksRange(Symb,Ticks,COPY_TICKS_ALL,Tick.time_msc)] = 32 ms.
IM      0       16:02:07.462    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 70: AccountInfoInteger(ACCOUNT_TRADE_EXPERT)] = 7 ms.
PJ      0       16:02:11.735    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 37: IsStopped()] = 4 ms.
OG      0       16:03:08.178    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 32: HistoryOrderGetInteger(0,ORDER_MAGIC)] = 14 ms.
JH      0       16:03:16.385    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 40: SymbolInfoDouble(Symb,SYMBOL_TRADE_TICK_VALUE)] = 5 ms.
FM      0       16:03:16.601    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 59: PositionGetString(POSITION_SYMBOL)] = 1 ms.
GH      0       16:03:21.841    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 72: TerminalInfoInteger(TERMINAL_TRADE_ALLOWED)] = 1 ms.
FJ      0       16:03:25.782    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 46: TimeTradeServer()] = 1 ms.
EO      0       16:03:26.772    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 45: TimeLocal()] = 10 ms.
HD      0       16:03:36.595    fxstest (EURUSD,M1)     Alert: Time[fxstest.mq5 13: SymbolInfoTick(Symb,Tick)] = 13 ms.
...

That's depressing. It would be interesting to see the result under Linux. Under Win there is no real time to talk about.

1...456789101112131415161718...94
New comment