MT5 and speed in action - page 80

[fxsaber]

fxsaber:

ZS It would be good to get rid of the hang-ups that have been going on for months. Run this script on a machine with infinite RAM. For example, I can't upload ticks from June 1st just one character at a time. It just hangs CopyTicks with zero resource consumption.

b2699 - fixed, Thanks.

[fxsaber]

To the piggy bank of codes that show the brakes. This time Tester.

[fxsaber]

Forum on trading, automated trading systems and testing trading strategies

Acceptance of SL/TP orders

fxsaber, 2020.12.11 09:17

// Измеряет размер лага между приходом тика на MT5-сервер и MT5-Терминал.
// Запускать на той же машине, на которой установлен MT5-сервер.

100 ticks were processed. Arrival lag between the server and the terminal varies from one to eight milliseconds. The average is a little over four milliseconds. This is just equal to the lag of TP order triggering, which is where this branch started.

The lag itself is inside the MT5 server. The Server->Terminal channel has nothing to do with it.

Big request to developers to eliminate this lag. Now with zero pings we have a constant delay of ticks incoming not only to the terminal, but to the Server as well. In particular, orders acceptance.

[fxsaber]

To reduce trading lags, I recommend transferring the battle terminal to RAM-drive.

[fxsaber]

Unexpectedly I came across a tick missing in the history, even though it came in Market Watch: SymbolInfoTick.

Printout of the same tick through MQL shows an interesting flag.

                         [time]     [bid]     [ask] [last] [volume]    [time_msc] [flags] [volume_real]
        [0] 2021.01.04 20:52:55 103.16300 103.16500 0.0000        0 1609793575267       4       0.00000
        [1] 2021.01.04 20:52:55 103.16300 103.16400 0.0000        0 1609793575788       4       0.00000
        [2] 2021.01.04 20:52:59 103.16400 103.16400 0.0000        0 1609793579367     130       0.00000
        [3] 2021.01.04 20:53:01 103.16400 103.16400 0.0000        0 1609793581817       2       0.00000
        [4] 2021.01.04 20:53:01 103.16300 103.16400 0.0000        0 1609793581969       2       0.00000

This flag was formed on the tick in the history just before the missing Market Watch tick. Perhaps this will tell us where the problem is.

ZS Unfortunately, this happens systematically. The tick history does not contain all ticks that come to the Terminal.

[Denis Sartakov]

fxsaber:

Unexpectedly I came across a tick missing in the history, even though it came in Market Watch: SymbolInfoTick.

Printout of the same tick through MQL shows an interesting flag.

This flag was formed on the tick in the history just before the missing Market Watch tick. Perhaps this will tell us where the problem is.

ZS Unfortunately, this happens systematically. The tick history does not contain all ticks that come to the Terminal.

It does. Let's say EA trades a whole day on a real account, make a profit.

the next day I run the tester on the previous day and get a loss.

I do not understand the reason, either the broker gives the wrong ticks or something else...

[fxsaber]

Forum on trading, automated trading systems and trading strategy testing

MT5 and Speed in Action

fxsaber, 2021.01.04 20:51

Unexpectedly I ran into a lack of tick in the history, despite the fact that it was coming to the Market Watch: SymbolInfoTick.

I have started a parallel terminal, where tkc was not generated by the Terminal, but uploaded from the Server.

In the screenshot this terminal on the left - tick is present. But on the other Terminal (on the right) - it is not!

It turns out that the Terminal itself does not put all incoming ticks into the tick history. If you want to have the history without omissions, you should add the tkc-file and pull it from the Server.

Unpleasant bug.

[fxsaber]

// Попытка поймать тик, который не попал в историю тиков.

// Сравнение двух тиков.
bool IsEqual( const MqlTick &Tick1, const MqlTick &Tick2 )
{
  return((Tick1.time_msc == Tick2.time_msc) &&
         !NormalizeDouble(Tick1.bid - Tick2.bid, _Digits) &&
         !NormalizeDouble(Tick1.ask - Tick2.ask, _Digits));
}

// Проверяет наличие тика в истории.
bool IsHistory( const MqlTick &Tick )
{
  bool Res = false;
  
  MqlTick Ticks[];
  
  const int Size = CopyTicksRange(_Symbol, Ticks, COPY_TICKS_ALL, Tick.time_msc, Tick.time_msc + 1);
  
  for (int i = 0; !Res && (i < Size); i++)
    Res = IsEqual(Tick, Ticks[i]);
    
  return(Res);
}

void OnTick()
{
  static MqlTick Ticks[];
  MqlTick Tick;
  
  if (SymbolInfoTick(_Symbol, Tick)) // Взяли текущий тик.
  {
    const int Size = ArrayResize(Ticks, ArraySize(Ticks) + 1);
    
    Ticks[Size - 1] = Tick; // Дописали его в массив
    
    MqlTick HistoryTick[1];
    
    if (CopyTicks(_Symbol, HistoryTick, COPY_TICKS_ALL, 0, 1) > 0) // Взяли последний исторический тик
    {
      int i = 0;
      
      while ((i < Size) && (Ticks[i].time_msc < HistoryTick[0].time_msc)) // Если исторический тик пришел позже проверяемого
      {
        if (!IsHistory(Ticks[i]))                                    // Если в истории тиков нет проверяемого тика,
        {
          Alert("!IsHistory(Ticks[i]) == true");
          ArrayPrint(Ticks, _Digits, NULL, i, 1, ARRAYPRINT_HEADER); // выводим его.
        }
          
        i++;
      }
      
      ArrayRemove(Ticks, 0, i); // Удалили тики, что проверили.
    }
  }  
}

Such an EA could not catch ticks missed in history. The combat one did. Apparently, these ticks don't initiate the OnTick.

The missed ticks themselves may be actual for tens of milliseconds.

[Sergey Golubev]

fxsaber:
There was a post above with the source code. It's now empty. The reason?

The post must have been when the site was updated (there are two such posts in the English part).

[fxsaber]

fxsaber:

I ran a parallel terminal in which the tkc was not generated by the terminal, but uploaded from the Server.

On the screenshot of this Terminal on the left - the tick is present. But on the other terminal (on the right) it is not!

I ran this script on both Terminals.

// Сохранение тиковой истории в текстовый файл.

string GetTickFlag( uint tickflag )
{
  string flag = " " + (string)tickflag;

#define  TICKFLAG_MACRO(A) flag += ((bool)(tickflag & TICK_FLAG_##A)) ? " TICK_FLAG_" + #A : ""; \
                        tickflag -= tickflag & TICK_FLAG_##A;
 TICKFLAG_MACRO(BID)
TICKFLAG_MACRO(ASK)
TICKFLAG_MACRO(LAST)
TICKFLAG_MACRO(VOLUME)
TICKFLAG_MACRO(BUY)
TICKFLAG_MACRO(SELL)
#undef  TICKFLAG_MACRO

  if (tickflag)
    flag += " FLAG_UNKNOWN (" + (string)tickflag + ")";

  return(flag);
}

#define  TOSTRING(A) " " + #A + " = " + (string)Tick.A
string TickToString( const MqlTick &Tick, const int FilterFlags = 0xFF )
{
  return(TOSTRING(time) + "." + ::IntegerToString(Tick.time_msc % 1000, 3, '0') +
         TOSTRING(bid) + TOSTRING(ask) + TOSTRING(last)+ TOSTRING(volume) + GetTickFlag(Tick.flags & FilterFlags));
}
#undef  TOSTRING


void OnStart()
{  
  MqlTick Ticks[];
  const int Size = CopyTicksRange(_Symbol, Ticks, COPY_TICKS_ALL, D'2021.01.05 01:00' * 1000, D'2021.01.05 10:50' * 1000);
  
  string From = (string)Ticks[0].time;
  string To = (string)Ticks[Size - 1].time;
  
  StringReplace(From, ":", ".");
  StringReplace(To, ":", ".");
  
  const int handle = FileOpen(_Symbol + "_" + From + "-" + To + ".txt", FILE_WRITE | FILE_ANSI | FILE_TXT);

  if (handle != INVALID_HANDLE)
  {        
    for (int i = 0; i < Size; i++)
      FileWrite(handle, TickToString(Ticks[i], 0x7F)); // Фильтр флагов, иначе очень много различий.

    FileClose(handle);
  }
  
  Alert("Done.");
}

Depending on what interval you request, tkc may change (sync with Server). So some ticks that were missing before the request may start to be present.

Despite this, it was still possible to detect a few on different characters. I had to apply a flags filter, because they are very different for ticks on different Terminals.

Here is how the differences look like.

EURJPY.

USDCHF .

In general, when trading in real time, some ticks may not be present in the history of ticks, as they are in the Terminal and can be on the Server.

This bug has to be fixed.