Tiki in real time - page 22

[Aleksey Mavrin]

Andrey Khatimlianskii:

Look at the time of the log. This all happened in one ms, and next to it (in the same ms) a bunch of OnBooks.

You can count all events by counters, but even visually you can see that there are more OnBooks.

Andrew, the numbers there are fixed microseconds, when OnFunctions are triggered, and all is printed together then from the array. Maybe there are more OnBooks in total - I will count them, but it is unclear why they skip ahead of the OnTicks in the queue. Or not every OnTick corresponds to an OnBook?

[Roman]

prostotrader:

Didn't you write that you use Async orders?
I was wondering, what algorithm do you use to control the execution of trades?

[Igor Makanu]

Aleksey Mavrin:
Andrew, the numbers there are fixed microseconds when OnFunctions are triggered, and everything is printed together then from the array. OnBooks in total may be more - I will count, but it is unclear why they skip ahead of the OnTicks in the queue or what. Or not every OnTick corresponds to an OnBook?

welcome to the world of networking ))))

The easiest thing to do is to runNetstat - a -b under the administrator.

You will see ports and software, I don't want to bother, but I think MT5 server asynchronously passes packets with different information which terminal puts into needed "shelves".

HH: Do you know about Print() and skipping of prints if you output a lot of them at once? - Just write your information in the file - that way you'll save everything and in order, but don't forget to close the file before closing. In theory and Print() in the logs in the file should be complete, but have not checked and in general I do not trust if a lot of data output. Discussed herehttps://www.mql5.com/ru/forum/329730, very often "missing prints" are discussed ))) - search

[Andrey Khatimlianskii]

Aleksey Mavrin:
Andrew, the numbers are fixed microseconds when OnFunctions are triggered, and then printed all together from the array. Maybe there are more OnBooks in total - I will count them, but it is unclear why they skip ahead of the OnTicks in the queue. Or not every OnTick corresponds to an OnBook?

Got it.

Well, there's a lot of OnBooks around anyway. It's hard to draw any conclusions with a log like that.

[prostotrader]

Roman:

I thought you wrote that you were using Async orders?
I was wondering, what algorithm do you use to control trade execution?

In OnTradeTransaction()+ check function if there is no server response for a long time.

Generally by magic.

I reserve 65535 magic symbols for each symbol when setting an EA,

and when sending an order I assign it a unique magic number which does not intersect other symbols in any way.

which does not overlap in any way with other instruments.

This is how I set the initial magic number for a symbol

//+------------------------------------------------------------------+
//|                                                    AutoMagic.mqh |
//|                                 Copyright 2017-2018 prostotrader |
//|                                             https://www.mql5.com |
//+------------------------------------------------------------------+
//version   "1.01
ulong symb_magic;
//-------------------------------------------------------------------+
// Split string function                                             |
//+------------------------------------------------------------------+
string SplitString(const string a_str,ulong &a_month,ulong &a_year)
  {
   int str_size=StringLen(a_str);
   int str_tire=StringFind(a_str, "-");
   int str_tochka=StringFind(a_str, ".", str_tire);
   if((str_tire>0) && (str_tochka>0) &&(str_size > 0))
     {
      a_month= ulong(StringToInteger(StringSubstr(a_str,str_tire+1,str_tochka-str_tire-1)));
      a_year = ulong(StringToInteger(StringSubstr(a_str,str_tochka+1,str_size-str_tochka-1)));
      if((a_month > 0) && (a_year > 0)) return(StringSubstr(a_str, 0, str_tire));
     }
   return("");
  }
//-------------------------------------------------------------------+
// Get Magic function                                                |
//+------------------------------------------------------------------+
ulong GetMagic(const string a_symbol)
{
  symb_magic = 0;
  if(SymbolSelect(Symbol(), true) == false)
  {
    Print(__FUNCTION__, ": Нет такого символа!");
    return(0);
  }
  ulong month = 0;
  ulong year = 0;
  string new_str = SplitString(a_symbol,month,year);
  if(StringLen(new_str)>0)
  {
    uchar char_array[];
    int result=StringToCharArray(new_str,char_array,0,WHOLE_ARRAY,CP_ACP);
    if(result>0)
   {
     ulong value;
     for(int i = 0; i < result - 1; i++)
     {
       value=ulong(char_array[i]);
       value<<=(56 -(i*8));
       symb_magic += value;
     }
     month<<=24;
     symb_magic += month;
     year<<=16;
     symb_magic += year;
     return(symb_magic);
   }
 }
  return(0); 
}
//-------------------------------------------------------------------+
// Is my magic function                                              |
//+------------------------------------------------------------------+
bool IsMyMagic(const ulong m_magic)
{
  if(m_magic > 0)
  {
    ulong stored_magic=symb_magic;
    stored_magic>>=16;
    ulong in_magic = m_magic;
    in_magic>>=16;
    if(in_magic == stored_magic) return(true);
  }  
  return(false);
}
//-------------------------------------------------------------------+
// Get stored magic function                                         |
//+------------------------------------------------------------------+
ulong GetStoredMagic()
{
  if(symb_magic > 0) return(symb_magic);
  return(0);  
}
//+------------------------------------------------------------------+

Magic - ulong (8 bytes) e.g.

GAZR-3.12

Byte[7] (high byte) is "G"

Byte[6] is "A"

Byte[5] for "Z"

Byte[4] is "R

Byte[3] is "3

Byte[2] is "12"

Byte[1] and Byte[0] are magiks reserve (65535)

This is how I go through the magiks when sending an order:

  mem_magic = magic_storage + 1;
  if(magic_storage >= (magic_number + 65530)) mem_magic = magic_number;

But this only works for FORTS as the symbol names are standardised!

Added

If the order is successfully sent, then

remember the time

  if(OrderSendAsync(request, result) == true)
  {
    if((result.retcode == TRADE_RETCODE_PLACED) || (result.retcode == TRADE_RETCODE_DONE)) 
    {
      req_id = result.request_id;
      magic_storage = mem_magic;
      state = ORD_DO_SET;
      mem_time = GetMicrosecondCount();
      mem_start_time = TimeCurrent();
      SetTransCount();
    }
    else
    {
      mem_magic = 0;
      mem_time = 0;
      mem_start_time = 0;
      CheckError(result.retcode, "Place: Ордер не установлен! Причина: ", order_status, ticket);
    }
  }

mem_time = GetMicrosecondCount(); - для проверки времени задержки OnTradeTransaction
mem_start_time = TimeCurrent();   - для сужения рамок поиска в истории

And then (if there is no response in OnTradeTransaction)

ticket = FindOrderBuyMagic(mem_magic, start_time);

And then theFindOrderBuyMagic function itself

#define  TIME_DELAY    180

//+------------------------------------------------------------------+
// Expert Find order Buy Magic function                              |
//+------------------------------------------------------------------+
ulong FindOrderBuyMagic(const ulong a_magic, const datetime set_time)
{
  if(a_magic > 0)
  {
    if(IsMyMagic(a_magic) == true)
    {
      ulong cur_ticket = 0;
      for(int i = OrdersTotal() - 1; i >= 0; i--)
      {
        cur_ticket = OrderGetTicket(i);
        if(OrderSelect(cur_ticket))
        {
          if( ulong(OrderGetInteger(ORDER_MAGIC)) == a_magic) return(cur_ticket);
        }  
     }
      cur_ticket = 0;
      datetime start_time = datetime(ulong(set_time) - TIME_DELAY);
      datetime end_time = datetime(ulong(TimeCurrent()) + TIME_DELAY);    
      if(HistorySelect(start_time, end_time))
      {
        for(int i = HistoryOrdersTotal() - 1; i >= 0; i--)
        {
          cur_ticket = HistoryOrderGetTicket(i);
          if(ulong(HistoryOrderGetInteger(cur_ticket, ORDER_MAGIC)) == a_magic) return(cur_ticket);
        }
      }
    }
  }
  return(0);
}

Added by

"Good idea" to add EA identifier(0-255) to the 1st byte of the automagic,

but I don't need it yet, so I haven't done it :)

[Roman]

prostotrader:

In OnTradeTransaction()+ check function if there is no server response for a long time.

Thanks for the tip.

[Roman Shiredchenko]

prostotrader:

1. The second and subsequent terminals at the broker are pay-as-you-go and I have no strategies where I trade only stocks(stock portfolios).

2. If you are going to output the accumulatedGetMicrosecondCount(), then

do it without timer in OnDeinit(), when EA exits, everything will print.

Please send me the link to the broker, it's possible in the LC.

This is an interesting branch... :-)

[Aleksey Mavrin]

Igor Makanu:

welcome to the world of networking ))))

The easiest thing to do is to runNetstat - a -b under the administrator.

You will see ports and software, I don't want to bother, but I think MT5 server asynchronously passes packets with different information which terminal puts into needed "shelves".

HH: Do you know about Print() and skipping of prints if you output a lot of them at once? - Just write your information in the file - that way you'll save everything in order, but don't forget to close the file before closing it. In theory and Print() in the logs in the file should be complete, but have not checked and in general I do not trust if a lot of data output. Discussed herehttps://www.mql5.com/ru/forum/329730, very often "missing prints" are discussed ))) - search

Igor, the loss of prints is discussed by those who haven't opened the log file, a hundred times and Rinat Fatkullin himself wrote that nothing is lost in the log file. But for your post was not in vain :) I've added output in a separate file, besides I've made the second file, where I output a little differently (collecting all events in CArrayObj) to bypass possible bugs of my design, which orders two arrays, i.e. I put everything in CArrayObj from two arrays, then sort by microseconds and output with marking what event Tick or Book.

And yes, what's that got to do with ports, what's that got to do with it? I'm just testing the EA event queue. If a tick came, two events should be formed - OnTick, and corresponding OnBook, and OnBook is always placed in queue, and OnTick can disappear if there is already OnTick in the queue (as in manual), ie . the situation when one after another OnTick without OnTick can be only if 1. the OnTicks go "without queuing" 2. there is a system delay of OnBook, this is what I want to check, this can explain the lag of seconds, previously identified by colleagues. Total OnBooks are 2+ times more in a day, but why are they lagging? If this delay is due to asynchronous packets and parsing, maybe, but I only check the fact of their arrival in the Expert Advisor so far. How to test with the rest of the nuances in mind, haven't thought about it yet.

Here is the new code, on opening I will test the correctness of work and run for the day.

s.w. The reason may also be: if Tick passed at the same prices without changing the cup - OnBook is not formed? I'm not a specialist in stock trading, who can tell me. I thought OnTick always causes OnBook.

//+------------------------------------------------------------------+
//|                                                   TestOnBook.mq5 |
//|                                           Copyright 2019, Allex@ |
//|                                                 [e-mail] |
//+------------------------------------------------------------------+
#property copyright "Copyright 2019, Allex@"
#property link      "[e-mail]"
#property version   "1.00"
#include <Allex\Logger.mqh>
#include <Arrays\ArrayObj.mqh>
//---
bool is_book;
enum ENUM_BOOK_OR_TICK
{
        USE_BOOK,       // Use OnBookEvent
        USE_TICK        // Use OnTick
};
class CMcsOn: public CObject
{
public:
ulong mcs;
ENUM_BOOK_OR_TICK Ontype;
CMcsOn(ulong m, ENUM_BOOK_OR_TICK t):mcs(m),Ontype(t){};
int       Compare(const CObject*Object,const int mode=0) const
     {
      const CMcsOn* obj1=dynamic_cast<const CMcsOn*>(Object);
      CMcsOn* obj=(CMcsOn*)(obj1);
      if(!obj)return 0;
      return (mcs-obj.mcs);
      }
};
input ENUM_BOOK_OR_TICK Mode = USE_BOOK;
input int   SecForPrint =  3600;
//---
ulong TimeArrayBook[65536];
ulong TimeArrayTick[65536];
ushort curBook,curTick;
ulong  DelaySum=0,DelayCount=0,CountOnBook=0,CountOnTick=0;
int delay,delayMax=0;
CLogger* Logger,*Logger2;
CArrayObj ArrayObj;
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
{
   curBook=0;
   curTick=0; 
   DelaySum=0;DelayCount=0;CountOnBook=0;CountOnTick=0;delayMax=0;
   ArrayInitialize(TimeArrayBook,INT_MAX);
   ArrayInitialize(TimeArrayTick,INT_MAX);
   Logger=CLogger::GetLogger();
   Logger2= new CLogger();
   Logger.SetSetting(__FILE__+"\\",Symbol()+"_"+EnumToString(Period())+"_"+TimeToString(TimeCurrent(),TIME_DATE));  
   Logger2.SetSetting(__FILE__+"\\","Alt_"+Symbol()+"_"+EnumToString(Period())+"_"+TimeToString(TimeCurrent(),TIME_DATE));  
  if(Mode == USE_BOOK) is_book = MarketBookAdd(Symbol());
  ArrayObj.Shutdown();
  if (EventSetTimer(SecForPrint) &&  Logger.Init() && Logger2.Init()) 
  return(INIT_SUCCEEDED);
  else return (INIT_FAILED);
}
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
  if(Mode == USE_BOOK)
  {
    if(is_book == true) MarketBookRelease(Symbol());
  }  
   delete Logger;
   delete Logger2;
}
//+------------------------------------------------------------------+
//| BookEvent function                                               |
//+------------------------------------------------------------------+
void OnBookEvent(const string &symbol)
{  
  TimeArrayBook[curBook++]=GetMicrosecondCount();
  CountOnBook++;
  //Print(__FUNCTION__, " ",curBook);
}
void OnTick()
{
  TimeArrayTick[curTick++]=GetMicrosecondCount();
  CountOnTick++;
  //Print(__FUNCTION__, " ",curTick);

}
//+------------------------------------------------------------------+
void OnTimer()
  {
   string out=NULL;
   int total=MathMax(curBook,curTick);
   int i=0,k=0;
   while(i<total)
     {
      while(i<total && TimeArrayBook[i]<TimeArrayTick[k] )
        {
          MyPrint("Book "+TimeArrayBook[i++]);
        }    
      if(k<curTick-1)
        {
        if(i<total)
          {
           delay=TimeArrayBook[i]-TimeArrayTick[k];
           if (delay>delayMax) 
            delayMax=delay;
           if (delay>0)
              {
                 DelaySum+=delay;
                 DelayCount++;
              }
          }
         MyPrint("Tick "+TimeArrayTick[k++]+ " delay mcs "+delay);
        }       
        i++;
     }
     if (curTick>0)
     {
     MyPrint("Tick "+TimeArrayTick[curTick-1]+ " last");
     string out="Count Event Book after Tick "+DelayCount+". Delay Average "+DoubleToString(DelaySum/DelayCount,2)+". Max "+delayMax+" OnBooks "+CountOnBook+" OnTicks "+CountOnTick;
     MyPrint (out);
     Comment(out);
     }
     Logger.Logger();
     Alt();
     curBook=0;
     curTick=0;
  }
//---
void MyPrint(string out)  
{
   Print(out);
   Logger.Log(__FUNCTION__,out,2,false);
}
//---
void Alt()
{
int last=ArrayObj.Total();
for(int i=0;i<curBook;i++)
  {
   if (!ArrayObj.Add(new CMcsOn(TimeArrayBook[i],USE_BOOK)))
      Logger2.Log(__FUNCTION__,"Error Book Add",0);   
  }
for(int i=0;i<curTick;i++)
  {
   if (!ArrayObj.Add(new CMcsOn(TimeArrayTick[i],USE_TICK)))
      Logger2.Log(__FUNCTION__,"Error Tick Add",0);   
  }
  ArrayObj.Sort();
  int total=ArrayObj.Total();
  total-=last;
  CMcsOn*Obj;
  for(int i=0;i<total;i++)
    {    
     Obj=ArrayObj.At(i);
     if(CheckPointer(Obj)==POINTER_INVALID )
      { Logger2.Log(__FUNCTION__,"Error At Array",0); continue;}
      string out = Obj.USE_BOOK ? "Book ": "Tick ";
      out+= Obj.mcs  ;
      Logger2.Log(__FUNCTION__,out,2);
    }
   Logger2.Log("ArrayObj_","Last "+last+" total "+ArrayObj.Total(),1);  
   Logger2.Logger();
   //ArrayObj.Shutdown(); 
}

[Vladimir Mikhailov]

prostotrader:
But I wonder if the person in charge is satisfied with the answers to his question.

I have already received all the answers and have drawn my own conclusions.
I need to analyse the strip of trades for a fixed period of time - prices of trades, realised volumes, etc.
And I also need to simulate the operation of the algorithm in the strategy tester.
The OnTick event copes perfectly with this, the results of real trading and the results of modeling in the tester agree with a small error to my satisfaction.
If you need faster analysis of the strip, you can use OnTimer.

And it is not necessary that every tick that comes to the terminal should be placed in OnBook - this is the specifics of market orders execution.

[Roman]

Vladimir Mikhailov:

And every tick that comes into the terminal does not necessarily have to go into OnBook - it is specific to the execution of market orders.

On the contrary, every tick(event) coming to the OnTick handler must be synchronized with OnBook.
There are three events in the OnTick handler, the change of price of the best bid, the change of price of the best ask, and the trade(last).
If the bid or ask price changes without a trade, this will be an event and OnTick will receive these events.
And OnBook also has to catch these events, but its own events, its handler, otherwise there will be a mismatch of bid and ask prices between handlers.

And if OnTick receives a last event, it means a trade has passed.
The trade generates the event in OnTick, because after the trade the price or the volume of bids and asks to change in the market.
It is a vicious circle.

In OnTick as well as in OnBook, there is an event Best Bid and Best Ask.
These events should always be synchronous in both handlers.
And the event last by itself, it generates an event in OnBook after the trade.
Therefore, any event that comes to OnTick handler must be synchronously reflected in OnBook.