OnTrade/OnTradeTransaction - Are deal results always async?

[Doerk Hilger]

Hi,

since some time I am facing the issue, that after a position was closed/partially closed the deal database does not provide valid results - and this happens only with live accounts, never with demo accounts and only from time to time. Means, after a position was closed, I am requesting the results of the deal and use the function which is attached here to retrieve the latest / most recent deal result. It all worked without any problems for more than a year, but since some weeks/months I recognize, that the results are not always valid and that the function returns a deal-result of zero. I now added the message "Statistics error: No deal data found ..." which you can also see in the code to figure out what happens and check the deal-table manually afterwards, after the message was shown. The deal-result-table within the Toolbox of MT always shows correct results. 

My only remaining suspect is, that some of the transactions are postponed and processed asynchronously. Maybe this is the standard with MT5 and I simply overlooked it, or it has been changed with one of the latest build and/or it is caused due to the circumstance, that more and more users occupy the MT5 server and it's caused by that. Can anyone help with a clarifying answer?

Thank you for helping.

>>To clarify the source code: The function allows for return of closing data, no matter if a position ticket is available or the deal ticket. Deal ticket has priority, cause I use this function mainly to figure out the result of the latest deal. If the position ticket is used, its also possible to set the cumulate parameter to true, then the whole profit of a position will be returned, not only the last deal of a position. 

      #define TICKET ulong
      #define TICKET_INVALID 0
      #define PRICE_INVALID -0.0000001111      

      //+------------------------------------------------------------------+
      //|  Get closing information                                         |
      //+------------------------------------------------------------------+

         bool  GetCloseData(TICKET position, TICKET deal, double &price, datetime &time, double &profit, double &swap, double &commission, bool cumulate=false)
            {
               //--- Try to find the closing information 
               price=PRICE_INVALID;
               profit=0;
               time=0;
               swap=0;
               commission=0;
               
               //--- Workaround, swap and commission seem to be included
               bool getswapcommission=false;
               
               //--- Use deal ticket (priority)
               if (deal!=TICKET_INVALID)
                  {
                  if (!HistoryDealSelect(deal))
                     return false;
                  long closetime;
                  if (!HistoryDealGetInteger(deal,DEAL_TIME,closetime))
                     return false;
                  time=closetime;
                  price=HistoryDealGetDouble(deal,DEAL_PRICE);   
                  profit=HistoryDealGetDouble(deal,DEAL_PROFIT);
                  if (getswapcommission)
                     {
                     commission=HistoryDealGetDouble(deal,DEAL_COMMISSION);
                     swap=HistoryDealGetDouble(deal,DEAL_SWAP);
                     }
                  return true;                  
                  }
               
               
               //--- Use position ticket 
               if (position==TICKET_INVALID)
                  return false;
                  
               if (!HistorySelectByPosition(position))
                  return false;
                     
               int cnt=HistoryDealsTotal();
               int start=cnt-1;int end=0;
               int mostrecent=-1;

               //--- Find most recent entry
               long closetimemax=0;
               TICKET ticket=TICKET_INVALID;
               
               for (int i=cnt-1;i>=0;i--)
                  {
                  ticket=HistoryDealGetTicket(i);
                  long closetime=HistoryDealGetInteger(ticket,DEAL_TIME);
                  double tprofit=HistoryDealGetDouble(ticket,DEAL_PROFIT);

                  ENUM_DEAL_TYPE type=(ENUM_DEAL_TYPE)HistoryDealGetInteger(ticket,DEAL_TYPE);
                  ENUM_DEAL_ENTRY entry=(ENUM_DEAL_ENTRY)HistoryDealGetInteger(ticket,DEAL_ENTRY);
                  if (closetime>closetimemax && (type==DEAL_TYPE_BUY || type==DEAL_TYPE_SELL) && (entry==DEAL_ENTRY_OUT || entry==DEAL_ENTRY_OUT_BY || entry==DEAL_ENTRY_INOUT))
                     {
                     closetimemax=closetime;
                     mostrecent=i;
                     }
                  }
               //--- Not found? Data may be async
               if (mostrecent<0)
                  {
                  Print("Statistics error: No deal data found for #",position,"/#",deal);
                  return false;
                  }

               if (!cumulate)
                  {
                  start=mostrecent;
                  end=mostrecent;
                  }
                        
               for (int i=start;i>=end;i--)
                  {
                  ticket=HistoryDealGetTicket(i);
                  double tprofit=HistoryDealGetDouble(ticket,DEAL_PROFIT);
                  double tcommission=HistoryDealGetDouble(ticket,DEAL_COMMISSION);
                  double tswap=HistoryDealGetDouble(ticket,DEAL_SWAP);
                  double closeprice=HistoryDealGetDouble(ticket,DEAL_PRICE);
                  long closetime=HistoryDealGetInteger(ticket,DEAL_TIME);
                  
                  //--- Cumulative result includes all deals
                  profit+=tprofit;
                  commission+=tcommission;
                  swap+=tswap;
                  Print("======> #",(string)(i),": Ticket ",(string)ticket, " / Profit ",(string)tprofit, " / Price ",(string)closeprice + " / Text "+ (string)text);
                  
                  //--- Close time and price is taken from most recent entry only
                  if (i==mostrecent)
                     {
                     price=closeprice;
                     time=(datetime)closetime;
                     break;
                     }
                  }
               if (!getswapcommission)
                  {
                  commission=0;
                  swap=0;
                  }
               
               return true;
            }
 

[fxsaber]

This is one of the many features of MT5. For example, a deal may be in history, but its orders will not exist for some time.

[Doerk Hilger]

Ok, I have updated the code in the original posting (to not produce too much text here), cause there was a bug when using deal directly instead of looping through the history - BUT - this part was never used before. Based on the comment of fxsaber I changed the logic that way, that if possible, the deal ticket is now used itself cause I suspect that the history table is updated asynchronously and does not necessarily include all deals immediately after a deal was executed. Anyway, this makes more sense. It works so far in strategy tester, but I will have to figure out if the problem is still existing when trading with a live account or not.

Any more ideas appreciated.

[Doerk Hilger]

I now tested it on a live account again and still have the same problem. What I figured out in the meanwhile is, that, after execution of OrderSend(), the MqlTradeResult structure returns the code 10009 (Request completed), but the members .price, .deal, etc. have a value of zero. Great. So either this is a bug of MT5 or I am right, that the execution and/or deal and history data is always postponed, no matter if we use OrderSend() or OrderSendAsync(). 

Result of OrderSend() (Not async!) with live-account:

Result in Toolbox-History:

And really no ideas from no one? Can't be that I am the first person who recognizes such behavior ... some help is really appreciated here.

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px 'Times New Roman'} p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px 'Times New Roman'}

[Enrique Dangeroux]

Like fxsaber mentioned, it is a feature. Albeit not a pleasant one. It has been described " I used  OrderSend, but at the same time tracked the end of the transaction through OnTradeTransaction (since 10009 does not guarantee you anything at all). "  here https://www.mql5.com/ru/forum/335184

Maybe the replies can give you some ideas to counter the problem.

[Doerk Hilger]

Thx guys.

So, to clarify this to those who will search for the same topic in the future, the answer is:

In case of problems/doubts - YES - such data is postponed and transmitted by OnTradeTransaction(), NO MATTER IF OrderSendAsync() or OrderSend() was used.

Solution:

When using OrderSend(), the MqlTradeResult structure will return an order-id which can be found at MqlTradeResult.order. This field contains the transaction-order-id, has nothing to do with an order id which is used for trading, this is internal stuff of the statistics database. This order-id should be stored somewhere, so that it can be used to identify the transaction during OnTradeTransaction()

The code within OnTradeTransaction() looks like this then

void  OnTradeTransaction ( const  MqlTradeTransaction & trans,
                         const  MqlTradeRequest & request,
                         const  MqlTradeResult & result)
   {
   bool report=true;
   if (report)
      {
      Print("OnTradeTransAction :",EnumToString(trans.type));
      Print("result.retcode =" , result.retcode);
      }
   if (trans.type == TRADE_TRANSACTION_REQUEST && result.retcode==10009 && (request.type==ORDER_TYPE_BUY || request.type==ORDER_TYPE_SELL || request.type==ORDER_TYPE_CLOSE_BY))
      {
      if (report)
         {
         Print("result.order =",result.order);
         Print("result.deal =",result.deal);
         Print("request.type =",EnumToString(request.type));
         }
         HistoryOrderSelect (result.order);
         long position = HistoryOrderGetInteger (result.order, ORDER_POSITION_ID );
         if (position==0)
            return;
         if (report)
            {   
            Print("position =",position);
            Print("request.magic =",request.magic);
            Print("request.symbol =",request.symbol);
            
            Print("request.volume =",request.volume);
            Print("request.price =",request.price);
            //Print("request.sl =",request.sl);
            //Print("request.tp =",request.tp);
            }
         if (!HistorySelectByPosition(position))
            {
            Print("MT5 statistics error / Trade transaction could not access position ",position);
            return;
            }
         int cnt=HistoryDealsTotal();
         for (int i=cnt-1;i>=0;i--)
            {
            ulong deal=HistoryDealGetTicket(i);
            long order=HistoryDealGetInteger(deal,DEAL_ORDER);

            // HERE WE GO!!!
            // ---------------
            // Check if this is the transaction order we are looking for

            if (order==order)
               {
               if (report)
                  Print("Transaction found in history: ",order);
               //--- Here you find the closing data including profits   
               double tprofit=HistoryDealGetDouble(deal,DEAL_PROFIT);
               double tcommission=HistoryDealGetDouble(deal,DEAL_COMMISSION);
               double tswap=HistoryDealGetDouble(deal,DEAL_SWAP);
               double tcloseprice=HistoryDealGetDouble(deal,DEAL_PRICE);
               long tclosetime=HistoryDealGetInteger(deal,DEAL_TIME);
               if (report)
                  {
                  Print("Profit: ",tprofit);
                  Print("Closetime: ",(datetime)tclosetime);
                  Print("Price: ",tcloseprice);
                  }
               }
            }
         }
   }

I still don't like the idea at all, and I still think it cannot be the right processing behavior when OrderSend() was used and not OrderSendAsync(), cause it extremely overloads and overcomplicates the final code, but anyway, this is the way how it works.

[Alain Verleyen]

Doerk Hilger:

Thx guys.

So, to clarify this to those who will search for the same topic in the future, the answer is:

In case of problems/doubts - YES - such data is postponed and transmitted by OnTradeTransaction(), NO MATTER IF OrderSendAsync() or OrderSend() was used.

Solution:

When using OrderSend(), the MqlTradeResult structure will return an order-id which can be found at MqlTradeResult.order. This field contains the transaction-order-id, has nothing to do with an order id which is used for trading, this is internal stuff of the statistics database. This order-id should be stored somewhere, so that it can be used to identify the transaction during OnTradeTransaction()

The code within OnTradeTransaction() looks like this then

I still don't like the idea at all, and I still think it cannot be the right processing behavior when OrderSend() was used and not OrderSendAsync(), cause it extremely overloads and overcomplicates the final code, but anyway, this is the way how it works.

OnTradeTransaction() is not mandatory, you can also use OnTrade(), but for sure, whatever the method used you need to be sure the history is synchronized with the server. OrderSend() is actually asynchronous in practice, even if Metaquotes doesn't want to recognize it.

And I have seen it happening on a demo server too.