How to monitor the market in one minute - Trading Systems

Renat Fatkhullin 2014.11.10 15:42 #21

It is advisable to keep the computer time accurate, so there is less chance of getting the logs wrong.

Thanks for the script: here are the results of this script in 1 minute 17:36

CollectTicks (ED-12.14,M1)      2014.11.10 17:36:06; Bid = 1.2474; Ask = 1.2475; Last = 1.2476; Vol = 3
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:06; Bid = 1.2474; Ask = 1.2476; Last = 1.2476; Vol = 3
 CollectTicks (ED-12.14,M1)      2014.11.10 17:36:12; Bid = 1.2474; Ask = 1.2476; Last = 1.2476; Vol = 5
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:26; Bid = 1.2474; Ask = 1.2475; Last = 1.2476; Vol = 5
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:28; Bid = 1.2474; Ask = 1.2476; Last = 1.2476; Vol = 5
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:29; Bid = 1.2474; Ask = 1.2476; Last = 1.2474; Vol = 8
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:29; Bid = 1.2473; Ask = 1.2475; Last = 1.2474; Vol = 8
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:30; Bid = 1.2473; Ask = 1.2474; Last = 1.2473; Vol = 5
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:30; Bid = 1.2472; Ask = 1.2474; Last = 1.2473; Vol = 5
CollectTicks (ED-12.14,M1)      2014.11.10 17:36:43; Bid = 1.2472; Ask = 1.2474; Last = 1.2474; Vol = 2

The peculiarity of OnTick is that it reacts to both Bid/Ask changes (info flow) and Last (trade flow).

As a result, the number of OnTick calls for exchange traded instruments is knowingly larger than the number of trade transactions.

Why is it done this way?

you can't choke an Expert Advisor only with trade transactions and not give it information about market changes (it's one thing to call it 5 times, and another thing is to call it 15 times)
it is impossible to give out zero last and last_volume in the live market review, because some traders will explode

There is another thing with the fact that on batch updates, when two or more ticks on an instrument arrive at the same time, OnTick is called once. This allows us not to slow down the flow of incoming quotes and not to make a huge queue of ticks for the slowing Expert Advisors.

Soon we will enable the Time & Sales flow, which will allow us to accurately analyse the trades feed.

Real and Generated Ticks Close By - Trade Close By - Trade

Mikhail Filimonov 2014.11.10 16:14 #22

Renat:

It is advisable to keep the computer time accurate, so there is less chance of getting the logs wrong.

Thanks for the script: here are the results of this script in 1 minute 17:36

The peculiarity of OnTick is that it reacts to both Bid/Ask changes (info flow) and Last (trade flow).

As a result, the number of OnTick calls for exchange traded instruments is knowingly larger than the number of trade transactions.

Why is it done this way?

you can't choke an Expert Advisor only with trade transactions and not give it information about market changes (it's one thing to call it 5 times, and another thing is to call it 15 times)
it is impossible to give out zero last and last_volume in the live market review, because some traders will explode

There is another thing with the fact that on batch updates, when two or more ticks by tool arrive at the same time, then OnTick is called once. This allows you not to slow down the flow of incoming quotes and not to make a huge queue of ticks for slowing down EAs.

Soon we will include a Time & Sales stream, which will give us the ability to accurately analyse the tape of trades.

This is all great, of course, but for FORTS, where there are stacks, a quote flag should have been added to the MqlTick structure.

The "multiplication" of trades is not terrible - the worst thing is that the real trades are lost.

P.S. My BAN is related to my IP address, please remove it, otherwise I have to reinstall my browser every time to answer you.

MQL5 Wizard: Development of OTP - Accounts - OTP - Accounts -

Dmitriy Skub 2014.11.10 16:14 #23

Есть еще один момент с тем, что на пакетных обновлениях, когда одновременно приходят два и более тика по инструменту, то OnTick вызывается однократно. Это позволяет не тормозить поток входящих котировок и не делать огромную очередь тиков для тормозящих экспертов.

This approach results in the following:

As it should be:

Paper Time(mcs) Price Qty

317922.000000 RTS-12.14 [FORTS] 18:44:54 199000.000000 104140.000000 1.000000

317923.000000 RTS-12.14 [FORTS] 18:44:54 261000.000000 104130.000000 1.000000
317924.000000 RTS-12.14 [FORTS] 18:44:54 587000.000000 104140.000000 1.000000
317925.000000 RTS-12.14 [FORTS] 18:44:54 589000.000000 104140.000000 1.000000
317926.000000 RTS-12.14 [FORTS] 18:44:54 691000.000000 104140.000000 1.000000
317927.000000 RTS-12.14 [FORTS] 18:44:54 691000.000000 104140.000000 1.000000
317928.000000 RTS-12.14 [FORTS] 18:44:54 842000.000000 104130.000000 2.000000
317929.000000 RTS-12.14 [FORTS] 18:44:54 919000.000000 104150.000000 1.000000
317930.000000 RTS-12.14 [FORTS] 18:44:55 337000.000000 104150.000000 1.000000
317931.000000 RTS-12.14 [FORTS] 18:44:55 337000.000000 104150.000000 1.000000
317932.000000 RTS-12.14 [FORTS] 18:44:55 337000.000000 104160.000000 1.000000
317933.000000 RTS-12.14 [FORTS] 18:44:55 337000.000000 104160.000000 1.000000
317934.000000 RTS-12.14 [FORTS] 18:44:55 337000.000000 104160.000000 3.000000
317935.000000 RTS-12.14 [FORTS] 18:44:55 375000.000000 104140.000000 1.000000
317936.000000 RTS-12.14 [FORTS] 18:44:55 375000.000000 104130.000000 5.000000
317937.000000 RTS-12.14 [FORTS] 18:44:55 796000.000000 104130.000000 1.000000
317938.000000 RTS-12.14 [FORTS] 18:44:55 799000.000000 104130.000000 2.000000
317939.000000 RTS-12.14 [FORTS] 18:44:56 54000.000000 104160.000000 1.000000
317940.000000 RTS-12.14 [FORTS] 18:44:57 235000.000000 104160.000000 1.000000
317941.000000 RTS-12.14 [FORTS] 18:44:57 451000.000000 104140.000000 2.000000
317942.000000 RTS-12.14 [FORTS] 18:44:57 487000.000000 104140.000000 6.000000
317943.000000 RTS-12.14 [FORTS] 18:44:57 521000.000000 104160.000000 1.000000
317944.000000 RTS-12.14 [FORTS] 18:44:57 621000.000000 104160.000000 1.000000
317945.000000 RTS-12.14 [FORTS] 18:44:58 38000.000000 104160.000000 3.000000
317946.000000 RTS-12.14 [FORTS] 18:44:58 695000.000000 104160.000000 13.000000
317947.000000 RTS-12.14 [FORTS] 18:44:58 883000.000000 104140.000000 2.000000
317948.000000 RTS-12.14 [FORTS] 18:44:58 986000.000000 104160.000000 1.000000
317949.000000 RTS-12.14 [FORTS] 18:44:58 993000.000000 104160.000000 5.000000
317950.000000 RTS-12.14 [FORTS] 18:44:58 993000.000000 104160.000000 5.000000
317951.000000 RTS-12.14 [FORTS] 18:44:58 995000.000000 104160.000000 5.000000
317952.000000 RTS-12.14 [FORTS] 18:44:58 996000.000000 104160.000000 1.000000
317953.000000 RTS-12.14 [FORTS] 18:44:59 56000.000000 104160.000000 1.000000
317954.000000 RTS-12.14 [FORTS] 18:44:59 560000.000000 104140.000000 1.000000
317955.000000 RTS-12.14 [FORTS] 18:44:59 604000.000000 104140.000000 1.000000
317956.000000 RTS-12.14 [FORTS] 18:44:59 647000.000000 104160.000000 1.000000

As we get in MT5:

NE 0 18:44:54.483 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:54 Bid = 104130 Ask = 104150 Last = 104140 Vol = 1
ES 0 18:44:54.639 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:54 Bid = 104130 Ask = 104150 Last = 104130 Vol = 2
OR 0 18:44:54.720 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:54 Bid = 104130 Ask = 104150 Last = 104150 Vol = 1
NQ 0 18:44:54.983 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:55 Bid = 104140 Ask = 104150 Last = 104150 Vol = 1
FP 0 18:44:55.139 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:55 Bid = 104140 Ask = 104160 Last = 104160 Vol = 3
JO 0 18:44:55.174 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:55 Bid = 104140 Ask = 104160 Last = 104130 Vol = 5
GN 0 18:44:55.206 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:55 Bid = 104130 Ask = 104160 Last = 104130 Vol = 5
RM 0 18:44:55.592 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:55 Bid = 104130 Ask = 104160 Last = 104130 Vol = 2
IL 0 18:44:55.891 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:56 Bid = 104130 Ask = 104160 Last = 104160 Vol = 1
HJ 0 18:44:55.921 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:56 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
RI 0 18:44:57.032 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:57 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
PH 0 18:44:57.242 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:57 Bid = 104140 Ask = 104160 Last = 104140 Vol = 2
FG 0 18:44:57.297 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:57 Bid = 104140 Ask = 104160 Last = 104140 Vol = 6
PF 0 18:44:57.328 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:57 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
JE 0 18:44:57.436 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:57 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
QD 0 18:44:57.838 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:58 Bid = 104140 Ask = 104160 Last = 104160 Vol = 3
DP 0 18:44:58.514 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:58 Bid = 104140 Ask = 104160 Last = 104160 Vol = 13
RR 0 18:44:58693 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:58 Bid = 104140 Ask = 104160 Last = 104140 Vol = 2
JP 0 18:44:58795 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:58 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
CO 0 18:44:58.852 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:59 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
GN 0 18:44:59.358 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:59 Bid = 104140 Ask = 104160 Last = 104140 Vol = 1
MM 0 18:44:59.406 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:59 Bid = 104140 Ask = 104160 Last = 104140 Vol = 1
IL 0 18:44:59.453 FORTSTicksAction (RTS-12.14,M1) Time = 18:44:59 Bid = 104140 Ask = 104160 Last = 104160 Vol = 1
CK 0 18:50:31.437 FORTSTicksAction (RTS-12.14,M1) Time = 18:50:24 Bid = 0 Ask = 0 Last = 104160 Vol = 1
KH 0 18:50:33.357 FORTSTicksAction (RTS-12.14,M1) Time = 18:50:24 Bid = 104080 Ask = 104180 Last = 104160 Vol = 1

These are all last just before the session closure (i.e. there are no lower quotes until the end of clearing).

As a starting point let's take the three lines marked in black. There are two questions:

1.Where did the ones highlighted in red go?

2.where did the highlighted in blue come from?

Another question (optional) - what time interval does "simultaneous arrival" mean?

Custom Financial Instruments - Spreads - For Advanced Margin Calculation: Retail Forex,

Renat Fatkhullin 2014.11.10 16:41 #24

Dima_S:

This approach results in the following:

317945.000000 RTS-12.14 [FORTS] 18:44:58 38000.000000 104160.000000 3.000000
317946.000000 RTS-12.14 [FORTS] 18:44:58 695000.000000 104160.000000 13.000000
317947.000000 RTS-12.14 [FORTS] 18:44:58 883000.000000 104140.000000 2.000000
317948.000000 RTS-12.14 [FORTS] 18:44:58 986000.000000 104160.000000 1.000000
317949.000000 RTS-12.14 [FORTS] 18:44:58 993000.000000 104160.000000 5.000000
317950.000000 RTS-12.14 [FORTS] 18:44:58 993000.000000 104160.000000 5.000000
317951.000000 RTS-12.14 [FORTS] 18:44:58 995000.000000 104160.000000 5.000000
317952.000000 RTS-12.14 [FORTS] 18:44:58 996000.000000 104160.000000 1.000000

These are all the last ones just before the session closes (i.e. no lower quotes until the end of clearing).

Let's take the three lines highlighted in black as a starting point. There are two questions:

1.Where did the ones highlighted in red go?

They came in one network packet most likely and were called as one OnTick. There were 8 ticks in one second, three of them were single calls, and the last 5 most likely came in one packet with the last quote shown as Last.

2.where did the blue highlighting come from?

This is postmarket. We also give the ability to catch postmarket events. That's where withdrawal of bids usually only works.

Another question (optional) - what time interval does "simultaneous arrival" mean?

It's not an interval, but rather a physical network packet. A packet arrives on 2-3-4 ticks, they are all overlaid in the base, but the OnTick call goes once.

There is also a chance that the Expert Advisor might be slowed down (busy) and a new quote is added to the database but OnTick is not called, since the Expert Advisor is working. If there were no such a mechanism, we would easily overflow the incoming queues of quotes for the Expert Advisor and start sending the old quotes to it.

Price Data - For Real and Generated Ticks MetaTrader 5 Built-in Trading

Mikhail Filimonov 2014.11.10 17:30 #25

Renat!

Thank you very much for the detailed explanation, but...

It seems to me that this approach is unacceptable for the FORTS.

It should be: Quote - volume, and nothing else.

There were 11 quotes, so there should be 11 of them in OnTick.

Isn't it so?

Placing Pending Orders - Request Execution - Opening Request Execution - Opening

Renat Fatkhullin 2014.11.10 17:50 #26

Wrong.

Do not confuse the issue of delivering correct and accurate data streams to the terminal with calling/notifying experts.

All data is delivered accurately and correctly to the terminal, its basis. Just compare the charts.

On the other hand, the start of the Expert Advisor is an independent and asynchronous notification. Incoming packet of data can not wait for some slow experts, but must be instantly applied to the market environment and only optionally (if expert has finished previous call) run expert.

Think about the situation: a flow of quotes 2 per second and working Expert Advisor, which is braking, makes trades, but it requires that the new waiting quotes in the queue are not imposed on the market environment of the terminal, until the EA finishes its work.

Can you see where this would lead to? The whole terminal will start to delay all processes for the sake of the silly idea of keeping a market snapshot on the state of the quotation being processed. And if you think about other instruments, you'll just shoot yourself.

That's why market updates overlay is always a priority, instantaneous and independent of other processes. All others must asynchronously keep up with the market.

Migration - Virtual Hosting Futures - For Advanced Trading Robots and Technical

Mikhail Filimonov 2014.11.10 19:45 #27

Renat:

Wrong.

Do not confuse the issue of delivering correct and accurate data streams to the terminal with calling/notifying experts.

All data is delivered accurately and correctly to the terminal, its basis. Just compare the charts.

On the other hand, the start of the Expert Advisor is an independent and asynchronous notification. Incoming packet of data can not wait for some slow experts, but must be instantly applied to the market environment and only optionally (if expert has finished previous call) run expert.

Think about the situation: a flow of quotes 2 per second and working Expert Advisor, which is braking, makes trades, but it requires that the new waiting quotes in the queue are not imposed on the market environment of the terminal, until the EA finishes its work.

Can you see where this would lead to? The whole terminal will start to delay all processes for the sake of the silly idea of keeping a market snapshot on the state of the quotation being processed. And if you think about other instruments, you'll just shoot yourself.

That's why market updates overlay is always a priority, instantaneous and independent of other processes. All others must asynchronously keep up with the market.

Disagree.

Let's say I don't use the cup (bid and ask are in the tick)

What idea about the FORTS market situation will I (an expert) have without knowing the quote

and volume?

Indeed, for FORTS, we may not pass bid and ask (they are in the ticker).

The changes in the code are minimal - bid and ask are not passed to the market.

Price Data - For Types of Orders - Types of Orders -

Renat Fatkhullin 2014.11.10 20:00 #28

Mikalas:

I disagree.

Reread my explanation several times, please.

Let's say I don't use a rate book (bid and ask are in the tick)

What view of the FORTS market I (the expert) will have with a quote that is not clear

With the current asynchronously updated quote and no other.

If you've been called, that's it - only the quote you have in the packet is in front of you. If there are 2 quotations in the batch, you will see only the last one. For no one will pass the first quote to the Expert Advisor and wait until the Expert Advisor finishes the operation. The old quote has gone and if you have not caught it in time to work it out, it means that the train has passed. Using that old quote, you will not be able to execute anything. And the current/last one is also doubtful to be executed, because the market might have changed by the moment of sending the order.

No "do not translate bid/ask" will help. Obsolete ticks are obsolete ticks - no one will freeze the market and do not let you work with old ticks.

If you want, you can:

put the terminal in colocation closer to the broker on a high-speed computer and catch more and more
loop the EA, constantly scan the market and market watch to get everything more often.
analyze the chart, analyze the market

Together with Time & Sales we will think about providing direct access to the latest tick price stream, so you can access consecutive ticks. In the market review, we collect it sessionally. This will give more opportunities for scalpers.

Trading Operations Real and Generated Ticks Depth of Market -

Dmitriy Skub 2014.11.10 20:34 #29

Renat:

Together with Time & Sales, we will consider providing direct access to the latest tick price stream, so that consistent ticks can be accessed. In Market Watch we collect it sessionally. This will give more opportunities to scalpers.

It is the only acceptable solution. Hopefully there will be phypho queues for given symbols. This will solve another problem, unsolvable within this event model, of getting lossless quotes for symbols other than the EA symbol.

Placing Pending Orders - Placing Pending Orders - Volumes - Volume Indicators

Mikhail Filimonov 2014.11.11 02:25 #30

I still have my doubts.

I'm going to put this into action:

//+------------------------------------------------------------------+
//|                                                    Kotirovki.mq5 |
//|                                                   Copyright 2014 |
//|                                              http://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2014, Mikalas"
#property link      "http://www.mql5.com"
#property version   "1.02"
//
MqlTick curr_tick;
double  start_last   = 0;
double  start_bid    = 0;
double  start_ask    = 0; 
ulong   start_volume = 0;  
//  
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
{
  return( INIT_SUCCEEDED );
}
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit( const int reason )
{

}
//+------------------------------------------------------------------+
//| Expert On Tick event function                                    |
//+------------------------------------------------------------------+  
void OnTick()
{
  if ( SymbolInfoTick( _Symbol, curr_tick ) )
  {
    if ( start_last == curr_tick.last )
    {
      if ( start_volume == curr_tick.volume )
      {
        if ( ( start_ask == curr_tick.ask ) &&
             ( start_bid == curr_tick.bid )  )
        {
          Print( curr_tick.time, "; Bid = ", curr_tick.bid,
                                 "; Ask = ", curr_tick.ask,
                                 "; Last = ", curr_tick.last,
                                 "; Vol = ", curr_tick.volume, "; Новая котировка(Всё одинаковое)" );
        }
        else
        {
          start_ask = curr_tick.ask;
          start_bid = curr_tick.bid;
          Print( curr_tick.time, "; Bid = ", curr_tick.bid,
                                 "; Ask = ", curr_tick.ask,
                                 "; Last = ", curr_tick.last,
                                 "; Vol = ", curr_tick.volume, "; Изменение ask или bid? Или же новая котировка?" );
        }
      }
      else
      {
        start_last = curr_tick.last;
        start_volume = curr_tick.volume;
        start_ask = curr_tick.ask;
        start_bid = curr_tick.bid;
        Print( curr_tick.time, "; Bid = ", curr_tick.bid,
                               "; Ask = ", curr_tick.ask,
                               "; Last = ", curr_tick.last,
                               "; Vol = ", curr_tick.volume, "Новая котировка (Изменён объём)!" );
      }
    }
    else
    {
      start_last = curr_tick.last;
      start_volume = curr_tick.volume;
      start_ask = curr_tick.ask;
      start_bid = curr_tick.bid;
      Print( curr_tick.time, "; Bid = ", curr_tick.bid,
                             "; Ask = ", curr_tick.ask,
                             "; Last = ", curr_tick.last,
                             "; Vol = ", curr_tick.volume, "Новая котировка! (Изменена котировка)" );
    }
  }
}

And tomorrow we'll see what happens...

Files:

Kotirovki.ex5 5 kb

Question on FORTS quotes - page 3