Testing 'CopyTicks' - page 37

 

Please advise which type of flag corresponds to number 24. Experimentally determined that

  • TICK_FLAG_BID = 2
  • TICK_FLAG_ASK = 4
  • TICK_FLAG_LAST = 8
  • TICK_FLAG_VOLUME = 16
  • TICK_FLAG_BUY = 32
  • TICK_FLAG_SELL = 64
 
Sergey Gritsay:

Please advise which type of flag corresponds to number 24. Experimentally determined that

  • TICK_FLAG_BID = 2
  • TICK_FLAG_ASK = 4
  • TICK_FLAG_LAST = 8
  • TICK_FLAG_VOLUME = 16
  • TICK_FLAG_BUY = 32
  • TICK_FLAG_SELL = 64

Flags can be stacked. To determine which events occurred (Here"tick" is an array with ticks):

         string flags="";
         if((tick.flags  &TICK_FLAG_BID)==TICK_FLAG_BID)
            flags=" TICK_FLAG_BID ";
         if((tick.flags  &TICK_FLAG_ASK)==TICK_FLAG_ASK)
            flags+=" TICK_FLAG_ASK ";
         if((tick.flags  &TICK_FLAG_LAST)==TICK_FLAG_LAST)
            flags+=" TICK_FLAG_LAST ";
         if((tick.flags  &TICK_FLAG_VOLUME)==TICK_FLAG_VOLUME)
            flags+=" TICK_FLAG_VOLUME ";
         if((tick.flags  &TICK_FLAG_BUY)==TICK_FLAG_BUY)
            flags+=" TICK_FLAG_BUY ";
         if((tick.flags  &TICK_FLAG_SELL)==TICK_FLAG_SELL)
            flags+=" TICK_FLAG_SELL ";
         Comment(flags);
 

Good afternoon, colleagues.

Maybe someone will be interested to model a problem, which I encountered - it seems that CopyTicks in strategy tester in mode "Every tick based on real ticks" in some cases begins to double ticks, which are returned by one request.

Checked it on MOEX RTS in Expert Advisor, by requesting new ticks in OnTimer() event. See attached code.

TheTICK_FLAG_BUYTICK_FLAG_SELLflagsof doubles may be repeated and may be different from the original records (though to understand which of them are originals and which are duplicates, is impossible without comparing with another source of ticks).

I tried to connect to the real "Opening" and MetaQuotes-Demo - the result is the same.

Checked also on build 1578. Deleted all ticks folders on my machine. After auto-updating them the result is the same.

Here is a fragment of the log showing that one request has suddenly started repeated ticks (with shorter time than previous ones)


Startup MetaTester 5 x64 build 1578 (07 Apr 2017)

Server MetaTester 5 started on 127.0.0.1:3000

Startup initialization finished

127.0.0.1 login (build 1578)

Network 38520 bytes of account info loaded

Network 1482 bytes of tester parameters loaded

Network 188 bytes of input parameters loaded

Network 4192 bytes of symbols list loaded

Tester expert file added: Experts\!sn_err.ex5. 11617 bytes loaded

Tester initial deposit 10000.00 RUR, leverage 1:100

Tester successfully initialized

Network 29 Kb of total initialization data received

Tester QEMU Virtual version (cpu64-rhel6), 2047 MB

Symbols RTS-6.17: symbol to be synchronized

Symbols RTS-6.17: symbol synchronized, 3784 bytes of symbol info received

History RTS-6.17: history synchronization started

History RTS-6.17: load 31 bytes of history data to synchronize in 0:00:00.000

History RTS-6.17: history synchronized from 2016.01.20 to 2017.04.07

Ticks RTS-6.17: ticks synchronization started

Ticks RTS-6.17: load 38 bytes of tick data to synchronize in 0:00:00.000

Ticks RTS-6.17: history ticks synchronized from 2017.04.06 to 2017.04.06

History RTS-6.17,M1: history cache allocated for 475200 bars and contains 23851 bars from 2016.01.20 12:28 to 2017.04.05 23:45

History RTS-6.17,M1: history begins from 2016.01.20 12:28

Tester RTS-6.17,M1 (MetaQuotes-Demo): generating based on real ticks

Tester RTS-6.17,M1: testing of experts\!sn_err.ex5 from 2017.04.06 00:00 to 2017.04.07 00:00 started

Ticks RTS-6.17 : real ticks start from 2017.04.06 00:00:00


...

CYCLE: 16 TICK: 2017.04.06 11:06:42;1491476802652;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 16 TICK: 2017.04.06.06 11:06:43;1491476803512;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 16 TICK: 2017.04.06.06 11:06:46;1491476806819;2;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 16 TICK: 2017.04.06.06 11:06:46;1491476806865;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 16 TICK: 2017.04.06.06 11:06:46;1491476806865;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807075;1;88; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_SELL

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;3;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;10;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;4;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807472;3;88; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_SELL

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807472;4;88; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_SELL

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807472;1;88; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_SELL

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807075;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06.06 11:06:47;1491476807470;3;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807470;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807470;10;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807470;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807470;4;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807470;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807472;3;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807472;4;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 17 TICK: 2017.04.06 11:06:47;1491476807472;1;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY DOUBLE

CYCLE: 19 TICK: 2017.04.06 11:06:49;1491476809747;3;88; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_SELL

CYCLE: 19 TICK: 2017.04.06.06 11:06:49;1491476809747;7;88; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_SELL

CYCLE: 19 TICK: 2017.04.06.06 11:06:49;1491476809747;3;56; TICK_FLAG_LAST TICK_FLAG_VOLUME TICK_FLAG_BUY

...

Files:
jsn_err.mq5  5 kb
 
Snest:

Perhaps someone would be interested to simulate a problem I encountered - it seems CopyTicks in strategy tester in "Every tick based on real" mode in some cases starts to double ticks returned by one request.

This is a very unpleasant thing in your situation.

So there is only one way out - to give up the tester for a while.


SZZ Looked your code and log. Definitely bug CopyTicks in the tester. Interesting, DOUBLE-ticks doesn't start with full duplication. Anyway, copy-paste your post to the SD. At the moment CopyTicks cannot be used in the tester and the tester itself cannot be used in "by real ticks" mode.

 
fxsaber:

A very unpleasant thing has come up in your situation.

So there's only one way out - to give up the tester for a while.


Looked your code and log. Definitely a bug CopyTicks in the tester. Interesting, DOUBLE-ticks doesn't start with full duplication. Anyway, copy-paste your post to the SD. At the moment CopyTicks cannot be used in the tester and the tester itself cannot be used in "by real ticks" mode.


Thank you, I have sent a message to servicedesk. I will write back with the results.
 
In history, the correct ticks go from a certain date (I wrote about this earlier). Before that, there are doubles. This is if the account is in Otkritie.
 
Dmitriy Skub:
In the history correct ticks go from a certain date (wrote earlier about it). Before that they go with duplicates. This is if the account is in the Open.


The logs show that the test was conducted for April 06, 2017. Similar results for April 07.

The situation is similar on both Opening-Real and MQ-Demo

 
Snest:


The logs show that the test was conducted for April 06, 2017. Similar results for 07 April.

The situation is similar on both Opening-Real and MQ-Demo

Tadas to the morgue. I mean, to the SD))
 
Dmitriy Skub:
To the morgue, then. I mean, the SR.)

That's a very good point.
 

It's been three months and the ticket still hasn't been closed. Trouble.


Additionally, I found another bug (if the problem is not related to the quote provider), this time with wrong direction of transaction. I'm copying the text of the ticket to warn the community:

A comparison of MT5 tick data obtained by the standard tool "Price Stack / Show All Trades Table" on a real AMP-Features account with quotes from CQG, with data from Ritmik has shown that MT5 in some apparently quite rare cases incorrectly determines the direction of the trade.

We unload the Depth of Market for 11.07 using the standard tool "Depth of Market / Show table of all deals". Look at the data on futures GCEQ7 as of 00:03 min Chicago time (in the table below it is not Chicago time but GMT)

Result.

2017.07.1105:03:00.3241209.71209.81209.71Sell
2017.07.1105:03:00.3241209.71209.81209.71Sell
2017.07.1105:03:00.3241209.71209.81209.71Sell
2017.07.1105:03:00.3241209.71209.81209.71Sell
2017.07.1105:03:00.3241209.71209.81209.71Sell
2017.07.1105:03:00.9701209.71209.81209.72Sell
2017.07.1105:03:00.9701209.71209.81209.71Sell
2017.07.1105:03:00.9701209.71209.81209.72Sell
2017.07.1105:03:00.9701209.71209.81209.71Sell
2017.07.1105:03:00.9701209.71209.81209.71Sell
2017.07.1105:03:00.9701209.71209.81209.71Sell
2017.07.1105:03:00.9771209.71209.81209.72Sell
2017.07.1105:03:00.9771209.71209.81209.71Sell
2017.07.1105:03:00.9771209.71209.81209.72Sell
2017.07.1105:03:00.9771209.71209.81209.71Sell
2017.07.1105:03:00.9771209.71209.81209.71Sell
2017.07.1105:03:00.9771209.71209.81209.71Sell
2017.07.1105:03:00.9771209.71209.81209.71Buy
2017.07.1105:03:01.3621209.71209.81209.71Sell
2017.07.1105:03:02.1171209.71209.81209.73Sell
2017.07.1105:03:02.1171209.71209.81209.72Sell
2017.07.1105:03:02.1171209.71209.81209.71Sell

Expected result

Data from Rithmic as per Moscow time:

2017-07-11 08:03:00.26023314997493802598473291209.71S
2017-07-11 08:03:00.26023314997493802598473291209.71S
2017-07-11 08:03:00.26023314997493802598473291209.71S
2017-07-11 08:03:00.26023314997493802598473291209.71S
2017-07-11 08:03:00.26023314997493802598473291209.71S
2017-07-11 08:03:00.94879814997493809485465951209.72S
2017-07-11 08:03:00.94879814997493809485465951209.71S
2017-07-11 08:03:00.94910014997493809487981271209.72S
2017-07-11 08:03:00.94910014997493809487981271209.71S
2017-07-11 08:03:00.94923314997493809487984631209.71S
2017-07-11 08:03:00.94946614997493809491073051209.71B
2017-07-11 08:03:00.95085414997493809494661771209.72B
2017-07-11 08:03:00.95100914997493809496342611209.71B
2017-07-11 08:03:00.95100914997493809496342611209.72B
2017-07-11 08:03:00.95158814997493809506472211209.71B
2017-07-11 08:03:00.95158814997493809506472211209.71B
2017-07-11 08:03:00.95158814997493809506472211209.71B
2017-07-11 08:03:00.95170114997493809508641431209.71B
2017-07-11 08:03:01.28457614997493812842645651209.71S
2017-07-11 08:03:02.0858251499749382853877211209.73S
2017-07-11 08:03:02.0858251499749382853877211209.72S
2017-07-11 08:03:02.0858251499749382853877211209.71S


Discrepancies in the direction of transactions are shown in red