Strategy Tester missing ticks

 

Dears,


I´ve executed my script in strategy tester mode, and found out that the same script, when executed in different times, is giving different results. The point is that, doing the same test several times, there are times I´ve got 3 trades, and there are time I´ve got 21 trades for the same back tested date (2021/05/11 / asset INDM21 / Brasil).

And now, looking deeper at the logs I´ve seen that there a difference in the logs bellow:

When it performs 21 trades:

QE      0       01:49:20.889    Tester  5679 Mb available, 70 blocks set for ticks generating

When it performs 3 trades:

EE      0       01:16:48.249    Tester  4265 Mb available, 52 blocks set for ticks generating


It seens the metatrader is downloading fewer data (less "ticks") for that day, but the log doesn´t leave any clue on why it is happening.


I wonder why would the MT download less ticks for a given day, and how I could overcome this.


One difference in the scenario where I´m getting fewer trades is that I`m executing the same script in five other instances of Metatrader (6 backtests in paralell, but each of them in a different date). How can I find out what is limiting the ticks in that day, and agai, how to overcome it?


Thanks in advance.

 

Not enough information to answer usefully.

Provide the full logs.

 
Alain Verleyen:

Not enough information to answer usefully.

Provide the full logs.

Below, I highlight the weird points.

When I have good result (21 trades) - 2 instances of metatrader:

CQ      0       01:49:20.374    Startup MetaTester 5 x64 build 2875 (02 Apr 2021)
CD      0       01:49:20.456    Server  MetaTester 5 started on 127.0.0.1:3000
FO      0       01:49:20.459    Startup initialization finished
HM      0       01:49:20.526    127.0.0.1       login (build 2875)
JK      0       01:49:20.561    Network 4372 bytes of account info loaded
RM      0       01:49:20.561    Network 1470 bytes of tester parameters loaded
DK      0       01:49:20.561    Network 3260 bytes of input parameters loaded
KO      0       01:49:20.870    Network 587951 bytes of symbols list loaded (77364 symbols)
DG      0       01:49:20.870    Tester  expert file added: Experts\rt_KeltnerHilo.ex5. 103050 bytes loaded
QE      0       01:49:20.889    Tester  5679 Mb available, 70 blocks set for ticks generating
LM      0       01:49:20.889    Tester  initial deposit 10000.00 BRL, leverage 1:100
RG      0       01:49:20.892    Tester  successfully initialized
LS      0       01:49:20.892    Network 525 Kb of total initialization data received
FI      0       01:49:20.892    Tester  Intel Core i7-3630QM  @ 2.40GHz, 8079 MB
ES      0       01:49:20.926    Symbols INDM21: symbol to be synchronized
KH      0       01:49:20.927    Symbols INDM21: symbol synchronized, 3720 bytes of symbol info received
RG      0       01:49:20.927    History INDM21: history synchronization started
FN      0       01:49:20.931    History INDM21: load 10 Kb of history data to synchronize in 0:00:00.003
OF      0       01:49:20.931    History INDM21: history synchronized from 2020.10.21 to 2021.05.12
HS      0       01:49:20.937    History INDM21,M1: history cache allocated for 11475 bars and contains 10190 bars from 2020.10.21 10:31 to 2021.05.10 18:01
IF      0       01:49:20.937    History INDM21,M1: history begins from 2020.10.21 10:31
PQ      0       01:49:20.937    Tester  INDM21,M1 (XPMT5-Demo): every tick generating
PG      0       01:49:20.937    Tester  INDM21,M1: testing of Experts\rt_KeltnerHilo.ex5 from 2021.05.11 00:00 to 2021.05.12 00:00 started with inputs:
MK      0       01:49:20.937    Tester    INP_ENSAIO_ID=
RS      0       01:49:20.937    Tester    INP_DATA_ENSAIO=
NP      0       01:49:20.937    Tester    INP_Remoto=
MF      0       01:49:20.937    Tester    INP_ST_MODE=
QN      0       01:49:20.937    Tester    ViewBars=0
OL      0       01:49:20.937    Tester    Interpolate=true
PG      0       01:49:20.972    rt_KeltnerHilo (INDM21,M1)      2021.05.11 00:00:00   Robo iniciado...
DP      0       01:49:20.973    Tester  program file added: \Indicators\Hilo.ex5. 18818 bytes loaded
KN      0       01:49:21.006    History INDM21,Daily: history cache allocated for 27 bars and contains 26 bars from 2020.10.21 00:00 to 2021.05.10 00:00
RN      0       01:49:21.006    History INDM21,Daily: history begins from 2020.10.21 00:00
HD      0       01:49:21.007    rt_KeltnerHilo (INDM21,M1)      2021.05.11 00:00:00   Operando IND. SL:150.0
KI      0       01:49:21.028    rt_KeltnerHilo (INDM21,M1)      2021.05.11 09:00:00   Iniciando dia. Resetando gatilhos
ML      0       01:49:21.029    Tester  program file added: \Indicators\keltner_channel315.ex5. 54482 bytes loaded
IN      0       01:49:27.193    rt_KeltnerHilo (INDM21,M1)      2021.05.11 09:27:06   LM estabilizada (Ok)
GJ      0       01:49:27.290    Trade   2021.05.11 09:29:04   buy limit 5 INDM21 at 121270 tp: 121360 (121270 / 121280 / 121270)
KK      0       01:49:27.292    rt_KeltnerHilo (INDM21,M1)      2021.05.11 09:29:04   sinal:TRADE_SINAL_COMPRA sinalDetectado:KC_CFG_SINAL_I prClose1:121270.0 prClose2:121265.0 tpCalc:121359.6195172827 sl:0.0
PL      0       01:49:27.343    Trade   2021.05.11 09:29:20   order [#2 buy limit 5 INDM21 at 121270] triggered
HH      0       01:49:27.343    Trades  2021.05.11 09:29:20   deal #2 buy 5 INDM21 at 121270 done (based on order #2)
OI      0       01:49:27.343    Trade   2021.05.11 09:29:20   deal performed [#2 buy 5 INDM21 at 121270]
FO      0       01:49:27.343    Trade   2021.05.11 09:29:20   order performed buy 5 at 121270 [#2 buy limit 5 INDM21 at 121270]
LR      0       01:49:27.465    Trade   2021.05.11 09:30:00   position modified [#2 buy 5 INDM21 121270 tp: 121350]
HN      0       01:49:27.611    Trade   2021.05.11 09:30:30   position modified [#2 buy 5 INDM21 121270 tp: 121345]
MI      0       01:49:27.747    Trade   2021.05.11 09:30:59   position modified [#2 buy 5 INDM21 121270 tp: 121350]
PD      0       01:49:27.755    Trade   2021.05.11 09:31:00   position modified [#2 buy 5 INDM21 121270 tp: 121335]
OP      0       01:49:27.797    Trade   2021.05.11 09:31:04   position modified [#2 buy 5 INDM21 121270 tp: 121330]
CL      0       01:49:27.917    Trade   2021.05.11 09:31:20   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121330]
NR      0       01:49:27.969    Trade   2021.05.11 09:31:27   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121335]
CP      0       01:49:28.225    Trade   2021.05.11 09:32:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121325]
DF      0       01:49:28.473    Trade   2021.05.11 09:32:32   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121320]
MD      0       01:49:28.679    Trade   2021.05.11 09:33:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310]
OJ      0       01:49:28.687    Trade   2021.05.11 09:33:03   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305]
RH      0       01:49:28.695    Trade   2021.05.11 09:33:07   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310]
HN      0       01:49:28.775    Trade   2021.05.11 09:33:42   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305]
RL      0       01:49:28.783    Trade   2021.05.11 09:33:45   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310]
GR      0       01:49:28.791    Trade   2021.05.11 09:33:49   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305]
EP      0       01:49:28.806    Trade   2021.05.11 09:34:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121290]
IJ      0       01:49:28.893    Trade   2021.05.11 09:34:18   stop loss triggered #2 buy 5 INDM21 121270 sl: 121120 tp: 121290 [#3 sell 5 INDM21 at 121120]
IL      0       01:49:28.893    Trades  2021.05.11 09:34:18   deal #3 sell 5 INDM21 at 121115 done (based on order #3)
HE      0       01:49:28.893    Trade   2021.05.11 09:34:18   deal performed [#3 sell 5 INDM21 at 121115]
GJ      0       01:49:28.893    Trade   2021.05.11 09:34:18   order performed sell 5 at 121115 [#3 sell 5 INDM21 at 121120]


This is the log with bad result  (3 trades) - 6 instances of metatrader in paralell:

NQ      0       01:16:47.626    Startup MetaTester 5 x64 build 2875 (02 Apr 2021)
FD      0       01:16:47.643    Server  MetaTester 5 started on 127.0.0.1:3008
QO      0       01:16:47.646    Startup initialization finished
PM      0       01:16:48.056    127.0.0.1       login (build 2875)
RK      0       01:16:48.088    Network 4372 bytes of account info loaded
NM      0       01:16:48.088    Network 1470 bytes of tester parameters loaded
HK      0       01:16:48.088    Network 3260 bytes of input parameters loaded
FO      0       01:16:48.240    Network 587951 bytes of symbols list loaded (77364 symbols)
QG      0       01:16:48.240    Tester  expert file added: Experts\rt_KeltnerHilo.ex5. 103050 bytes loaded
EE      0       01:16:48.249    Tester  4265 Mb available, 52 blocks set for ticks generating
FL      0       01:16:48.249    Tester  initial deposit 10000.00 USD, leverage 1:100
EF      0       01:16:48.251    Tester  successfully initialized
IL      0       01:16:48.251    Network 525 Kb of total initialization data received
OH      0       01:16:48.251    Tester  Intel Core i7-3630QM  @ 2.40GHz, 8079 MB
HS      0       01:16:48.268    Symbols INDM21: symbol to be synchronized
FI      0       01:16:48.269    Symbols INDM21: symbol synchronized, 3720 bytes of symbol info received
GF      0       01:16:48.512    History INDM21: load 10 Kb of history data to synchronize in 0:00:00.045
NQ      0       01:16:48.512    History INDM21: history synchronized from 2020.10.21 to 2021.05.12
GK      0       01:16:48.518    History INDM21,M1: history cache allocated for 11475 bars and contains 10190 bars from 2020.10.21 10:31 to 2021.05.10 18:01
JN      0       01:16:48.518    History INDM21,M1: history begins from 2020.10.21 10:31
HJ      0       01:16:48.519    Tester  INDM21,M1 (XPMT5-Demo): every tick generating
HN      0       01:16:48.519    Tester  INDM21,M1: testing of Experts\rt_KeltnerHilo.ex5 from 2021.05.11 00:00 to 2021.05.12 00:00 started with inputs:
LO      0       01:16:48.519    Tester    INP_ENSAIO_ID=32
GI      0       01:16:48.519    Tester    INP_DATA_ENSAIO=2021-05-11
JQ      0       01:16:48.519    Tester    INP_Remoto=0
QP      0       01:16:48.519    Tester    INP_ST_MODE=1
MD      0       01:16:48.519    Tester    ViewBars=0
EI      0       01:16:48.519    Tester    Interpolate=true
ON      0       01:16:48.551    rt_KeltnerHilo (INDM21,M1)      2021.05.11 00:00:00   Robo iniciado...
GN      0       01:16:48.567    Tester  program file added: \Indicators\Hilo.ex5. 18818 bytes loaded
GD      0       01:16:48.610    History INDM21,Daily: history cache allocated for 27 bars and contains 26 bars from 2020.10.21 00:00 to 2021.05.10 00:00
FI      0       01:16:48.610    History INDM21,Daily: history begins from 2020.10.21 00:00
KN      0       01:16:48.610    rt_KeltnerHilo (INDM21,M1)      2021.05.11 00:00:00   Operando IND. SL:150.0
OP      0       01:16:48.614    rt_KeltnerHilo (INDM21,M1)      2021.05.11 09:00:00   Iniciando dia. Resetando gatilhos
KF      0       01:16:48.646    Tester  program file added: \Indicators\keltner_channel315.ex5. 54482 bytes loaded
KD      0       01:16:53.444    rt_KeltnerHilo (INDM21,M1)      2021.05.11 09:27:06   LM estabilizada (Ok)
MP      0       01:16:53.523    Trade   2021.05.11 09:29:04   buy limit 5 INDM21 at 121270 tp: 121360 (121270 / 121280 / 121270)
GM      0       01:16:53.523    rt_KeltnerHilo (INDM21,M1)      2021.05.11 09:29:04   sinal:TRADE_SINAL_COMPRA sinalDetectado:KC_CFG_SINAL_I prClose1:121270.0 prClose2:121265.0 tpCalc:121359.6195172827 sl:0.0
RJ      0       01:16:53.557    Trade   2021.05.11 09:29:20   order [#2 buy limit 5 INDM21 at 121270] triggered
JF      0       01:16:53.557    Trades  2021.05.11 09:29:20   deal #2 buy 5 INDM21 at 121270 done (based on order #2)
QO      0       01:16:53.557    Trade   2021.05.11 09:29:20   deal performed [#2 buy 5 INDM21 at 121270]
DP      0       01:16:53.557    Trade   2021.05.11 09:29:20   order performed buy 5 at 121270 [#2 buy limit 5 INDM21 at 121270]
GI      0       01:16:53.649    Trade   2021.05.11 09:30:00   position modified [#2 buy 5 INDM21 121270 tp: 121350]
DD      0       01:16:53.762    Trade   2021.05.11 09:30:30   position modified [#2 buy 5 INDM21 121270 tp: 121345]
EG      0       01:16:53.863    Trade   2021.05.11 09:30:59   position modified [#2 buy 5 INDM21 121270 tp: 121350]
PS      0       01:16:53.868    Trade   2021.05.11 09:31:00   position modified [#2 buy 5 INDM21 121270 tp: 121335]
FN      0       01:16:53.896    Trade   2021.05.11 09:31:04   position modified [#2 buy 5 INDM21 121270 tp: 121330]
IE      0       01:16:53.989    Trade   2021.05.11 09:31:20   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121330]
KK      0       01:16:54.030    Trade   2021.05.11 09:31:27   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121335]
MI      0       01:16:54.215    Trade   2021.05.11 09:32:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121325]
GO      0       01:16:54.408    Trade   2021.05.11 09:32:32   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121320]
LM      0       01:16:54.561    Trade   2021.05.11 09:33:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310]
LS      0       01:16:54.566    Trade   2021.05.11 09:33:03   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305]
OQ      0       01:16:54.572    Trade   2021.05.11 09:33:07   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310]
MG      0       01:16:54.627    Trade   2021.05.11 09:33:42   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305]
OE      0       01:16:54.633    Trade   2021.05.11 09:33:45   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310]
QK      0       01:16:54.639    Trade   2021.05.11 09:33:49   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305]
II      0       01:16:54.650    Trade   2021.05.11 09:34:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121290]
QO      0       01:16:54.778    Trade   2021.05.11 09:34:34   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121285]   --> missing ticks comparing with the log above
FM      0       01:16:54.867    Trade   2021.05.11 09:35:00   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121270]
JS      0       01:16:55.077    Trade   2021.05.11 09:35:19   position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121265]

Entire log of the bad result is attached.

Files:
20210514.log  4234 kb
 
I can't answer to your issue but if you are worried about ticks, then use real ticks and not generated ticks.
 
Alain Verleyen:
I can't answer to your issue but if you are worried about ticks, then use real ticks and not generated ticks.

Alain, 

No, my concern is about the different results, it is not reliable, not even predictable. For me, it´s preferable to have less ticks getting always the same result, instead of run the same script twice, with the same parameters, and have different results. It sounds like a not reliable work.

In your feeling, a short time hardware high consumption in my machine (for example, few moments of high cpu usage, or memory) would make the platform to reduce the tick volume? Just to clarify, I´m just supposing it´s a tick generation misbehavior, not sure. But it can be another thing.

Another question is about the Metatrader infrastructure (where my metatrader is connected to), would it limit the ticks for some reason? Do you know if it is a responsabiity of the broker company (XP Investimentos / Brasil) or the Metratader Company?

In anyway, my concern is the reliability of the platform, not so much the tick volume.


I´ll try the real ticks mentioned by you.


Thank you.

 
mcavalcante: No, my concern is about the different results, it is not reliable, not even predictable. For me, it´s preferable to have less ticks getting always the same result, instead of run the same script twice, with the same parameters, and have different results. It sounds like a not reliable work.

In your feeling, a short time hardware high consumption in my machine (for example, few moments of high cpu usage, or memory) would make the platform to reduce the tick volume? Just to clarify, I´m just supposing it´s a tick generation misbehavior, not sure. But it can be another thing.

Another question is about the Metatrader infrastructure (where my metatrader is connected to), would it limit the ticks for some reason? Do you know if it is a responsabiity of the broker company (XP Investimentos / Brasil) or the Metratader Company?

In anyway, my concern is the reliability of the platform, not so much the tick volume.

I´ll try the real ticks mentioned by you.

I think that the virtual ticks are created locally by MetaTrader terminal and not the broker. The broker is only the supply of the OHLC data on which the virtual ticks are based. A extremely high CPU usage could theoretically be a a possible cause of less ticks being created, but I don't know if that would really be the case.

However, if you test against real tick data, which is downloaded form the broker only once and then reused on every test run or pass, then that should eliminate to problem of inconsistent data that you are having with the virtual ticks.

Can you please experiment with real tick data and see how it handles your situation?

EDIT: I personally always test against real ticks, and have never seen inconsistent behaviour in my testing.

 
Fernando Carreiro:

I think that the virtual ticks are created locally by MetaTrader terminal and not the broker. The broker is only the supply of the OHLC data on which the virtual ticks are based. A extremely high CPU usage could theoretically be a a possible cause of less ticks being created, but I don't know if that would really be the case.

However, if you test against real tick data, which is downloaded form the broker only once and then reused on every test run or pass, then that should eliminate to problem of inconsistent data that you are having with the virtual ticks.

Can you please experiment with real tick data and see how it handles your situation?

EDIT: I personally always test against real ticks, and have never seen inconsistent behaviour in my testing.

Fernando and Alain, thank you guys for the prompt response.

I´ve tested with the "real tick" mode, but no lucky. Besides, I used a python script to balance the load between three physical machines (multithreading), everything in parallel. Below, the test scenario and test summary.

Scenario:

Machine 1: Samsung Core i7 - 8 GB RAM (year 2013)

Machine 2: Vaio Core I7 10th gen - 8 GB RAM

Machine 3: PowerEdge Dell R410 - 2 x CPUs Xeon (year ~2014 gen) - 20 GBytes RAM

Back test date range: from 5/may/2021 to 12/may/2021 (one instance per date)

Asset: INDM21

Broker: XP Investimentos (Demo mode)

Script (EA): the same for all tests

Test summary:




It seems that the fewer the number of instances, the higher the test reliability (number of trades), regardless of the "model" parameter, what can bee seen in the test numbers 6 and 12. Anyway, still no clue where the root cause. Notice that, from 3 instances, the number of trades starts to decrease.

 
mcavalcante:

Alain, 

No, my concern is about the different results, it is not reliable, not even predictable. For me, it preferable to have less ticks getting always the same result, instead of run the same script twice, with the same parameters, and have different results. It sounds like a not reliable work.

In your feeling, a harware short time limitation (for example, few moments of high cpu usage, or memory) would make the platform to reduce the tick volume? Just to clarify, I´m just supposing it´s a tick generation misbehavior, not sure. But I can be another thing.

Another question is about the Metatrader infrastructure (in close, where my metatrader is connected to), would it limit the ticks for some reason? Is it responsabiity of the broker company (XP Investimentos / Brasil) or the Metratader Company?

In anyway, my concern is the reliability of the platform, not so much the tick volume.


I´ll try the real ticks mentioned by you.


Thank you.

I understand your concern and agree.

The data are provided by your broker, but the generated ticks and the reliability of the platform is Metaquotes job of course.

But unfortunately I can't answer about the issue, it could a lot of different things, including hardware or Internet issue.

 
mcavalcante: Fernando and Alain, thank you guys for the prompt response.

I´ve tested with the "real tick" mode, but no lucky. Besides, I used a python script to balance the load between three physical machines (multithreading), everything in parallel. Below, the test scenario and test summary.

Scenario:

Machine 1: Samsung Core i7 - 8 GB RAM (year 2013)

Machine 2: Vaio Core I7 10th gen - 8 GB RAM

Machine 3: PowerEdge Dell R410 - 2 x CPUs Xeon (year ~2014 gen) - 20 GBytes RAM

Back test date range: from 5/may/2021 to 12/may/2021 (one instance per date)

Asset: INDM21

Broker: XP Investimentos (Demo mode)

Script (EA): the same for all tests

Test summary:

It seems that the fewer the number of instances, the higher the test reliability (number of trades), regardless of the "model" parameter, what can bee seen in the test numbers 6 and 12. Anyway, still no clue where the root cause. Notice that, from 3 instances, the number of trades starts to decrease.

Please note, that according to the MetaQuotes specification guidelines for MetaTester, they recomend a minimum of 2GB per core for the tests, and one core per instance.

So, if you only have 8GB RAM, or only have 4 cores (not threads), then you should not be running more the a maximum of 4 instances on that PC.

Yet according to your table, you are running 6-8 instances at the same time on your Samsung i7 8GB. That is too many!

Since you have 20GB and dual CPU on your Dell, run the more instances on that one instead of your Samsung. Just don't go over the limit of at least 2GB per simultaneous instance and don't go over the number of instances per core.

 
Fernando Carreiro:

Please note, that according to the MetaQuotes specification guidelines for MetaTester, they recomend a minimum of 2GB per core for the tests, and one core per instance.

So, if you only have 8GB RAM, or only have 4 cores (not threads), then you should not be running more the a maximum of 4 instances on that PC.

Yet according to your table, you are running 6-8 instances at the same time on your Samsung i7 8GB. That is too many!

Since you have 20GB and dual CPU on your Dell, run the more instances on that one instead of your Samsung. Just don't go over the limit of at least 2GB per simultaneous instance and don't go over the number of instances per core.

Perfect! Thank you for the information, Fernando! That´s new for me. I´ll keep it in the instances management!

 
mcavalcante:

Below, I highlight the weird points.

When I have good result (21 trades) - 2 instances of metatrader:


This is the log with bad result  (3 trades) - 6 instances of metatrader in paralell:

Entire log of the bad result is attached.

I have serious doubts that a lack of resources could lead to missing ticks, it's not impossible though but unlikely. It should be checked in last resort.

But I just get that these tests comes from different MT5 instances, maybe even from different computers, right ?

So about the generated ticks, they are based on M1 data, you need to check that these data are the same between instances, if they are not, then the generated ticks will be different, can you check that ? Please also check if there are any difference in the generated ticks between tests on the same instance.