Strategy Tester missing ticks
Not enough information to answer usefully.
Provide the full logs.
Not enough information to answer usefully.
Provide the full logs.
Below, I highlight the weird points.
When I have good result (21 trades) - 2 instances of metatrader:
CQ 0 01:49:20.374 Startup MetaTester 5 x64 build 2875 (02 Apr 2021) CD 0 01:49:20.456 Server MetaTester 5 started on 127.0.0.1:3000 FO 0 01:49:20.459 Startup initialization finished HM 0 01:49:20.526 127.0.0.1 login (build 2875) JK 0 01:49:20.561 Network 4372 bytes of account info loaded RM 0 01:49:20.561 Network 1470 bytes of tester parameters loaded DK 0 01:49:20.561 Network 3260 bytes of input parameters loaded KO 0 01:49:20.870 Network 587951 bytes of symbols list loaded (77364 symbols) DG 0 01:49:20.870 Tester expert file added: Experts\rt_KeltnerHilo.ex5. 103050 bytes loaded QE 0 01:49:20.889 Tester 5679 Mb available, 70 blocks set for ticks generating LM 0 01:49:20.889 Tester initial deposit 10000.00 BRL, leverage 1:100 RG 0 01:49:20.892 Tester successfully initialized LS 0 01:49:20.892 Network 525 Kb of total initialization data received FI 0 01:49:20.892 Tester Intel Core i7-3630QM @ 2.40GHz, 8079 MB ES 0 01:49:20.926 Symbols INDM21: symbol to be synchronized KH 0 01:49:20.927 Symbols INDM21: symbol synchronized, 3720 bytes of symbol info received RG 0 01:49:20.927 History INDM21: history synchronization started FN 0 01:49:20.931 History INDM21: load 10 Kb of history data to synchronize in 0:00:00.003 OF 0 01:49:20.931 History INDM21: history synchronized from 2020.10.21 to 2021.05.12 HS 0 01:49:20.937 History INDM21,M1: history cache allocated for 11475 bars and contains 10190 bars from 2020.10.21 10:31 to 2021.05.10 18:01 IF 0 01:49:20.937 History INDM21,M1: history begins from 2020.10.21 10:31 PQ 0 01:49:20.937 Tester INDM21,M1 (XPMT5-Demo): every tick generating PG 0 01:49:20.937 Tester INDM21,M1: testing of Experts\rt_KeltnerHilo.ex5 from 2021.05.11 00:00 to 2021.05.12 00:00 started with inputs: MK 0 01:49:20.937 Tester INP_ENSAIO_ID= RS 0 01:49:20.937 Tester INP_DATA_ENSAIO= NP 0 01:49:20.937 Tester INP_Remoto= MF 0 01:49:20.937 Tester INP_ST_MODE= QN 0 01:49:20.937 Tester ViewBars=0 OL 0 01:49:20.937 Tester Interpolate=true PG 0 01:49:20.972 rt_KeltnerHilo (INDM21,M1) 2021.05.11 00:00:00 Robo iniciado... DP 0 01:49:20.973 Tester program file added: \Indicators\Hilo.ex5. 18818 bytes loaded KN 0 01:49:21.006 History INDM21,Daily: history cache allocated for 27 bars and contains 26 bars from 2020.10.21 00:00 to 2021.05.10 00:00 RN 0 01:49:21.006 History INDM21,Daily: history begins from 2020.10.21 00:00 HD 0 01:49:21.007 rt_KeltnerHilo (INDM21,M1) 2021.05.11 00:00:00 Operando IND. SL:150.0 KI 0 01:49:21.028 rt_KeltnerHilo (INDM21,M1) 2021.05.11 09:00:00 Iniciando dia. Resetando gatilhos ML 0 01:49:21.029 Tester program file added: \Indicators\keltner_channel315.ex5. 54482 bytes loaded IN 0 01:49:27.193 rt_KeltnerHilo (INDM21,M1) 2021.05.11 09:27:06 LM estabilizada (Ok) GJ 0 01:49:27.290 Trade 2021.05.11 09:29:04 buy limit 5 INDM21 at 121270 tp: 121360 (121270 / 121280 / 121270) KK 0 01:49:27.292 rt_KeltnerHilo (INDM21,M1) 2021.05.11 09:29:04 sinal:TRADE_SINAL_COMPRA sinalDetectado:KC_CFG_SINAL_I prClose1:121270.0 prClose2:121265.0 tpCalc:121359.6195172827 sl:0.0 PL 0 01:49:27.343 Trade 2021.05.11 09:29:20 order [#2 buy limit 5 INDM21 at 121270] triggered HH 0 01:49:27.343 Trades 2021.05.11 09:29:20 deal #2 buy 5 INDM21 at 121270 done (based on order #2) OI 0 01:49:27.343 Trade 2021.05.11 09:29:20 deal performed [#2 buy 5 INDM21 at 121270] FO 0 01:49:27.343 Trade 2021.05.11 09:29:20 order performed buy 5 at 121270 [#2 buy limit 5 INDM21 at 121270] LR 0 01:49:27.465 Trade 2021.05.11 09:30:00 position modified [#2 buy 5 INDM21 121270 tp: 121350] HN 0 01:49:27.611 Trade 2021.05.11 09:30:30 position modified [#2 buy 5 INDM21 121270 tp: 121345] MI 0 01:49:27.747 Trade 2021.05.11 09:30:59 position modified [#2 buy 5 INDM21 121270 tp: 121350] PD 0 01:49:27.755 Trade 2021.05.11 09:31:00 position modified [#2 buy 5 INDM21 121270 tp: 121335] OP 0 01:49:27.797 Trade 2021.05.11 09:31:04 position modified [#2 buy 5 INDM21 121270 tp: 121330] CL 0 01:49:27.917 Trade 2021.05.11 09:31:20 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121330] NR 0 01:49:27.969 Trade 2021.05.11 09:31:27 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121335] CP 0 01:49:28.225 Trade 2021.05.11 09:32:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121325] DF 0 01:49:28.473 Trade 2021.05.11 09:32:32 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121320] MD 0 01:49:28.679 Trade 2021.05.11 09:33:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310] OJ 0 01:49:28.687 Trade 2021.05.11 09:33:03 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305] RH 0 01:49:28.695 Trade 2021.05.11 09:33:07 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310] HN 0 01:49:28.775 Trade 2021.05.11 09:33:42 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305] RL 0 01:49:28.783 Trade 2021.05.11 09:33:45 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310] GR 0 01:49:28.791 Trade 2021.05.11 09:33:49 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305] EP 0 01:49:28.806 Trade 2021.05.11 09:34:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121290] IJ 0 01:49:28.893 Trade 2021.05.11 09:34:18 stop loss triggered #2 buy 5 INDM21 121270 sl: 121120 tp: 121290 [#3 sell 5 INDM21 at 121120] IL 0 01:49:28.893 Trades 2021.05.11 09:34:18 deal #3 sell 5 INDM21 at 121115 done (based on order #3) HE 0 01:49:28.893 Trade 2021.05.11 09:34:18 deal performed [#3 sell 5 INDM21 at 121115] GJ 0 01:49:28.893 Trade 2021.05.11 09:34:18 order performed sell 5 at 121115 [#3 sell 5 INDM21 at 121120]
This is the log with bad result (3 trades) - 6 instances of metatrader in paralell:
NQ 0 01:16:47.626 Startup MetaTester 5 x64 build 2875 (02 Apr 2021) FD 0 01:16:47.643 Server MetaTester 5 started on 127.0.0.1:3008 QO 0 01:16:47.646 Startup initialization finished PM 0 01:16:48.056 127.0.0.1 login (build 2875) RK 0 01:16:48.088 Network 4372 bytes of account info loaded NM 0 01:16:48.088 Network 1470 bytes of tester parameters loaded HK 0 01:16:48.088 Network 3260 bytes of input parameters loaded FO 0 01:16:48.240 Network 587951 bytes of symbols list loaded (77364 symbols) QG 0 01:16:48.240 Tester expert file added: Experts\rt_KeltnerHilo.ex5. 103050 bytes loaded EE 0 01:16:48.249 Tester 4265 Mb available, 52 blocks set for ticks generating FL 0 01:16:48.249 Tester initial deposit 10000.00 USD, leverage 1:100 EF 0 01:16:48.251 Tester successfully initialized IL 0 01:16:48.251 Network 525 Kb of total initialization data received OH 0 01:16:48.251 Tester Intel Core i7-3630QM @ 2.40GHz, 8079 MB HS 0 01:16:48.268 Symbols INDM21: symbol to be synchronized FI 0 01:16:48.269 Symbols INDM21: symbol synchronized, 3720 bytes of symbol info received GF 0 01:16:48.512 History INDM21: load 10 Kb of history data to synchronize in 0:00:00.045 NQ 0 01:16:48.512 History INDM21: history synchronized from 2020.10.21 to 2021.05.12 GK 0 01:16:48.518 History INDM21,M1: history cache allocated for 11475 bars and contains 10190 bars from 2020.10.21 10:31 to 2021.05.10 18:01 JN 0 01:16:48.518 History INDM21,M1: history begins from 2020.10.21 10:31 HJ 0 01:16:48.519 Tester INDM21,M1 (XPMT5-Demo): every tick generating HN 0 01:16:48.519 Tester INDM21,M1: testing of Experts\rt_KeltnerHilo.ex5 from 2021.05.11 00:00 to 2021.05.12 00:00 started with inputs: LO 0 01:16:48.519 Tester INP_ENSAIO_ID=32 GI 0 01:16:48.519 Tester INP_DATA_ENSAIO=2021-05-11 JQ 0 01:16:48.519 Tester INP_Remoto=0 QP 0 01:16:48.519 Tester INP_ST_MODE=1 MD 0 01:16:48.519 Tester ViewBars=0 EI 0 01:16:48.519 Tester Interpolate=true ON 0 01:16:48.551 rt_KeltnerHilo (INDM21,M1) 2021.05.11 00:00:00 Robo iniciado... GN 0 01:16:48.567 Tester program file added: \Indicators\Hilo.ex5. 18818 bytes loaded GD 0 01:16:48.610 History INDM21,Daily: history cache allocated for 27 bars and contains 26 bars from 2020.10.21 00:00 to 2021.05.10 00:00 FI 0 01:16:48.610 History INDM21,Daily: history begins from 2020.10.21 00:00 KN 0 01:16:48.610 rt_KeltnerHilo (INDM21,M1) 2021.05.11 00:00:00 Operando IND. SL:150.0 OP 0 01:16:48.614 rt_KeltnerHilo (INDM21,M1) 2021.05.11 09:00:00 Iniciando dia. Resetando gatilhos KF 0 01:16:48.646 Tester program file added: \Indicators\keltner_channel315.ex5. 54482 bytes loaded KD 0 01:16:53.444 rt_KeltnerHilo (INDM21,M1) 2021.05.11 09:27:06 LM estabilizada (Ok) MP 0 01:16:53.523 Trade 2021.05.11 09:29:04 buy limit 5 INDM21 at 121270 tp: 121360 (121270 / 121280 / 121270) GM 0 01:16:53.523 rt_KeltnerHilo (INDM21,M1) 2021.05.11 09:29:04 sinal:TRADE_SINAL_COMPRA sinalDetectado:KC_CFG_SINAL_I prClose1:121270.0 prClose2:121265.0 tpCalc:121359.6195172827 sl:0.0 RJ 0 01:16:53.557 Trade 2021.05.11 09:29:20 order [#2 buy limit 5 INDM21 at 121270] triggered JF 0 01:16:53.557 Trades 2021.05.11 09:29:20 deal #2 buy 5 INDM21 at 121270 done (based on order #2) QO 0 01:16:53.557 Trade 2021.05.11 09:29:20 deal performed [#2 buy 5 INDM21 at 121270] DP 0 01:16:53.557 Trade 2021.05.11 09:29:20 order performed buy 5 at 121270 [#2 buy limit 5 INDM21 at 121270] GI 0 01:16:53.649 Trade 2021.05.11 09:30:00 position modified [#2 buy 5 INDM21 121270 tp: 121350] DD 0 01:16:53.762 Trade 2021.05.11 09:30:30 position modified [#2 buy 5 INDM21 121270 tp: 121345] EG 0 01:16:53.863 Trade 2021.05.11 09:30:59 position modified [#2 buy 5 INDM21 121270 tp: 121350] PS 0 01:16:53.868 Trade 2021.05.11 09:31:00 position modified [#2 buy 5 INDM21 121270 tp: 121335] FN 0 01:16:53.896 Trade 2021.05.11 09:31:04 position modified [#2 buy 5 INDM21 121270 tp: 121330] IE 0 01:16:53.989 Trade 2021.05.11 09:31:20 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121330] KK 0 01:16:54.030 Trade 2021.05.11 09:31:27 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121335] MI 0 01:16:54.215 Trade 2021.05.11 09:32:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121325] GO 0 01:16:54.408 Trade 2021.05.11 09:32:32 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121320] LM 0 01:16:54.561 Trade 2021.05.11 09:33:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310] LS 0 01:16:54.566 Trade 2021.05.11 09:33:03 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305] OQ 0 01:16:54.572 Trade 2021.05.11 09:33:07 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310] MG 0 01:16:54.627 Trade 2021.05.11 09:33:42 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305] OE 0 01:16:54.633 Trade 2021.05.11 09:33:45 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121310] QK 0 01:16:54.639 Trade 2021.05.11 09:33:49 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121305] II 0 01:16:54.650 Trade 2021.05.11 09:34:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121290] QO 0 01:16:54.778 Trade 2021.05.11 09:34:34 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121285] --> missing ticks comparing with the log above FM 0 01:16:54.867 Trade 2021.05.11 09:35:00 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121270] JS 0 01:16:55.077 Trade 2021.05.11 09:35:19 position modified [#2 buy 5 INDM21 121270 sl: 121120 tp: 121265]
Entire log of the bad result is attached.
I can't answer to your issue but if you are worried about ticks, then use real ticks and not generated ticks.
Alain,
No, my concern is about the different results, it is not reliable, not even predictable. For me, it´s preferable to have less ticks getting always the same result, instead of run the same script twice, with the same parameters, and have different results. It sounds like a not reliable work.
In your feeling, a short time hardware high consumption in my machine (for example, few moments of high cpu usage, or memory) would make the platform to reduce the tick volume? Just to clarify, I´m just supposing it´s a tick generation misbehavior, not sure. But it can be another thing.
Another question is about the Metatrader infrastructure (where my metatrader is connected to), would it limit the ticks for some reason? Do you know if it is a responsabiity of the broker company (XP Investimentos / Brasil) or the Metratader Company?
In anyway, my concern is the reliability of the platform, not so much the tick volume.
I´ll try the real ticks mentioned by you.
Thank you.
In your feeling, a short time hardware high consumption in my machine (for example, few moments of high cpu usage, or memory) would make the platform to reduce the tick volume? Just to clarify, I´m just supposing it´s a tick generation misbehavior, not sure. But it can be another thing.
Another question is about the Metatrader infrastructure (where my metatrader is connected to), would it limit the ticks for some reason? Do you know if it is a responsabiity of the broker company (XP Investimentos / Brasil) or the Metratader Company?
In anyway, my concern is the reliability of the platform, not so much the tick volume.
I´ll try the real ticks mentioned by you.
I think that the virtual ticks are created locally by MetaTrader terminal and not the broker. The broker is only the supply of the OHLC data on which the virtual ticks are based. A extremely high CPU usage could theoretically be a a possible cause of less ticks being created, but I don't know if that would really be the case.
However, if you test against real tick data, which is downloaded form the broker only once and then reused on every test run or pass, then that should eliminate to problem of inconsistent data that you are having with the virtual ticks.
Can you please experiment with real tick data and see how it handles your situation?
EDIT: I personally always test against real ticks, and have never seen inconsistent behaviour in my testing.
I think that the virtual ticks are created locally by MetaTrader terminal and not the broker. The broker is only the supply of the OHLC data on which the virtual ticks are based. A extremely high CPU usage could theoretically be a a possible cause of less ticks being created, but I don't know if that would really be the case.
However, if you test against real tick data, which is downloaded form the broker only once and then reused on every test run or pass, then that should eliminate to problem of inconsistent data that you are having with the virtual ticks.
Can you please experiment with real tick data and see how it handles your situation?
EDIT: I personally always test against real ticks, and have never seen inconsistent behaviour in my testing.
Fernando and Alain, thank you guys for the prompt response.
I´ve tested with the "real tick" mode, but no lucky. Besides, I used a python script to balance the load between three physical machines (multithreading), everything in parallel. Below, the test scenario and test summary.
Scenario:
Machine 1: Samsung Core i7 - 8 GB RAM (year 2013)
Machine 2: Vaio Core I7 10th gen - 8 GB RAM
Machine 3: PowerEdge Dell R410 - 2 x CPUs Xeon (year ~2014 gen) - 20 GBytes RAM
Back test date range: from 5/may/2021 to 12/may/2021 (one instance per date)
Asset: INDM21
Broker: XP Investimentos (Demo mode)
Script (EA): the same for all tests
Test summary:
It seems that the fewer the number of instances, the higher the test reliability (number of trades), regardless of the "model" parameter, what can bee seen in the test numbers 6 and 12. Anyway, still no clue where the root cause. Notice that, from 3 instances, the number of trades starts to decrease.
Alain,
No, my concern is about the different results, it is not reliable, not even predictable. For me, it preferable to have less ticks getting always the same result, instead of run the same script twice, with the same parameters, and have different results. It sounds like a not reliable work.
In your feeling, a harware short time limitation (for example, few moments of high cpu usage, or memory) would make the platform to reduce the tick volume? Just to clarify, I´m just supposing it´s a tick generation misbehavior, not sure. But I can be another thing.
Another question is about the Metatrader infrastructure (in close, where my metatrader is connected to), would it limit the ticks for some reason? Is it responsabiity of the broker company (XP Investimentos / Brasil) or the Metratader Company?
In anyway, my concern is the reliability of the platform, not so much the tick volume.
I´ll try the real ticks mentioned by you.
Thank you.
I understand your concern and agree.
The data are provided by your broker, but the generated ticks and the reliability of the platform is Metaquotes job of course.
But unfortunately I can't answer about the issue, it could a lot of different things, including hardware or Internet issue.
I´ve tested with the "real tick" mode, but no lucky. Besides, I used a python script to balance the load between three physical machines (multithreading), everything in parallel. Below, the test scenario and test summary.
Scenario:
Machine 1: Samsung Core i7 - 8 GB RAM (year 2013)
Machine 2: Vaio Core I7 10th gen - 8 GB RAM
Machine 3: PowerEdge Dell R410 - 2 x CPUs Xeon (year ~2014 gen) - 20 GBytes RAM
Back test date range: from 5/may/2021 to 12/may/2021 (one instance per date)
Asset: INDM21
Broker: XP Investimentos (Demo mode)
Script (EA): the same for all tests
Test summary:
It seems that the fewer the number of instances, the higher the test reliability (number of trades), regardless of the "model" parameter, what can bee seen in the test numbers 6 and 12. Anyway, still no clue where the root cause. Notice that, from 3 instances, the number of trades starts to decrease.Please note, that according to the MetaQuotes specification guidelines for MetaTester, they recomend a minimum of 2GB per core for the tests, and one core per instance.
So, if you only have 8GB RAM, or only have 4 cores (not threads), then you should not be running more the a maximum of 4 instances on that PC.
Yet according to your table, you are running 6-8 instances at the same time on your Samsung i7 8GB. That is too many!
Since you have 20GB and dual CPU on your Dell, run the more instances on that one instead of your Samsung. Just don't go over the limit of at least 2GB per simultaneous instance and don't go over the number of instances per core.
Please note, that according to the MetaQuotes specification guidelines for MetaTester, they recomend a minimum of 2GB per core for the tests, and one core per instance.
So, if you only have 8GB RAM, or only have 4 cores (not threads), then you should not be running more the a maximum of 4 instances on that PC.
Yet according to your table, you are running 6-8 instances at the same time on your Samsung i7 8GB. That is too many!
Since you have 20GB and dual CPU on your Dell, run the more instances on that one instead of your Samsung. Just don't go over the limit of at least 2GB per simultaneous instance and don't go over the number of instances per core.
Perfect! Thank you for the information, Fernando! That´s new for me. I´ll keep it in the instances management!
Below, I highlight the weird points.
When I have good result (21 trades) - 2 instances of metatrader:
This is the log with bad result (3 trades) - 6 instances of metatrader in paralell:
Entire log of the bad result is attached.
I have serious doubts that a lack of resources could lead to missing ticks, it's not impossible though but unlikely. It should be checked in last resort.
But I just get that these tests comes from different MT5 instances, maybe even from different computers, right ?
So about the generated ticks, they are based on M1 data, you need to check that these data are the same between instances, if they are not, then the generated ticks will be different, can you check that ? Please also check if there are any difference in the generated ticks between tests on the same instance.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Dears,
I´ve executed my script in strategy tester mode, and found out that the same script, when executed in different times, is giving different results. The point is that, doing the same test several times, there are times I´ve got 3 trades, and there are time I´ve got 21 trades for the same back tested date (2021/05/11 / asset INDM21 / Brasil).
And now, looking deeper at the logs I´ve seen that there a difference in the logs bellow:
When it performs 21 trades:
When it performs 3 trades:
It seens the metatrader is downloading fewer data (less "ticks") for that day, but the log doesn´t leave any clue on why it is happening.
I wonder why would the MT download less ticks for a given day, and how I could overcome this.
One difference in the scenario where I´m getting fewer trades is that I`m executing the same script in five other instances of Metatrader (6 backtests in paralell, but each of them in a different date). How can I find out what is limiting the ticks in that day, and agai, how to overcome it?
Thanks in advance.