MT5 tester bug: same conditions, different results
- Look and compare single trades to find the reason for the difference.
- Don't assume that backtests reflect reality, they are maximally similar.
Hi Carl,
As I stated in the initial request, all the conditions are the same. It's just MT5 tester that gives sometimes different results with the same input.
Hi Carl,
As I stated in the initial request, all the conditions are the same. It's just MT5 tester that gives sometimes different results with the same input.
I made an experiment now - I backtested one EA two times with same input.
EA is uploaded on this post #7 (by source code).
It is the condition (EA with default settings):
---------------------------
First backtesting:
Second backtesting:
Hi Carl,
As I stated in the initial request, all the conditions are the same. It's just MT5 tester that gives sometimes different results with the same input.
That just mean the "conditions" are not the same, or your EA contains some "randomness".
There is no MT5 bug unless your prove it otherwise by providing all the information, parameters and code to reproduce it.
So, we can see about why those two backtesting results are slightly different (one from an other):
the number of ticks are different.
- It is not a bug. It is because the Metatrader is automatically loading the ticks which strategy tester needs for backtesting.
- Besides, it is not a weekend today. Today is Monday and forex market is open.
- And it also depends on the quality of the data/datafeed of the broker
(because backtesting with "every tick based on real ticks" is backtesting with the data provided by the broker).
It is not big problem for EAs which using classical trading systems, but it may be the problem for martingale and tick scalper EA, EA using open bar and close bar and high/low of open bar all together on the way as well-known Firebird EA, and some more.
--------------------
It is not a bug.
So, we can see about why those two backtesting results are slightly different (one from an other):
the number of ticks are different.
- It is not a bug. It is because the Metatrader is automatically loading the ticks which strategy tester needs for backtesting.
- Besides, it is not a weekend today. Today is Monday and forex market is open.
- And it also depends on the quality of the data/datafeed of the broker
(because backtesting with "every tick based on real ticks" is backtesting with the data provided by the broker).
It is not big problem for EAs which using classical trading systems, but it may be the problem for martingale and tick scalper EA, EA using open bar and close bar and high/low of open bar all together on the way as well-known Firebird EA, and some more.
--------------------
It is not a bug.
Hi Sergey,
Yes, my EA works on every tick. However, I use time range with latest date in the past (e.g. 01.01.2016-05.05.2023) so the MT should not add automatically new ticks to the testing volume.
Please find below the example: I ran optimization with change in only magic number. All the results but one are the same, one is wrong.
What's wrong here? Why one result is different? It's only magic number, nothing more.
If I re-run this one (double-click and run the single test) this will show normal result as others (958 trades). But here in optimization results it is obviously wrong.
Yes, my EA works on every tick. However, I use time range with latest date in the past (e.g. 01.01.2016-05.05.2023) so the MT should not add automatically new ticks to the testing volume.
From 2016 to 2023?
I think - some ticks had been adding to strategy tester from Metatrader.
Because if I backtested simple (simple means: no tickscalping EA, no martingale, no pyramiding, and so on) from
2 of June till 12 of June (this year), and Metatrader added more than 20K ticks for few minutes to strategy tester after first backtest so
the ticks should be added in case of you are backtesting from 2016 ...
I did not check the optimization now but I think ... if you are repeating same optimization (with same condition) many times so
strategy tester can use the previous optimization results to speed the process up so
"too much optimization is not good" (especially for the EAs working with ticks such as TickSpalper EA, PolyfitScalper EA and more).
--------------------
Besides, it also depends on EA.
Do you remember old Firebird EA? (it was uploaded to some very old threads here by source codes with many versions, with trading results and with the settings proposed).
This EA was coded to work on open bar together with high/low of open bar and close/previous bar as well (all together) - M15 as I remember.
Yes, it was MT4 ... and the backtesting results were different and depended oon the qualuty of the history/datafeed and when we start backtesting and more and more (and which broker for example).
--------------------
So, if the next (same but next) backtesting result is different from the previous one so it means the following:
the condition for this EA were not same one (not for you; it may be same condition for you but not same for the EA).
From 2016 to 2023?
I think - some ticks had been adding to strategy tester from Metatrader.
Because if I backtested simple (simple means: no tickscalping EA, no martingale, no pyramiding, and so on) from
2 of June till 12 of June (this year), and Metatrader added more than 20K ticks for few minutes to strategy tester after first backtest so
the ticks should be added in case of you are backtesting from 2016 ...
I did not check the optimization now but I think ... if you are repeating same optimization (with same condition) many times so
strategy tester can use the previous optimization results to speed the process up so
"too much optimization is not good" (especially for the EAs working with ticks such as TickSpalper EA, PolyfitScalper EA and more).
--------------------
Besides, it also depends on EA.
Do you remember old Firebird EA? (it was uploaded to some very old threads here by source codes with many versions, with trading results and with the settings proposed).
This EA was coded to work on open bar together with high/low of open bar and close/previous bar as well (all together) - M15 as I remember.
Yes, it was MT4 ... and the backtesting results were different and depended oon the qualuty of the history/datafeed and when we start backtesting and more and more (and which broker for example).
--------------------
So, if the next (same but next) backtesting result is different from the previous one so it means the following:
the condition for this EA were not same one (not for you; it may be same condition for you but not same for the EA).
Hi!
I think it may be a lot of particularities, and I am using backtesting to be sure that EA is working and can open the trades,
about how stop loss and trailing stop is working, and to have some ideas that EA may be profitable in some cases with some brokers and so on.
I know that there are the people who use backtesting result as a proof that EA is profitable ... but I am not one of them ...
I just explained everything in theoretical points of view according to my experience (as the ideas for example).
I think it may be a lot of particularities, and I am using backtesting to be sure that EA is working and can open the trades,
about how stop loss and trailing stop is working, and to have some ideas that EA may be profitable in some cases with some brokers and so on.
I know that there are the people who use backtesting result as a proof that EA is profitable ... but I am not one of them ...
Got it, thanks!
Will continue to search for an explanation.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi all!
I found a few similar posts (questions) with no solution: when run the same backtest two times (one after another) , the MT5 tester sometimes gives different results.
The conditions (real ticks, time range, etc.) are the same.
I run the backtests on my laptop. The latest stable build of MT5 terminal.
Can anyone help with this?