Help with testing past results as if they are live results to the expert advisor
Hi, I'm being told that the tester for MT4 cannot be used to test "past live" results.
This is what I mean:
I buy some EA and I optimize it for a period from 1 year + 6 months ago to now (2015.4.14). Now, I could start the EA but instead I wait a month.
It is now 2015.5.14. I did nothing to the settings, so the settings are still optimized for the period of 1 year + 7 months ago to 2015.4.14, last month. Because I waited, I now have the full tick history for the past month, from a month ago (2015.4.14) to now (2015.5.14). I test the EA with the optimized settings from the range of 1 year + 7 months ago to 2014.4.14 on the full tick history for the months time between 2015.4.14 and now, 2015.5.14.
When I do that, I see it would have made me profit if I didn't wait a month to put it into play. This means the EA was profitable from 2015.4.14 to 2015.5.14; nice! However, that's only one month on one currency pair, now I can repeat the process over multiple months across many pairs to see, on average, how reliable it is in turning a profit on a live account.
What is wrong with my reasoning? Is it wrong at all?
Thanks
Sounds like you trained your EA over a period of 1 year and 6 month. Then you did an out of sample test for a period of one month. You got a profitable result. That´s fine.
But is it reliable? May be. Go on testing to make your results more reliable:
- Test your EA over the training period in every possible setting of the variable parameters
- Test your EA over a longer out of sample period
- Test your EA with the best settings from the training period and with the worst settings from the training period in the out of sample period: The results should stay acceptable with both settings
- Go live on a small account. Deposit money you can afford to lose
- Go live on different brokers with small accounts
- See how it works and adapt your strategy
- Never find a single setting that is working in backtest mode and over a limited out of sample period and take it as reliable
- Always test the full spectrum of possibilities. Your EA must stay profitable over most of them. In live mode your EA will face favourable market conditions and unfavourable too
Sounds like you trained your EA over a period of 1 year and 6 month. Then you did an out of sample test for a period of one month. You got a profitable result. That´s fine.
But is it reliable? May be. Go on testing to make your results more reliable:
- Test your EA over the training period in every possible setting of the variable parameters
- Test your EA over a longer out of sample period
- Test your EA with the best settings from the training period and with the worst settings from the training period in the out of sample period: The results should stay acceptable with both settings
- Go live on a small account. Deposit money you can afford to lose
- Go live on different brokers with small accounts
- See how it works and adapt your strategy
- Never find a single setting that is working in backtest mode and over a limited out of sample period and take it as reliable
- Always test the full spectrum of possibilities. Your EA must stay profitable over most of them. In live mode your EA will face favourable market conditions and unfavourable too
Hey, yeah. I think you understand what I was doing. I was just verifying that the singular method would be good to try out multiple times over different time frames to judge the EA's performance. For example, I'd do the same as what I did in the year range of 2010-2011 instead and repeat again multiple times across different time frames (2008-2009, 2009-2010, etc). I figured if I do it enough times on different time frames and different currency pairs, I can verify that the EA is either good or bad. Already the EA settings/parameters are all accounted for when I do the optimization on the time frames.
I just felt like I was going crazy because people didn't understand what I was doing and were telling me it's an inaccurate method to judge the ability of an EA. Well, to me, it seemed like the best way to test an EA - because you simulate "live" results to the EA. Is it a bad method if done multiple times? It sounds like you think it isn't. The EA in particular I'm testing is a trend following EA, so testing it the month after optimization was necessary. In this case, I thought this kind of testing could really help me determine if it's worth its salt.
Thanks for the tips.
Hey, yeah. I think you understand what I was doing. I was just verifying that the singular method would be good to try out multiple times over different time frames to judge the EA's performance. For example, I'd do the same as what I did in the year range of 2010-2011 instead and repeat again multiple times across different time frames (2008-2009, 2009-2010, etc). I figured if I do it enough times on different time frames and different currency pairs, I can verify that the EA is either good or bad. Already the EA settings/parameters are all accounted for when I do the optimization on the time frames.
I just felt like I was going crazy because people didn't understand what I was doing and were telling me it's an inaccurate method to judge the ability of an EA. Well, to me, it seemed like the best way to test an EA - because you simulate "live" results to the EA. Is it a bad method if done multiple times? It sounds like you think it isn't. The EA in particular I'm testing is a trend following EA, so testing it the month after optimization was necessary. In this case, I thought this kind of testing could really help me determine if it's worth its salt.
Thanks for the tips.
You should also take care that you have enough trades in your out of sample testing period. If you follow a trend trading approach you might have to few trades in one month to get valid results. Take a longer testing period for your out of sample period. Take a period including different trends: up, down and most important no trend at all.
Btw. glad that I got your first post right ;)
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi, I'm being told that the tester for MT4 cannot be used to test "past live" results.
This is what I mean:
I buy some EA and I optimize it for a period from 1 year + 6 months ago to now (2015.4.14). Now, I could start the EA but instead I wait a month.
It is now 2015.5.14. I did nothing to the settings, so the settings are still optimized for the period of 1 year + 7 months ago to 2015.4.14, last month. Because I waited, I now have the full tick history for the past month, from a month ago (2015.4.14) to now (2015.5.14). I test the EA with the optimized settings from the range of 1 year + 7 months ago to 2014.4.14 on the full tick history for the months time between 2015.4.14 and now, 2015.5.14.
When I do that, I see it would have made me profit if I didn't wait a month to put it into play. This means the EA was profitable from 2015.4.14 to 2015.5.14; nice! However, that's only one month on one currency pair, now I can repeat the process over multiple months across many pairs to see, on average, how reliable it is in turning a profit on a live account.
What is wrong with my reasoning? Is it wrong at all?
Thanks