Interpreting Optimization Reports and the signifigance of curve-fitted optimized back-tested performance of an EA in MT4 and MT5.
Hello All,
I am trying to best interpret the data below and attached. However, I realize that back-testing and optimization has it's benefits and its disadvantages. That is, results in back-testing or from optimizations do not always correlate to forward-testing results. Nonetheless, I would like to to determine if the following are "normal" of if they are "extraordinary" with regards to the performance of an EA that I have optimized and back-tested:
It goes without saying that this EA had considerable draw-downs although considerable expected payoff as well and lucrative return of 40,000%. Is it normal to find such an anomaly or outlier when completing an optimization which returned over 3500 passes and disregarded 4000 as insignificant?
Please see attached Optimization Chart and Statistics (yes I have cherry-picked it and its the best from the batch I have -and trying to determine its value).
| Bars in test | 151837 | Ticks modelled | 27679334 | Modelling quality | 90.00% |
| Mismatched charts errors | 0 | ||||
| Initial deposit | 10000.00 | ||||
| Total net profit | 3579543.27 | Gross profit | 10568262.61 | Gross loss | -6988719.33 |
| Profit factor | 1.51 | Expected payoff | 613.88 | ||
| Absolute drawdown | 377.31 | Maximal drawdown | 1069047.33 (51.11%) | Relative drawdown | 85.57% (71414.06) |
| Total trades | 5831 | Short positions (won %) | 3097 (74.98%) | Long positions (won %) | 2734 (76.12%) |
| Profit trades (% of total) | 4403 (75.51%) | Loss trades (% of total) | 1428 (24.49%) | ||
| Largest | profit trade | 198048.00 | loss trade | -161285.55 | |
| Average | profit trade | 2400.24 | loss trade | -4894.06 | |
| Maximum | consecutive wins (profit in money) | 32 (6143.61) | consecutive losses (loss in money) | 11 (-32133.06) | |
| Maximal | consecutive profit (count of wins) | 837800.26 (8) | consecutive loss (count of losses) | -648283.83 (5) | |
| Average | consecutive wins | 6 | consecutive losses | 2 | |
I am hoping to hear from people experienced with interpreting such results in a meaningful way, I realize there is no holy grail in trading. I am just trying to get a fair value assessment for the "significance" of the above results.
Thank you,
MT4HAT
You have probably curve-fitted the result (in other words, overoptimized the EA).
Do a simple test and attach this EA on a demo account for 3-4 weeks. I am quite sure that the results will be completely different.
The other thing is that such drawdowns are completely unacceptable for live trading. Remember that in live trading you will always eventually have a bigger largest loss trade or drawdown than during testing period.
You have probably curve-fitted the result (in other words, overoptimized the EA).
Do a simple test and attach this EA on a demo account for 3-4 weeks. I am quite sure that the results will be completely different.
The other thing is that such drawdowns are completely unacceptable for live trading. Remember that in live trading you will always eventually have a bigger largest loss trade or drawdown than during testing period.
Hi Enigma71fx,
Thanks for your reply.
1) Yes indeed the result above was the most curve-fitted to the data and what interested me most where those specific parameters that yielded those results using this EA, even though they were curve-fitted. And using those same parameters, I applied them on other currency pairs and saw good results in many cases during a simple backtest. Therefore, the parameters are thought to be of value to some degree despite the parameters being over-optimized on the EUR/USD time frame used.
2) Naturally the results with forward testing will be different and this ratio is something we are already calculating as the EA has been running on multiple demos and live accounts (of my own) already for nearly two months and the results so far are consistent to some degree with the testing although the risk has been adjusted lower.
3) The level of acceptable risk varies per trader and per strategy as 70% DD could be fine for someone looking to make 20,000% (who would even be willing to accept a 100% DD for such a high risk). Versus a trader who is targeting 15% for year and looking at a max DD of no more than 7-10%. So it's all relative..
I appreciate your feedback, and just wanted to make it clear that I am well aware of these things you pointed out, and going back to my original question:
A) does the above result seem "common" to for an extensive optimization (or over optimized EA)? What I am simply trying to ask is: Is it normal during an optimization to get such a large return even when the strategy is curve-fitted as in this case?
Cheers~
P.S. All are welcome to reply.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hello All,
I am trying to best interpret the data below and attached. However, I realize that back-testing and optimization has it's benefits and its disadvantages. That is, results in back-testing or from optimizations do not always correlate to forward-testing results. Nonetheless, I would like to to determine if the following are "normal" of if they are "extraordinary" with regards to the performance of an EA that I have optimized and back-tested:
It goes without saying that this EA had considerable draw-downs although considerable expected payoff as well and lucrative return of 40,000%. Is it normal to find such an anomaly or outlier when completing an optimization which returned over 3500 passes and disregarded 4000 as insignificant?
Please see attached Optimization Chart and Statistics (yes I have cherry-picked it and its the best from the batch I have -and trying to determine its value).
I am hoping to hear from people experienced with interpreting such results in a meaningful way, I realize there is no holy grail in trading. I am just trying to get a fair value assessment for the "significance" of the above results.
Thank you,
MT4HAT