Interpreting Optimization Reports and the signifigance of curve-fitted optimized back-tested performance of an EA in MT4 and MT5.

 

Hello All, 

 I am trying to best interpret the data below and attached. However, I realize that back-testing and optimization has it's benefits and its disadvantages. That is, results in back-testing or from optimizations do not always correlate to forward-testing results. Nonetheless, I would like to to determine if the following are "normal" of if they are "extraordinary" with regards to the performance of an EA that I have optimized and back-tested:

It goes without saying that this EA had considerable draw-downs although considerable expected payoff as well and lucrative return of 40,000%. Is it normal to find such an anomaly or outlier when completing an optimization which returned over 3500 passes and disregarded 4000 as insignificant?


Please see attached Optimization Chart and Statistics (yes I have cherry-picked it and its the best from the batch I have -and trying to determine its value).


Bars in test151837Ticks modelled27679334Modelling quality90.00%
Mismatched charts errors0




Initial deposit10000.00



Total net profit3579543.27Gross profit10568262.61Gross loss-6988719.33
Profit factor1.51Expected payoff613.88

Absolute drawdown377.31Maximal drawdown1069047.33 (51.11%)Relative drawdown85.57% (71414.06)

Total trades5831Short positions (won %)3097 (74.98%)Long positions (won %)2734 (76.12%)

Profit trades (% of total)4403 (75.51%)Loss trades (% of total)1428 (24.49%)
Largestprofit trade198048.00loss trade-161285.55
Averageprofit trade2400.24loss trade-4894.06
Maximumconsecutive wins (profit in money)32 (6143.61)consecutive losses (loss in money)11 (-32133.06)
Maximalconsecutive profit (count of wins)837800.26 (8)consecutive loss (count of losses)-648283.83 (5)
Averageconsecutive wins6consecutive losses2



I am hoping to hear from people experienced with interpreting such results in a meaningful way, I realize there is no holy grail in trading. I am just trying to get a fair value assessment for the "significance" of the above results. 


Thank you,

MT4HAT

 
MT4HAT:

Hello All, 

 I am trying to best interpret the data below and attached. However, I realize that back-testing and optimization has it's benefits and its disadvantages. That is, results in back-testing or from optimizations do not always correlate to forward-testing results. Nonetheless, I would like to to determine if the following are "normal" of if they are "extraordinary" with regards to the performance of an EA that I have optimized and back-tested:

It goes without saying that this EA had considerable draw-downs although considerable expected payoff as well and lucrative return of 40,000%. Is it normal to find such an anomaly or outlier when completing an optimization which returned over 3500 passes and disregarded 4000 as insignificant?


Please see attached Optimization Chart and Statistics (yes I have cherry-picked it and its the best from the batch I have -and trying to determine its value).


Bars in test151837Ticks modelled27679334Modelling quality90.00%
Mismatched charts errors0




Initial deposit10000.00



Total net profit3579543.27Gross profit10568262.61Gross loss-6988719.33
Profit factor1.51Expected payoff613.88

Absolute drawdown377.31Maximal drawdown1069047.33 (51.11%)Relative drawdown85.57% (71414.06)

Total trades5831Short positions (won %)3097 (74.98%)Long positions (won %)2734 (76.12%)

Profit trades (% of total)4403 (75.51%)Loss trades (% of total)1428 (24.49%)
Largestprofit trade198048.00loss trade-161285.55
Averageprofit trade2400.24loss trade-4894.06
Maximumconsecutive wins (profit in money)32 (6143.61)consecutive losses (loss in money)11 (-32133.06)
Maximalconsecutive profit (count of wins)837800.26 (8)consecutive loss (count of losses)-648283.83 (5)
Averageconsecutive wins6consecutive losses2



I am hoping to hear from people experienced with interpreting such results in a meaningful way, I realize there is no holy grail in trading. I am just trying to get a fair value assessment for the "significance" of the above results. 


Thank you,

MT4HAT

 You have probably curve-fitted the result (in other words, overoptimized the EA).

Do a simple test and attach this EA on a demo account for 3-4 weeks. I am quite sure that the results will be completely different.

The other thing is that such drawdowns are completely unacceptable for live trading. Remember that in live trading you will always eventually have a bigger largest loss trade or drawdown than during testing period. 

 
Enigma71fx:

 You have probably curve-fitted the result (in other words, overoptimized the EA).

Do a simple test and attach this EA on a demo account for 3-4 weeks. I am quite sure that the results will be completely different.

The other thing is that such drawdowns are completely unacceptable for live trading. Remember that in live trading you will always eventually have a bigger largest loss trade or drawdown than during testing period. 

Hi Enigma71fx,

Thanks for your reply. 

1) Yes indeed the result above was the most curve-fitted to the data and what interested me most where those specific parameters that yielded those results using this EA, even though they were curve-fitted. And using those same parameters, I applied them on other currency pairs and saw good results in many cases during a simple backtest. Therefore, the parameters are thought to be of value to some degree despite the parameters being over-optimized on the EUR/USD time frame used.

2) Naturally the results with forward testing will be different and this ratio is something we are already calculating as the EA has been running on multiple demos and live accounts (of my own)  already for nearly two months and the results so far are consistent to some degree with the testing although the risk has been adjusted lower.

3) The level of acceptable risk varies per trader and per strategy as 70% DD could be fine for someone looking to make 20,000% (who would even be willing to accept a 100% DD for such a high risk). Versus a trader who is targeting 15% for year and looking at a max DD of no more than 7-10%. So it's all relative..  

I appreciate your feedback, and just wanted to make it clear that I am well aware of these things you pointed out, and going back to my original question:

A) does the above result seem "common" to for an extensive  optimization (or over optimized EA)?  What I am simply trying to ask is: Is it normal during an optimization to get such a large return even when the strategy is curve-fitted as in this case?


Cheers~

P.S. All are welcome to reply.