Preventing overfitting - page 2

 
RaptorUK:
Isn't it obvious ?  it's because they are all curve fitted.  

OK, I admit my response was an extreme one,  but I think it would be better to take my extreme,  no optimization,  over the other extreme of optimize by as many inputs as possible even if it makes no logical sense to use them for optimization.

Look at it this way,  optimization is just that,  optimizing something that is already good to make it even better.  It's not meant to make a losing strategy win . . .

I suspected that, but I believed that they did it because there was some value to the approach. And I have to agree that no optimization is better to the self-mesmerizing approach of overfitting...

Ubzen:

....

 First of all, thanks for your thorough answer. I have a strong background in data mining, and the most usual approach (leaving apart things like cross fold validation) is something similar to your fourth bullet, so my point of view may be distorted towards something similar. I loved your example, since it's something that I also expected (your first suggested figure) and didn't find. Another common result in data mining is that optimizing your training set (backtest) also optimizes the validation set (forward test) up to a point where they start diverging, but I've seen much wilder results applied to forex series.

 
Thanks, I did use the search before I opened this thread without finding them, I suppose it's a matter of which words you use to refer to very similar things :)
 
jlwarrior:
Thanks, I did use the search before I opened this thread without finding them, I suppose it's a matter of which words you use to refer to very similar things :)
Yup. Hope those are useful. Share if you have found anything.
 

Restricting the amount of Expert variables has certainly worked for me (I only optimise 2 variables). The other big one is to optimise over multiple currency pairs AT THE SAME TIME - i.e. the two variables being optimised are optimised over multiple currency pairs and the total P/L (or whatever other fitness function you like to use) from all these currency pair trades is used to determine the suitability (or not) or a particular Expert configuration.

My world view is that "fine tuning" Experts for a particular currency pair is a waste of time. If an Expert isn't profitable or at least breaks even over multiple pairs using the same values for the configuration variables, I throw it away. 

This approach does require quite a bit more programming. The current MT5 platform doesn't support this as a high-level function so it's a matter of writing an Expert that is optimised on one currency pair but accesses and trades all the other pairs within that same Expert during the optimisation. Fortunately, MT5 allows you to access data, place trades etc for any Symbol you want within a single Expert - otherwise, this obviously wouldn't be possible.

 
cowil:

Restricting the amount of Expert variables has certainly worked for me (I only optimise 2 variables). The other big one is to optimise over multiple currency pairs AT THE SAME TIME - i.e. the two variables being optimised are optimised over multiple currency pairs and the total P/L (or whatever other fitness function you like to use) from all these currency pair trades is used to determine the suitability (or not) or a particular Expert configuration.

My world view is that "fine tuning" Experts for a particular currency pair is a waste of time. If an Expert isn't profitable or at least breaks even over multiple pairs using the same values for the configuration variables, I throw it away. 

This approach does require quite a bit more programming. The current MT5 platform doesn't support this as a high-level function so it's a matter of writing an Expert that is optimised on one currency pair but accesses and trades all the other pairs within that same Expert during the optimisation. Fortunately, MT5 allows you to access data, place trades etc for any Symbol you want within a single Expert - otherwise, this obviously wouldn't be possible.

Interesting post, maybe the approach described here can be used for multi-symbols optimization.
 
angevoyageur:
Interesting post, maybe the approach described here can be used for multi-symbols optimization.

Hi,

Thanks for the info - that's an interesting way of doing things as well. However, the period still ends up being an optimisation parameter so each optimisation iteration is still only carried out using one Period. What would be interesting to try is optimising an Expert on one currency pair, but that each iteration trades ALL the Periods of that currency pair using the same configuration variables and sums the results of each period's trades as the fitness variable. Basically the same as I'm doing with multiple currencies but then with multiple periods. 

In fact, you could optimise using multiple currency pairs AND Periods, although then each iteration could be trading a large number of different Symbol/Period combinations at the same time. Would make for interesting programming. :)

 
Simon Gniadkowski:
Isn't it obvious ?  it's because they are all curve fitted.  

OK, I admit my response was an extreme one,  but I think it would be better to take my extreme,  no optimization,  over the other extreme of optimize by as many inputs as possible even if it makes no logical sense to use them for optimization.

Look at it this way,  optimization is just that,  optimizing something that is already good to make it even better.  It's not meant to make a losing strategy win . . .
It could also be used to torture test something that might be questionable, and see how it really holds up.  Like the saying goes, you can't fix it if you don't know what's broken.  Optimization and testing like that can be used to check and see what could be improved, what is good as it stands.
 

Optimizing to get to a better set of fixed variables?

Maybe it's a better idea to have it calculate it's own best variables, what i am trying to convey is that you do not need the back tester, at all you can just build it into the software.

 
Marco vd Heijden:

Optimizing to get to a better set of fixed variables?

Maybe it's a better idea to have it calculate it's own best variables, what i am trying to convey is that you do not need the back tester, at all you can just build it into the software.

That is a possibility, but that being doable is another issue altogether.  It really depends on the underlying skill of the programmer trying to make it work.  Speaking from personal experience, I have great ideas on a lot of things, and I know exactly what it is I want them to do (in the case of EAs for example), but where I fall down is the knowledge of how to code it to make those things actually happen.