Tester: How Not to Fall into Optimization Traps?

New comment
 

New article How Not to Fall into Optimization Traps? has been published:

The article describes the methods of how to understand the tester optimization results better. It also gives some tips that help to avoid "harmful optimization".

Author: Shashev Sergei

 

Excellent article.


One question, I've done some 2 parameters optimizations before but I don't get the green graph from metatrader. How can I get it? Or you just downloaded the data and did it in excel?


Thank you.

 
lococo:

Excellent article.


One question, I've done some 2 parameters optimizations before but I don't get the green graph from metatrader. How can I get it? Or you just downloaded the data and did it in excel?


Thank you.

If you right click on the line graph and "un-check" the "2D Surface" option you'll see the green bars. If you added a lot of parameter options/ranges then you'll see a lot of little green boxes.
 
This is definitely a fantastic article. I was working on a self optimized algorithm. But it got too complicated due to the number of parameters. Now, i guess i must reduce it for better performance.
 

Hi,

As I understood, one should optimize regularly in order to take into account market changes (volat, etc...) --> ie. optimize during one year, then trade 3 months, and so every three months....

We would therefore need a way to select the optimal parameter in a "industrial" way, like with precise rules, not a graph observation.

For example: take the 25 best profits, with some restriction on # of trades, P/R,... and take the average parameter value of these 25 best-profit-combinations?

I would like to find a automated way to choose the right parameters combination in Excel...

If I was not clear, let me know, I will explain it more clearly.

See ya all.

Best

 
I didn't understand how to to choose the parameters when there are more than 2
 

" Combining them, get the best solution – (5,4,80). "


I don't understand how to to choose best  solution with 3  parameters Or more ? can anybody explain this ?
 


Optimization traps are situations where you optimize for something that is not actually important, or you optimize too much and end up making things worse. Here are some tips on how to avoid optimization traps:

  • Define your goals clearly. What are you trying to optimize for? Once you know your goals, you can start to identify the metrics that are important to you.
  • Understand the trade-offs. Any optimization will have trade-offs. For example, if you optimize for speed, you may sacrifice accuracy. It is important to understand these trade-offs before you start optimizing.
  • Use a variety of metrics. Don't just focus on one metric. Use a variety of metrics to get a more complete picture of your optimization efforts.
  • Be careful of overfitting. Overfitting occurs when you optimize your model too much and it starts to fit the noise in your data instead of the underlying signal. To avoid overfitting, you can use cross-validation or other techniques.
  • Monitor your results. Once you have implemented your optimization changes, monitor your results to see if they are actually improving. If they are not, you may need to make adjustments to your optimization strategy.

By following these tips, you can avoid optimization traps and ensure that your optimization efforts are successful.

Here are some additional tips to avoid optimization traps:

  • Start with a baseline. Before you start optimizing, it is important to have a baseline understanding of your current performance. This will help you to measure the effectiveness of your optimization efforts.
  • Be patient. Optimization can take time. Don't expect to see results overnight.
  • Be flexible. As you learn more about your system, you may need to adjust your optimization strategy. Be prepared to make changes as needed.

By following these tips, you can increase your chances of avoiding optimization traps and achieving your optimization goals.

New comment