The genetic algorithm and its possible applications - page 4

 
Edgar Akhmadeev:

I wrote about this a long time ago, when I used frames in EA. I don't remember the exact point, but I don't seem to be getting all the frames (and the best results). I'll look for old posts and try to clarify.

But I distinctly remember it was clearly reproducible in my Expert Advisor - as soon as the number of overshoots exceeded a certain number and it was displayed in a scientific form, my genetics was broken. The important thing was not only that the variable had a big number of steps, but that the number of variables was big as well.

It all makes sense.

There is a problem with frames on "big" genetics.

We will fix it.

 
Slava:

What do you mean "it doesn't work at all correctly"?

How can we reproduce the malfunction?

https://www.mql5.com/ru/forum/321656/page17#comment_13569022

I don't check with frames anymore, I gave up.

But with regular genetics I checked it on the last build. Here's the result without "digit overflow".


After interruption:

2020.03.16 20:50:57.436 Tester  genetic optimization finished on pass 646 (of 160164854439975000)
2020.03.16 20:50:57.436 Statistics      optimization done in 8 minutes 36 seconds
2020.03.16 20:50:57.436 Statistics      shortest pass 0:00:00.002, longest pass 0:00:23.060, average pass 0:00:06.276

With "overflow":


After going through several interrupted passes by EA (checking that the input variables are correct), the genetics stopped forever. After interruption:

2020.03.16 20:59:08.235 Tester  genetic optimization finished on pass 18
2020.03.16 20:59:08.235 Statistics      optimization done in 6 minutes 10 seconds
2020.03.16 20:59:08.235 Statistics      shortest pass 0:00:00.003, longest pass 0:00:05.978, average pass 0:00:01.339

Not a single real result.

Tried Advisors/MAPSARSizeOptimized.ex5 as an example, it works. It's clear that the "digit overflow" and frames problem is only reproduced with my EA, but how to find the problem... Everything is very complicated there, OnTradeTransaction etc. I also removed frames. I can't show you the code and it's huge, about one megabyte. And cutting it down to a reproducible example would take a hopelessly long time. If I have time, I'll try to remove OnTradeTransaction and maybe some other tricks.

The fact is that if we don't exceed the number of passes, everything works fine.

And frames worked fine (without exceeding) until build 2286 inclusive.

Тестер стратегий MetaTrader 5: ошибки, баги, предложения по улучшению работы
Тестер стратегий MetaTrader 5: ошибки, баги, предложения по улучшению работы
  • 2019.10.16
  • www.mql5.com
В настоящий момент тестер стратегий MetaTrader 5 подвергается глубокой переработки командой MQ...
 
Igor Makanu:

...

one problem so far - GAs can start to converge around a small group of optimisation parameters after a while - this seems normal to me, all GAs work this way and this is a problem of their use

...

convergence to some extremum is a perfectly normal phenomenon for any optimization algorithm, there is no suggestion to assume that this section is not/is not global or local.

It is another matter that AO should have a mechanism that allows amplifying mutation percentage (or other equivalent in logic that allows amplifying expansion of search neighborhood), when starting to trample on some segment, extremum will be found better or not - it is unknown beforehand, but it is definitely necessary to search in other places.

for example, a criterion of maximal balance is set and several or many points may correspond to it in absolute terms but only some of them are of real value, i.e. balance 98756423 was reached at 1523 trades with a drawdown of 11% and the same balance was reached at 12 trades with a drawdown of 95%, what is the difference between the two?

 
Edgar Akhmadeev:

After going through several interrupted passes by the expert (checking that the input variables are correct), the genetics has stopped forever. After the interruption:

Not a single real result.

I don't remember if I wrote about it on the forum, but it's really a problem and it's not clear why it's implemented in MT. In idea, if expert returned error code "wrong parameters", tester is obliged to generate another instance instead, so that population is complete.

 
Stanislav Korotky:

I don't remember if I wrote about it on the forum, but it's really a problem and it's not clear why it's implemented in MT. In idea, if expert returned error code "wrong parameters", tester is obliged to generate another instance instead, so that population is complete.

If I return INIT_PARAMETERS_INCORRECT for all incorrect parameter combinations, there are too many of them and the generation will end with an error. So, I only return INIT_PARAMETERS_INCORRECT when a specific parameter is wrong, if it is out of range. And if the combination is wrong (one parameter should not exceed the other) I stop the pass, return INIT_SUCCEEDED and Custom = -N. This probably screws up the genetics, but I don't see any options. Or rather there is an option, getting rid of wrong combinations (in particular case - making one parameter a delta to another: v1=X, v2=Y+v1), but it's too strong mutagen. The two parameters will be rigidly linked, and if you change one, everything moves off. I've moved away from this option in favour of a fake result instead of an error.

 
Edgar Akhmadeev:

If you return INIT_PARAMETERS_INCORRECT for all wrong parameter combinations, there are too many of them and the generation will end with an error. So I only return INIT_PARAMETERS_INCORRECT when a particular parameter is wrong, if it is out of bounds. And if the combination is wrong (one parameter should not exceed the other) I stop the pass, return INIT_SUCCEEDED and Custom = -N. This probably screws up the genetics, but I don't see any options. Or rather there is an option, getting rid of wrong combinations (in particular case - making one parameter a delta to another: v1=X, v2=Y+v1), but it's too strong mutagen. The two parameters will be rigidly linked, and if you change one, everything moves off. I gave up this option in favor of a fake result instead of an error.

A good option is to return -DBL_MAX on invalid variant instead of error.

in general, i managed to implement external Ga while retaining all the charms of MT tester multicurrency fidelity, while being able to use all CPU cores and/or network agents, including cloud agents, while avoiding situations where agents are idle due to premature pass completions or understaffed Ga populations.

there is information, only tSSSSS.... that there are plans to implement several types of AOs in the regular optimizer, and even with parameters, but this is inaccurate.

 
Andrey Dik:

a good option is to return -DBL_MAX on invalid variant instead of error.

and if you return a random value - would it be worse for AO ?

 
Igor Makanu:

and if you return a rendered value, is it worse for the AO ?

much worse.

If the goal is to mess with AO's head, the best way is to return a random number.

 
Andrey Dik:

a good option is to return -DBL_MAX at invalid variant, instead of error.

This is too much, the graph is scaled so that you can't see the useful results. I return a slightly higher value than the worst case Custom. The main thing, though, is to set the right direction for improvement.

Igor Makanu:

And if you return a rendered value - will it be worse for AO ?

What's the point? The main thing is to set the right direction, so you have to show GA that he has shown the worst result here, not just a weak one.

 
Slava:

What do you mean "it doesn't work at all"?

How can I reproduce the incorrectness of the operation?

RemovedOnTradeTransaction, didn't help. I'll keep thinking.