Retraining - page 2
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
And I thought "blah blah" was judged not by people, but by what they write.
I don't usually reply to irritated posts, but I will personally reply to you.
So, what I mean by "blah blah".
CASE 1.
Suppose you have written a robot, found its parameters with a TESTER and then, without changing them, tried it on new quotes areas and preferably on other symbols and TFs. We got a positive result.
The question is: To what extent may we trust the result? Will it go down or not?
As an answer to the question, look at the "Signals" tab here on the site and see that the signals with the profitability higher than the bank deposit and with the lifetime over a year are quite rare: there are not even a hundred out of several thousand. Thus, I personally conclude that the development of a stable TS is an art, and the tester is one of the tools for developing such a stable TS. But the most important conclusion: the tester does not guarantee a profit loss.
This is what I considered the case when the tester confirmed the TS.
CASE 2
This is a more interesting case in the sense of "blah-blah" - this is when the tester gives a negative result. What to do? After all, the tester doesn't give any ideas in what direction to dig and most importantly doesn't answer the question: why a negative result. The tester states the fact - it's bad. Then comes the use of a universal method called the TYKA METHOD or blah-blah, which usually means an intuitive change of indicators in an attempt to get the set of indicators that will lead us to the case 1 described above.
Are there any ideas, tools, that can make the process of case 2 more meaningful and make sure that results obtained in the tester will be obtained in the future on the real account?
Yes there are, I gave a link to a parallel thread. It deals with such tools for analyzing the initial set of input data (predictors), which will not lead to retraining (over-fitting) of the TS. These are not "blah-blah" - these are specific tools with mathematical justification of their effectiveness, without intuitive grail search, and after finding such a grail, draining the depo.
If a forward test is used, "overtraining" will be visible by degradation of results outside the optimization period. Unfortunately, MT does not provide any built-in methods of comparing the results of optimization and forward testing, i.e. it is suggested to do it manually (by eye), by external programs or scripts at one's own discretion.
...
CASE 2.
The tester states a fact - bad. Then comes the application of a universal method called the TYKA METHOD or blah-blah, which usually involves intuitively changing a set of indicators in an attempt to get a set of indicators that will lead us to case 1, described above.
Are there any ideas, tools, that can make the process of case 2 more meaningful and make sure that results obtained in the tester will be obtained in the future on the real account?
Yes there are, I gave a link to a parallel thread. It deals with such tools for analyzing the initial set of input data (predictors), which will not lead to retraining (over-fitting) of the TS. These are not "blah-blah" - these are specific tools with mathematical justification of their effectiveness, without intuitive grail search, and after finding such a grail, draining the depo.
What if this poke method is automated! Interesting thoughts in this direction. Indicators are changing, so are their interactions, the interactions are represented as separate functions, so the number of parameters may change, only the simplest optimization according to them occurs. That is why I am interested in the question asked in this thread, a more universal approach, which would not depend on strategy parameters. The branch you are proposing has completely different objectives. But if you show the applicability of those methods to this problem, you are welcome.
The term "retraining" itself is silly, designed to justify the inoperability of the EA itself and completely loses its meaning with a volking forward. Whether a variable is overtrained or undertrained, in fact, you can't tell from the degradation. It can only be seen when comparing the forward results under different optimization and test conditions. Both history depth and forward step are selected in each case personally and then it is already visible what is over and what is undertrained.
In this situation the term "overtraining" is applicable, the training refers not only to parameter setting but also to strategy building.
I would like to estimate the probability of overtraining, so that the search system can at least "bypass" the doubtful places, if just compare with a forward history plot, it may happen that the system perceives two history plots(training, test) as one training plot. Listening to ideas)).
In this situation the term "overtraining" is applicable, the training refers not only to parameter setting but also to strategy building.
I would like to estimate the probability of overtraining, so that the search system can at least "bypass" the doubtful places, if just compare with a forward history plot, it may happen that the system perceives two history plots(training, test) as one training plot. Listening to ideas)).
The term "overtraining" or "overoptimization" was coined by tweakers because, indeed, the more you optimize an EA on a particular part of history and the more optimization parameters there are, the better the results in training.
For some reason, there is an opinion that there is an inverse correlation with test results, i.e. the less, the better, but my experience shows that this is not true. The test result does not depend on the number of optmization passes, but on the quality of the Expert Advisor itself and the optimal choice of training and test runs. The results of schemes where the story is subjected to minimal optimization are not the best I have seen.
For example, under the wolfkin forward scheme in a 4 month training - 1 month test, each interval of the history is optimized 4 times, but under the 2 month training - 2 month test scheme only one time. Is it too much or too little? Who knows! We just need to look at the test results. If the sum of forwards is better, then that's the best option.
When building a strategy, the term "overtraining" is all the more inapplicable, since we are only comparing the test result under the same training conditions to select a variant of the code. And how optimally these conditions are chosen is not so important, the main thing is that they are the same for all variants of the code. Otherwise the choice is meaningless.
This term is used quite often:"Overtraining,overfitting is an undesirable phenomenon that occurs inprecedent-based learning tasks when the probability of error of the trained algorithm on thetest sample objectsis significantly higher than the average error onthe training sample."
According to the definition, the sameness of the training conditions does not prevent the applicability of the term to our problem.
Re-training is like the word counter-revolution. Then why teach at all if you have to retrain. And if it makes sense to retrain, then you know the floating limits for retraining, otherwise it's the same roulette in the end. And since there are decision points as to when/how often/which parameters... need to be retrained, why not put this understanding into the logic of the training/algorithm itself from the outset.
Moving away from price analysis to feedback analysis between training and reality (price). Same thing from a different angle.
This term is used quite often:"Overtraining,overfitting is an undesirable phenomenon that occurs inprecedent-based learning tasks when the probability of error of the trained algorithm on thetest sample objectsis significantly higher than the average error onthe training sample."
According to the definition, the sameness of the training conditions does not prevent the applicability of the term to our problem.
So when building a strategy, the task is different. not precedents, not optimization, but writing code.
Besides, I don't agree with this definition. According to it, if we're not optimizing at all and the probability of failure on the test is greater than on the training (which is a common thing) - it will also be considered overoptimization. And what does it mean substantially? Twice? By ten times?
Just compare learning to memory. It's not like you're wondering why you need to remember if you have to forget anyway. The problem is that EAs generally don't have long term and short term memory separately. In addition, the evaluation of their performance is very primitive. Therefore, we should ideally teach every single permenent separately (which I'm trying to do, by the way), and check it not on training segments, but on test segments.
Don't you forget, it is just that the memory of the section where the training took place is transformed into the memory of the results of the training from that section. It is the same as if we filtered an area and later used the filtered information for analysis, then filtered it and used another and so on, while there is a connection between the nature of filtering/areas in which these filters were performed.
Nothing is forgotten there, the same analysis of history from a different angle. Whatever you want to call it, over-learning/over-optimisation/adaptation/fitting/over-fitting.