Trader's self-deception: distrust of forwards. - page 7

 
Дмитрий:

Selection of indicators by forward profitability is in 99% of cases a fit. Only in this case the fitting is not only by the parameters of the training series, but also by the parameters of the forward series. Fit, but fit to a longer series

I agree. Only I didn't mean it. Profitable forward doesn't mean that the code is good or bad. Profitable forward means that the market had such a property as inertia - it moved according to patterns identified in the backtest.

 
Yuri_Evseenkov:

I agree. But that's not what I meant. Profitable forward does not mean that the code is good or bad. Profitable forward means that the market had such property as inertia - it moved according to the patterns identified in the backtest.

This conclusion can only be drawn if back and forward results are at least approximately the same
 
Дмитрий:
This conclusion can only be drawn if back and forward results are at least approximately the same
The more the back and forward differ, the lower the strength of market inertia and/or the lower the quality of performance or code optimisation.
 

How can you be so ignorant?)

Don't read the forums about analysis and verification of test results, but read smart books. The vast majority of questions in the thread will disappear at once.

 
Мастер над криптомонетой:

How can you be so ignorant?)

Don't read the forums about analysis and verification of test results, but read smart books. The vast majority of questions in the thread will disappear at once.

I don't seem to have seen any books about checking test results. Maybe you can advise which one.
 
Дмитрий:

Selection of indicators by forward profitability is in 99% of cases a fit. Only in this case the fitting is not only by the parameters of the training series, but also by the parameters of the forward series. Fitting, but fitting on a longer series

Let's clarify the terms.

When we fit anything to historical data (logics, indicators, variables for these indicators, etc.), in general, it can be called "fitting" in the "good" and "bad" senses of this word. But it's better to say that there is a process of CELECTING of code parameters or its properties - i.e.repeated selection of the best combinations of one and another. As a rule of thumb, "fitting" is still a half-joint selection of the most impressive values of variables, which are obtained by repeated tests on the same part of history and shown for thesame part and not for neighbouring ones. On the forward, however, there is no multiple selection.

Forward is just a one-time test on the history which hasn't been optimized. All we have to do on the forward is either be sad or happy with his result or draw some conclusions, but we cannot influence him, because it is against his principle. In order to influence the forward, we have to go back to the original piece and redo something there. The fundamental difference is not that by adding a forward we get a "longer row", but that our code hits the segment of the forward's history for the first time. There is no "lengthening" of the optimized segment. Therefore the term "fitting" in the sense of "selection for beauty" cannot be applied here in principle. But are forwarders really used for selection? Why else, of course. But this selection, as a rule, is not aimed at selection of momentary code variables. It is the code itself - its logic and quality - which is its purpose.

Phrases like "Selection of indicators according to the profitability of a forward is in 99% of cases a fitting" are a striking example of self-deception, when at some point people just replace the concepts, probably subconsciously justifying their unwillingness to more deeply and laboriously understand the shortcomings of the code. I.e. when you change indicators, not their settings, trying to achieve a more beautiful forward - this is a normal selection, but only under the condition of observing the principle of one-time checking for the forward. It's hardly a fitting process.

 
Youri Tarshecki:

Let's get the terms straight.

When we fit anything to historical data (logic, indicators, variables for these indicators, etc.) - it can be called "fitting", both in "good" and "bad" sense of the word. But it's better to say that there is a process of CELECTING of code parameters or its properties - i.e.repeated selection of the best combinations of one and another. As a rule of thumb, "fitting" is still a half-joint selection of the most impressive values of variables, which are obtained by repeated tests on the same part of history and shown for thesame part and not for neighbouring ones. On the forward, however, there is no multiple selection.

Forward is just a one-time test on the history which hasn't been optimized. All we have to do on the forward is either be sad or happy with his result or draw some conclusions, but we cannot influence him, because it is against his principle. In order to influence the forward, we have to go back to the original piece and redo something there. The fundamental difference is not that by adding a forward we get a "longer row", but that our code hits the segment of the forward's history for the first time. There is no "lengthening" of the optimized segment. Therefore the term "fitting" in the sense of "selection for beauty" cannot be applied here in principle. But are forwarders really used for selection? Why else, of course. But this selection, as a rule, is not aimed at selection of momentary code variables. It is the code itself - its logic and quality - which is its purpose.

Phrases like "Selection of indicators according to the profitability of a forward is in 99% of cases a fitting" are a striking example of self-deception, when at some point people just replace the concepts, probably subconsciously, justifying their unwillingness to more deeply and laboriously understand the shortcomings of the code. I.e. when you change indicators trying to get a nicer forward - this is the normal selection, but only under the condition of observing the principle of one-time validation for the forward. You can hardly call this process adjustment.

It is a lot of nothing.

You take an indicator, change its settings in 100 ways, 20 of them give positive MO in the backtest.

You run these 20 on forward - 5 of them will give positive values on forward. You select out of 5 indicator settings that give maximum forward MO.

So - this is all the same kind of fitting, only not only on backtest, but also on forward.

 
Yuri_Evseenkov:

I agree. But that's not what I meant. Profitable forward does not mean that the code is good or bad. Profitable forward means that the market had such a property as inertia - it moved according to the patterns identified in the backtest.

Once again, the most important thing is the SUSTAINABILITY of the results. Not just a profitable forward, but at least approximately the same results if the sample is representative, complete and sufficient.
 
Дмитрий:

It's a lot and nothing.

You take an indicator, change its settings 100 ways, 20 of them give positive MO on the back test.

Run these 20 on forward - 5 of them give positive MO on forward. You select out of 5 indicator settings that give maximum forward MO.

So - this is all the same kind of fitting, only not only on backtest, but also on forward.

And once again, in terms of terms ANY selection from the results on history is not forward, it is optimization. The fact that those choices you made from forward doesn't change or even improve anything (how will you get 5 forwards when history ends on the present one?). I.e. you first violated the principle of forward checking and then claimed it's a fit and it's bad. Of course it's a fit, but a fit from a smaller selection of indicator settings. First you don't violate the principle and then we can discuss something. Without forward dynamics you cannot say anything about properties of the indicator itself to be stable or not, since stability is repeatability and you have only one forward. Besides, you were first talking about the selection of INDICATORS, not the variants of their settings. A real forward check is precisely designed to find out the robustness of the code, not the settings, just by comparing the result of a number of forwards, not a bunch of settings obtained by stomping on one place. I.e. comparing different forwards (the only one for a particular site) is the result of comparing many unoptimised sites, not one with itself.
 
Youri Tarshecki:
And again on the terms ANY selection from the results on history is no longer a forward but an optimisation. The fact that those choices you made from forward doesn't change or even improve anything (how will you get 5 forwards, when history ends on the present one?). I.e. you first violated the principle of forward checking and then claimed it's a fit and it's bad. Of course it's a fit, but a fit from a smaller selection of indicator settings. First you don't violate the principle and then we can discuss something. Without having forward dynamics you can't say anything about the properties of the indicator itself to be stable or not. In addition, you were first talking about the selection of INDICATORS, not the variants of their settings. The real forward checking is exactly intended to find out the stability of the code, not the settings, just by comparing the result of a number of forwards, not a bunch of settings obtained by stomping on one place. I.e. comparing different forwards (the only one for a particular site) is the result of comparing many unoptimised sites, not one with itself.

How forward checking is done - we start from the basics.

You take a sample, divide it in a certain proportion, e.g. 80/20, into back and forward.

And how do you think forward checking is done - we take the entire sample up to the present moment and accept it as back? And then the results are real time tested forwards as the quotes come in? And how long does it take you to forward??????