Non-fitting system - main features
Forgive the amateur.
Thoughts aloud.
Systems that are optimised in the sense of the MT tester are flawed in nature.
Another thing is if some of the parameter calculations are done externally. or from data not available to MT.
.. And the branch is Super! Thanks!
And a manifesto!
I disagree. There's already a valuable seed in the idea.
It's not as if I'm rejecting MSUA methods.
Especially familiar with the authors in the 70s. and the science goes on
As you know...
A variant of amateurish performance:
Choose a short segment (say 3 weeks) 2-3 weeks before today for optimization, optimize. We run the best results through all available history and see. If the nature of the curve before and after optimization is similar to that of the optimization period, MTS has a right to refine it. I give more importance to the drawdown than to the profit factor when assessing it.
I believe that a TS is not a fitting if all external variables are logically chosen, not as a result of optimization. For example, Take Profit is not equal to const, but to volatility (ATR value) for N bars from the older TF. I believe we need to carry out optimization over a large interval, then analyze the area of the best values of the optimized parameter and find a logical reason why these values are exactly those. For example, as a result of optimization we got the best MoD and drawdown at TP {10;15} . Why is it so? Maybe this is ... (further rationale).
Do you think it is logical?
A variant of amateur performance:
We select a short period for optimization (say, 3 weeks) 2-3 weeks before today and optimize it. We run the best results through all available history and look at them. If the nature of the curve before and after optimization is similar to that of the optimization period, MTS has a right to refine it. I give more importance to the drawdown than to the profit factor when assessing it.
The drawdown can be anything on the short side. It can be large on the sample and small on the forward. A good system, it is as fast out of the drawdown as it is in it.
Actually, drawdowns are a result of dispersion. That is, a series of profitable and losing trades may be evenly distributed, or they may clump together. As they say, it is a matter of chance and does not depend on the robust TS. If Z-Score were adjustable, TS could be based on the naked MM. I.e. it would be no problem to select the most adequate MM for a stationary Z-score.
That is why I do not pay attention to the drawdown, and for optimization I choose a huge deposit size to avoid margin calls even in the most horrible situation. Otherwise the GA in the tester will not be kosher to select variants.
I believe that a TS is not a fitting if in it the values of all external variables are chosen logically, not as a result of optimization.
Makes sense - seconded. In general, the optimisation for non-different - is an alchemical flask. They make a TS out of a wild bunch of poorly understood indicators, input their parameters into the external tool and it's a shuffle. They will see how it works. But what if it is a grail? Perhaps, the alchemists were searching for the Philosopher's Stone.
Naturally, there is the probability (I forgot - it is a known figure) that a monkey, accidentally pressing the keys, will play "War and Peace".
===
Maybe we should look at the problem from this perspective: What is optimization for? What is it not for.
As you know,
A sliding window of optimisation... Although it does not give a complete guarantee either. But the robustness (in the sense of low sensitivity of the system to changes in the input parameters, and the price is the same input parameter) still increases. And, yes, the statistical significance of the window should be at least some.
It would be nice to formulate all the requirements for such a system. There is only one so far. I would like to hear more opinions. There must be a lot of collective experience. All you need is a willingness to share your knowledge. Otherwise it will be a clamour as usual.
If I had known better, I would have lived in Sochi.
So far we have managed to identify only the above-mentioned minimum requirements allowing us to knowingly eliminate those TPs unable to pass the forward tests.
Well, perhaps, there is one more reasonable remark of Leov saying that extremely inadequate expectation values at a constant lot suggest that it is a naked adjustment.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
As you know, many TS are well-fit on a representative sample and drain on forward tests. Such systems should not be used in trading.
Today I analyzed different TS I have and this is what I found out:
The signs of a working system is that it optimises almost equally on both sampling and double sampling with a constant lot. For example, we take 10,000 bars. We divide it approximately in half, i.e. by 5000 bars. Optimize it for 5000 bars. Then we apply it to 10 000. If the profit factor remains almost unchanged or has insignificantly changed (becomes independent of the sample length), the system is likely to pass the forward test. Naturally, trades should be about 600 - 1000 on 10 000 bars (300 - 500 on 5000 bars).
It turns out that if optimization of an TS significantly worsens as the test sample increases, the probability of a successful forward test is very low.
And some TS, for example, optimize on 300 trades, and on 600 the optimization yields nothing but deep drawdown and low expected payoff. Such a system can be immediately burned.
Actually, there is already an answer concerning the necessary and sufficient sample length. Sampling length must be such that optimization with a constant lot would yield almost the same profit factor both in the whole sampling and in its half.
----------------------------
Actually this means that a robust TS should be (nearly) stationary in different optimization areas with a constant lot by the following criteria: expected payoff, profit factor (percentage of profitable deals, if Take Profit and Stop Loss are fixed). And it can be absolutely non-stationary by Z-score.
I.e. one should not pay any attention to such Z-score relations as, for example, drawdown or number of simultaneous (without) losing trades. As correctly pointed out by Leov, a suspiciously small drawdown may hint at the fitting to the history. Simply put, a small drawdown is an even distribution of profitable and losing trades. And an even distribution in a non-stationary environment is nonsense.