Where is the line between fitting and actual patterns? - page 50
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Yeah, I'm aware of that. I'm not going to do that. But it is useless to bother with it until the computer distinguishes the cat from the cat according to external features.And when it does, the market will look completely different... :о)
"Tail aftereffect" is such a common phenomenon in optimized TS that it's boring to even demonstrate it.
The only way not to actually detect it is to chase not profit but to prove that a pattern cannot exist.
Good luck.
I thought and thought and came up with: Doesn't the search for patterns go a little "deeper" than fitting? Let me explain with the example of an experiment I am conducting. There is a neural network that I use. Usually I train it for an hour or two. I usually get 30-50% of results and then successfully pass forward testing (1/6 of a training period). Problem to identify them, simple guessing works against me in this case. I tried to teach 4 hours - sadly only about 20% of successful forwarders. Conclusion: re-training, a fitting has gone. Frustrated and.... decided to continue training. 12 hours, same 20%. 24 hours, again 20%, but I've noticed that they pass the forwards only slightly worse than the training period. Before, even the best results showed some noticeable drop in performance, and this 20% was actually the best training results of all the sets. They used to be spread out roughly evenly amongst the others. It got interesting. Spent almost 72 hours of training to date. Oh miracle. out of the top 200 results, more than 123 successfully pass forward testing. You can try to play the guessing game profitably.
Increasing the learning time led to a gradual improvement in the result at Sample period. Which is natural. The 72 hour result outperformed the 2 hour result by a factor of more than 4. The OOS result also eventually improved, but there was no gradualism to speak of, there was an obvious failure.
__________
What does this tell you?
A sad and obvious option: about imperfect training methodology and overloaded NS) Classic GA is a bit heavy for teaching NS, but yes there is a lot to tweak... The NS has a lot of freedom, the NS inputs are also clearly redundant and not all informative enough, the NS has "ruled them out" quite a lot in the learning process. We could also experiment with the architecture.
Optimistic and perhaps premature: A TS capable of identifying the regularities with its "gut", is simply "obliged" to do so, in order to achieve its best result in the period of training. Of course, if these patterns are meaningful to this result.
Tags. I don't know what this argument/holivar has to do with the search/birth of truth, but it seems that I have managed to identify for myself the answer to the topic of the thread.
The boundary is between the left and right parts of the graph and is defined as follows:
If this particular TS, being optimized on the left part of the chart is statistically inclined to "profit-tail "** on the right part of the chart - then there is a pattern.
Otherwise it is a useless fitting.
// statistically inclined* - in this context, we mean multiple (1 ) shifts of optimization timeframes and (2) change of trading instruments
// "Profit-tail "** - "after-effects". Statistical plus-profitability in the adjoining future segment.
This definition takes into account the possibility of both "eternal" andtemporal patterns.
The marked line is accurate in a qualitative sense. Then there are only quantitative assessments (lifespan, degree of manifestation, etc.). Or the rubbish bin.
I agree 100%.
Gerasim puts a somewhat different notion of fitting as in searching for regularities, equating them, thus erasing the line. But they are just different in that when fitting on history - the system is losing forward, but when "tuning" to the real regularity during the optimization period - forward is profitable (at least up to 25% of the optimization period).
This is exactly the edge that you wrote about. Another question is how not to turn searching for ("tuning to") a real pattern into a fitting based on history.
here already depends and optimization time, step of input parameters change, preparation of input parameters, etc., everyone who is in the subject knows and is guided...
I repeat - on this subject ("how...") you can look in more detail here.
1. Spent almost 72 hours in training so far. Oh miracle. out of the top 200 results, more than 123 pass the forward test successfully. You can try to play the guessing game profitably.
Increase in training time led to gradual improvement of results on Sample period. Which is natural. The 72 hour result outperformed the 2 hour result by more than 4 times. The OOS result also improved in the long run, but there was no such thing as gradualism, it was an obvious failure.
__________
2. what does this tell you?
1. If repeated reproduction of the experiment shows the repeatability of this "miracle" for different times and peoples of instruments, we can talk about a regularity of the result obtained.
2. So far, almost nothing. See point 1.
Let me try to reason logically.
1) What is a pattern? The same price behaviour under certain conditions.
2) What do the conditions describe? Some selected characteristics of a price chart.
3) Are the characteristics of the price chart constant? Generally not constant.
4) How is a characteristic defined? By time and price characteristics.
5) Therefore, when is the same price behaviour possible? It is possible when the indicators are not constant (different).
6) What characterises a non-variable indicator? The characteristics of a time- and price-driven process that describes the change in indicators.
7) Hence, what is a pattern in the market? A regularity is the same behavior of the price when there are certain changes in the characteristics of the process considering time and price and describing the change of indicators on the price chart describing the regularity.
It turns out that regularity differs from the fitting in that conditions of regularity realization (the same behavior) change synchronously with the change of the price chart according to certain laws but the same things do not change in the fitting. It turns out that statically determined conditions can rarely describe the regularity as there are much less constant characteristics than variables in the market. Fringe thus singles out dynamic analysis systems as the type with the best ability to describe patterns.
Yeah, I'm aware of that. I'm not going to do it. But until the computer can't tell a cat from a cat by its appearance, it's useless. And when it does, the market will have a completely different look... :о)
Let me try to reason logically.
1) What is a pattern? The same price behaviour under certain conditions.
Price cannot be the same under certain conditions. Simply put, the probability of history repeating itself to within a pip is close to 0. The reason for this is that there is noise in quotations.
Since TA is based on searching for something in the past in order to exploit this something in the future, then:
1. noise - some patterns of the past without memory - random processes. Since there is dispersion, the patterns in the historical data are unevenly distributed, i.e. dense, then empty. Having encountered a significant accumulation of noise patterns preceding some price behavior with a high probability, during optimization (training) the TS can consider these very patterns as "trading signals". Naturally, such "patterns" are very unlikely to pass forward tests, since excessive accumulation in different parts of historical data is unlikely, while the absence of stable cause-and-effect relations, i.e. memory will give a loss.
2. Real trading signals - some past patterns preceding some future price behavior, i.e. non-random processes with memory. Since these patterns precede the trading signals, they accumulate uniformly, i.e. first the pattern, then the trading signal - a stable cause-and-effect relationship (If it is unstable - it is no longer a pattern). If the TS reveals these very patterns, at least in part, it can pass forward tests.
Theoretically, one could try to filter out the noise from the patterns. I.e. take all trading signals on forward tests and divide them into two categories:
1. The signal shows a loss - noise
2. The signal gave a profit - a pattern
Then we can, for example, teach the NS to distinguish noise from the patterns by additional attributes. As a result we obtain TS with noise suppressor. Some percentage of noise will anyway leak through but there are no 100% noise suppressors in nature.
In short, the bazaar should be filtered by results of forward tests - out of sample, i.e. OOS, but not on a representative sample - Sample. If you filter signals on Sample, which many are trying to do - you get a squared fit.
1. The signal gave a loss - noise
2. The signal gave a profit - a pattern
hahahahahahaha
It's like dividing animals into 'harmful' and 'useful'... So here too - there is a price movement... But if we've made a penny on it, then we condescend to call it "legitimate"... Otherwise it is meaningless "noise" - of course, it did not make us feel better, it must be an accident...
Shamans! Anthropocentrists! Don't make God angry!
:)
What do cats have to do with it?
I'll answer with a song ( Eduard the Cruel)... We can do without cats, but the point doesn't change... We make the machine measure the soul with a ruler... and we try to reveal regularities. They are undoubtedly there, but we can either look at them all at once (which will take up quite a lot of resources) or don't bother adding one or more instruments to the current story because it's a hundred percent fit, and since it's actually a Brownian motion, we can only get fragmentary results. As here for example (+/-/++/---/+/+/--/+/+/--/-/).Visually it seems that there are more pluses, because we want them, but in reality we don't...
Probably the next question is - Where's the song? :о))