Machine learning in trading: theory, models, practice and algo-trading - page 123
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
1. I still haven't got an answer, how is the pattern built/defined/detected? - I realize that the question is probably too intimate, so you don't have to answer.
The patterns are samples of a training sample. I.e. it's a line in the sample: several values of predictors and at least one value of the dependent variable.
Why does mytarmailS call patterns, i.e. what comes out of machine learning, not quite clear?
2. Turns - this is not even too "hard" an answer, but in general from the category of "I don't know where from and I don't know what". Here's a reversal on the next candle, no? - One more? - No, wrong! - Maybe on the fourth candle there will be a reversal? - Yes, a reversal, 150 points passed, and it turned back, but no, it wasn't a reversal but a correction, though it was a reversal nevertheless... There's no way to define "pivot"! - This means there is no possibility to teach how to detect it, not only in advance, but even in the current moment.
As for the potential reversals, there are indicators that can mark points on the chart for them (usually by arrows). The simplest example - it is B. Williams' fractals. Williams' fractals, which are known to redraw. In this case, we can try to predict, say, with the help of a classifier, whether a fractal is true or false, and whether it will re-draw (false) or stay on the chart (true). It is evident that if you correctly predict the veracity of a fractal, enter the market according to the signal, and exit at the following fractal, you can make profit. Or to lose money if the forecast proves to be false, but to exit the market without waiting for the next fractal, thereby minimizing the potential losses. I am working on this subject at the moment.
Of the inhabitants of this topic, Mihail Marchukajtes specializes in classifying of reversal points by the indicator tdsequenta
There is one more inhabitant of this topic, SanSanych Fomenko, who tries to forecast the reversal points by ZigZag.
The other inhabitants often try to predict the color of the candle following the classified pattern.
Another resident here, SanSanych Fomenko, is trying to predict ZigZag reversals.
In most cases the other residents try to predict the color of the candlestick following the classified pattern.
I have not tried to predict ZZ reversals. There are many reasons for that. I predict the ZZ shoulder. I don't like this target variable, but an error of less than 30% is fine with me, as this prediction is only part of the TS.
If I have time, I will definitely try to predict the ZZ reversal with some adjustments that have been made on this thread.
1. The patterns are examples of a training sample. That is, a line in the sample: several values of predictors and at least one value of the dependent variable.
2. Why mytarmailS calls patterns, i.e., what the results of machine learning produce, not quite clear?
As for potential reversals, there are indicators that can mark points on the chart for them (most often with arrows). The simplest example is B. Williams' fractals. Williams' fractals, which are known to redraw. In this case, we can try to predict, say, with the help of a classifier, whether a fractal is true or false, and whether it will re-draw (false) or stay on the chart (true). It is evident that if you correctly predict the veracity of a fractal, enter the market according to the signal, and exit at the following fractal, you can make profit. Or to lose money if the forecast proves to be false, but to exit the market without waiting for the next fractal, thereby minimizing the potential losses. I am working on it now.
4. Mihail Marchukajtes is one of the inhabitants of this topic on the classification of turning points by the indicator tdsequenta in practice.
5. One more inhabitant of this thread, SanSanych Fomenko, tries to predict the reversals by ZigZag.
6. 6. The other inhabitants often try to predict the color of the candle following the pattern.
1. What is a pattern - I understand, but I do not know what it is measured bymytarmailS(what kind of indicators or what else), so I asked (in order to try to help identify the causes of the problems with the model).
2. Yes, unclear.
3, 4, 5 All tools that allow us to determine the reversal either redraw or do it with a lag, and with a variable lag. I do not want to focus on reversals, but I can tell you with 100% certainty that it is not reasonable (to put it mildly) to detect reversals. Much more correct in terms of reliability and simplicity of detection, is the approach described below(you can try it out in the analyzed topic). The fractal is detected with a delay of 2-3 candles, it means uncertainty which should be avoided by all available means in trading.
6. 6. The color is 50/50. Regardless of the chosen horizon, it is absolutely futile.
ZS.
So, let's try to figure out what is undetermined in the market quotes and what can be represented as a clear and "soft" answer for a network, a tree, the classical TS on indicators or whatever we want to use in trading. The matter is that having answered these questions, it becomes possible to create profitable systems not only on grids and other methods of MO, but also on classic indicators. I understand that someone will say right now "give me the straights, monitoring! - I will not, try it yourself, think for yourself.
So, let's try to figure out what is undefined in the market quotes, and what can be represented as a clear and at the same time "soft" answer for the grid, tree, classical TS on indicators, or whatever we want to use for trading. The matter is that having answered these questions, it becomes possible to create profitable systems not only on grids and other methods of MO, but also on classic indicators. I understand that someone will say right now "give me the straights, monitoring! - I will not, try it yourself, think for yourself.
This thread is about machine learning and the problem with re-drawing described by you is absent in principle, because nothing is re-drawn on history.
The whole point of machine learning is to find such combinations of predictors on history that would determine one of the classes of the target variable. If we're talking about random forests, that's no more than 300 trees (usually 50 to 100), which can be obtained on a sample of no more than 5000 bars. Further increasing the sample does not increase the number of trees, i.e. tree types - "patterns" are over for a given set of predictors and target variable
The main problem is in overtraining, when in future samples the available trees do not correctly predict the class, or rather predict the class worse and worse. Exactly this question is considered in this branch, namely: we try to prove that the obtained trees will occur in the future and the classification error on future samples will be approximately the same as on the training sample.
This thread is about machine learning and there is no problem with redrawing as you described, because nothing is redrawn on history.
The whole point of machine learning is to find such combinations of predictors on history that would determine one of the classes of the target variable. If we talk about random forests, this is no more than 300 trees (usually 50 to 100), which can be obtained on a sample of no more than 5000 bars. Further increasing the sample does not increase the number of trees, i.e. tree types - "patterns" are over for a given set of predictors and target variable
The main problem is in overtraining, when in future samples the available trees do not correctly predict the class, or rather they predict the class worse and worse. This is the issue addressed in this thread, namely, trying to prove that the resulting trees will occur in the future and the classification error on future samples will be about the same as on the training sample.
Well, that's roughly what I thought. I expected a similar reaction.
Yury Reshetov, if there is interest, write to me in the personal, I will tell.
1. I still haven't got an answer, how is the pattern built/defined/detected? - I understand that the question is probably too intimate, you don't have to answer.
If i've got a good idea, i'll try lots of them, candlestick combinations + level combinations, price clustering, ssa + clustering, indicators with adaptive periods, Fourier, i have an idea how to describe Elliot waves in a formalized way and some ideas for crowd forecasting, but i don't know how to implement them, they're all too complicated.
The patterns are examples from a training sample. I.e. it's a line in the sample: several predictor values and at least one value of the dependent variable.
Why mytarmailS calls patterns, i.e. machine learning results, not quite clear?
I didn't know it was called a pattern, I'll keep that in mind.
Another inhabitant of this area, SanSanych Fomenko is trying to forecast reversals by ZigZag.
Actually, I deal with ZZ reversals.
I do not want to focus on the reversals, but I can say with 100% certainty that it is not reasonable (to put it mildly) to determine the reversals. Much more correct in terms of both reliability of detection, and simplicity of description, is the approach that I will describe below(just try it out in the work in progress)
I'm looking forward to it, and it would also be interesting to hear the rationale of why zigzag reversals are a bad idea.
Well, that's pretty much what I thought. I expected a similar reaction.
Yury Reshetov, if there is any interest, write me in the personal message, I will tell you.
Well, that's pretty much what I thought. I expected a similar reaction.
Yury Reshetov: Don't pay any attention. San Sanych can get a little cocky in his attempts to impose some personal and "final truth" rules of the game here. The point is that in machine learning there is no unambiguity, and there are a lot of not fully developed problems on the subject of application of "black boxes". That is why they are "black boxes", because they are obviously not obvious.
The essence of this thread is to discuss everything that has to do with machine learning, regardless of whether it corresponds to one's worldview or contradicts it. There are as many people as there are opinions, and there is no difference in taste or color.
If there is any confusion about ML, then discuss it here - no one will bite you for that.
Sailors who have no questions and those who have a clear idea where to dig and only wish to measure pips, may go to a branch: Machine learning: theory and practice (trade only; only those who have a state can enter).
Andrey Dik:
I am a beginner and I am a beginner with a lot of experience in this field.
The method is as follows (all the following applies equally to both ML and classical TS).
The point is to get rid of uncertainty, which means to get rid of fitting in training and optimization on the one hand, and on the other hand to unambiguously formulate "soft" requirements to the system. TP and SL on entry - this is also uncertainty, so we use it only for emergency exit (it depends on the instrument volatility and we select them empirically, so that 100% trades triggered stops as rarely as possible, ideally - never. We predict the whole trade, from entry to exit. We consider the result with a profit without taking into account the spread and commissions to be a successful trade (the system's performance is determined after the mo, whether it covers the spread and commissions). As soon as the signal to enter the market is received, the system enters and then waits for a certain number of bars (it is empirically determined and depends on predictors and a trade instrument), after that look, is equity in profit? - Close the position, if it is losing, we wait for one more bar. Sometimes I use two parameters: minimum and maximum (if a trade is not closed after the maximum number of bars, it is closed anyway), and sometimes I use only one - the minimum number of bars.
Many people will be surprised, but many, even seemingly hopeless systems start working, including TS on wands, not to mention all sorts of systems with ML. The trick is not to require ironclad rules of TC from yourself and the machine, not to try to fully describe market movements and give those very "soft" rules for ML. Besides, we get rid of the bad market legacy of heavy tails (or more precisely - tails stop mattering), the lack of stationarity in BP stops mattering - because we know that market shapes can be scaled vertically and horizontally without changing their internal properties (and this makes life extremely difficult for emleoners).
I wrote about it on forum 4, I think in a branch of the Swinosaurs, about two types of all the TS (with definite boundaries and indefinite), but there the idea was not completed. Now everything is more clearly visible, or something like that...
In general, I use my rule when developing trading systems: If changes in logics lead to increasing the share of successful variants of parameters out of all possible variants, then this is a good change (the probability of choosing a bad variant for trading is reduced, no matter how we change the parameters - we will be in the plus). This approach allowed a very large increase in this proportion in my TS.
The method is as follows (all the following applies equally to both ML and classical TS).
The point is to get rid of uncertainty, which means to get rid of fitting in training and optimization on the one hand, and on the other hand to unambiguously formulate "soft" requirements to the system. TP and SL on entry - this is also uncertainty, so we use it only for emergency exit (it depends on the instrument volatility and we select them empirically, so that 100% trades triggered stops as rarely as possible, ideally - never. We predict the whole trade, from entry to exit. We consider the result with a profit without taking into account the spread and commissions to be a successful trade (the system's performance is determined after the mo, whether it covers the spread and commissions). As soon as the signal to enter the market is received, the system enters and then waits for a certain number of bars (it is empirically determined and depends on predictors and a trade instrument), after that look, is equity in profit? - Close the position, if it is losing, we wait for one more bar. Sometimes I use two parameters: minimum and maximum (if a trade is not closed after the maximum number of bars, it is closed anyway), and sometimes I use only one - the minimum number of bars.
Many people will be surprised, but many, even seemingly hopeless systems start working, including TS on wands, not to mention all sorts of systems with ML. The trick is not to require ironclad rules of TC from yourself and the machine, not to try to fully describe market movements and give those very "soft" rules for ML. Besides, we get rid of the bad market legacy of heavy tails (or more exactly, tails stop mattering), the lack of stationarity in BP stops mattering - because we know that market shapes can be scaled vertically and horizontally without changing their internal properties (and this makes life extremely difficult for emleoners).
I wrote about it on forum 4, I think in a branch of the Swinosaurs, about two types of all the TS (with definite boundaries and indefinite), but there the idea was not completed. Now everything is more clearly visible, or something like that...
In general, I use my rule when developing trading systems: If changes in logics lead to increasing the share of successful variants of parameters out of all possible variants, then this is a good change (the probability of choosing a bad variant for trading is reduced, no matter how we change the parameters - we will be in the plus). This approach allowed a very large increase in this proportion in my TS.