Machine learning in trading: theory, models, practice and algo-trading - page 57
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Well, stability is achieved by preventing potential overlearning. And unbalanced training sample is a potential cause of overtraining for low-representative classes. After all, the learning algorithm tries to act as it sees fit rather than as it needs to in order to increase generalizability. If the sample is unbalanced, it will minimize learning errors on the least representative classes, since there are few examples for such classes and it is easiest to rote them by heart instead of doing generalization. After such rote learning, there is nothing to be surprised that outside the training sample, algorithm errors are most likely to occur in classes that are less representative.
The point is simple. In the real world, no one will allow you to take a mixed sample containing observations from the future to assess the quality of real trading. All observations will go after day X.
Hence, by taking a mixed sample in validation (without date separation), you are overestimating the quality metric on validation. That's it. Then there will be unpleasant surprises.
What does thought have to do with it? Thought is subjective. Thinking can be right and knowingly wrong. Because imagination is problematically limited. The criterion of truth is always experience.
Take two training samples, one pre-balanced and the other highly unbalanced. Train the algorithm on both samples and measure the generalization ability on the test parts. After that, compare the generalization ability. The one that gives the best generalizing ability will be the criterion of truth.
Otherwise we will think and speculate to the point of losing our pulse. After all, disagreement is born in argument, but truth is born in experience.
And so I'll stop with the further discussion of the topic of balancing the training sample. Otherwise this hullabaloo can continue indefinitely, as there are two different opinions and continue to measure who of us thinks more correctly - a waste of time.
Well, stability is achieved by preventing overtraining. Unbalanced training sample is a potential cause of overtraining for low-representative classes. After all, the learning algorithm tries to act as it sees fit, not as it needs to in order to increase generalizability. If the sample is unbalanced, it will minimize learning errors on the least representative classes, since there are few examples for such classes and it is easiest to rote them by heart instead of doing generalization. After such rote learning, it is not surprising that, outside the training sample, the algorithm's errors on the less representative classes are most likely.
1. On unbalanced classes, you get a hell of a lot: the error between classes can vary by several times. Which is the right one?
2. It is far from always possible to balance classes.
Your example with BUY|SELL. When the number of observations (bars) is over 3000, the difference in unbalance will be 10% at most 20%. It is quite possible to balance.
And hereDr. Trader has suggested the target variable "Pivot/Not Pivot". I think he took it from ZZ. So in such a target variable, the unbalanced classes will be different by orders of magnitude. If we augment to the maximum class, can we teach on such a balanced sample? It seems to me that no.
So it's not that simple with balancing.
From my own experience:
I could not find other solutions.
I'm going to drop everything and become an adept of R in order to play numbers with a serious face on my face.
So far I haven't noticed that you're playing numbers, but I don't know about your face - I can't see it.
What does thought have to do with it? Thought is subjective. Thinking can be right or wrong. Because imagination is problematically limited. The criterion of truth is always experience.
Take two training samples, one pre-balanced and the other highly unbalanced. Train the algorithm on both samples and measure the generalization ability on the test parts. After that, compare the generalization ability. The one that gives the best generalizing ability will be the criterion of truth.
Otherwise we will think and speculate to the point of losing our pulse. After all, disagreement is born in argument, but truth is born in experience.
And so I am done with further discussion of the topic of the balance of the training sample. Otherwise this hullabaloo can continue indefinitely, as there are two different opinions and continue to measure who among us thinks right - a waste of time.
I mean one thing, you mean another. I say we should divide the set strictly by dates. And you speak about balance.
I'm sorry, but I've already said that I don't see any point in continuing this chorus. I already tried to explain the shortcomings of balance with examples. I guess I wasn't very convincing, was I? Well, I'm not good at black rhetoric, so I'm not good at giving black for white with a serious look on my face. So don't be too hard on me.
Most likely the reason is that you're trying to convince me that you're supposedly trying to "force" you to balance reality? But I have no such intention. I know that reality, and unfortunately for me, is often unbalanced and the possibilities for balancing it are not always available. So in my posts I tried to explain to you that it is not necessary to try to balance the real reality outside of the training sample, but it is necessary and enough just to balance the training sample, so that the model obtained from it would not be skewed in the direction of strongly representative classes. When dividing the general sample into parts by dates, it is also often impossible to achieve balance. That's why I balance training sample not by dates, but by equal representativity of classes in it.
I won't answer any more questions about balancing of the training set. So this hooliganism has already dragged on.
I'm sorry, but I've already said that I don't see any point in continuing this chorus. I already tried to explain the shortcomings of balance with examples. I guess I wasn't very convincing, was I? Well, I'm not good at black rhetoric, so I'm not good at giving black for white with a serious look on my face. So don't be too hard on me.
Most likely the reason is that you're trying to convince me that you're supposedly trying to "force" you to balance reality? But I have no such intention. I know that reality, and unfortunately for me, is often unbalanced and the possibilities for balancing it are not always available. So in my posts I tried to explain to you that it is not necessary to try to balance the real reality outside of the training sample, but it is necessary and enough just to balance the training sample, so that the model obtained from it would not be skewed in the direction of strongly representative classes. When dividing the general sample into parts by dates, it is also often impossible to achieve balance. So I balance the training sample not by dates, but by equal representativeness of classes in it.
I will not answer any more questions about balancing the training sample. So, this hullabaloo has gone on long enough.
I want to interject to complete the picture and repeat my opinion, expressed above the branch.
1. It is necessary to have two sets of data: the second is an extension of the first in time.
2. Balance the first data set. We definitely balance it.
3. We randomly divide the first data set into three parts: training, testing and validation.
4. We obtain an error on the second set, which is a continuation of the first set in time.
If the error on all FOUR sets is about the same, then the model is not retrained. If the error has a decent value, then we can safely go on, i.e. run it through the tester.
If there is a significant difference (more than 30%), then the initial set of predictors leads to retraining of the model and from personal experience replacing the model type cannot fix anything, in terms of retraining. We need to get rid of the noise predictors. It can easily turn out that there are NO noise predictors among the predictors at all.