Machine learning in trading: theory, models, practice and algo-trading - page 161
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
What if I told you that there is no such thing as "Newton's Law" in nature? And it's just a formula derived to simplify calculations. And the phrase "Newton's Law works or it doesn't work" implies that this formula can be used to calculate some process, or vice versa it cannot be applied due to the complexity of the problem and the chaotic nature of the world.
Suppose there is a steel ball. Knowing its mass, you can determine how fast it will fall, how fast it will reach the ground, etc., all quite accurately. In the case of a fluff, however, there are so many influences on it that applying Newton's laws will not help you calculate where and when the fluff will fall. Even if you shut yourself up in a windless room at the bottom of the ocean, even any seismic activity will change and the fluff will not fall where you calculated. Even such a complex experiment is already beyond the bounds of ordinariness, but it is still not accurate enough.
The fluff behavior is an analogy to the trading symbol behavior in Forex. You can make an Expert Advisor with thousands of formulas, but all they will describe only the phenomena you observe in this process. You will never fully understand the underlying processes, and that's why no matter how accurate the formulas are, they will work only under ideal conditions, describing only the previously observed phenomena. And in reality something unexpected will happen and the market will go against all of your formulas and take out all of your stops.
I was engaged in calculation of dynamics of flow of liquids and gases. With full responsibility I can declare to you that if the object "fluff" behaves as "unusually" at any observer, it does not mean that laws of Newton do not work, it means that the observer has not considered any forces influencing on object, in this case movements of air masses which, taking into account a ratio of object mass to the area of its surface, have much more influence, than force of gravitation. But these are all the same Newtonian forces, which were not taken into account by the observer.
Do you see what I mean? It's elementary stuff, man. If something doesn't work, then something is left unaccounted for. It's hardly anybody's fault, except for the observer.
Now draw an analogy, if you will, mentally, between the above and the market.The law works, including for fluff. But when one looks at things "on a domestic level", that's what you get....
Fa, the known variables are discarded at once, offering to pray)))
Thanks, I read it.
I think the author is too optimistic.
When using standard preprocessed data (fx), this approach is thought to work
This approach will not work better than notorious committees, and if you make it more complicated (the number of bricks), you'll get the cruelest fit.
All imho of course...
movements of air masses, which, taking into account the ratio of the object's mass to its surface area, have a much greater impact than the gravitational force. But these are all the same Newtonian forces that have not been accounted for by the observer.
We're talking about the difficulty of applying Newton's law to determine the motion of a lump at the domestic level. At first you sneered that it was a piece of cake. And now suddenly it turns out that you have to take into account the movement of air masses, and all Newtonian forces in general. What kind of everyday life is it with you that using a data center to calculate air masses is commonplace? You must be building rockets, or some other spaceships.
And what kind of Newtonian forces are these? The force used by Isaac Newton to revolve once every 1 second? (Just kidding.)
The same! Both on the fluff and on the ball)))
I'm so stupid, yeah ) It's pretty much the same at one point in time. But the balloon will just fall and that's it. And the trajectory of the falling bullet is more twisted, so it has to fly a longer distance, and the sum of the effects for the entire time of falling will be different. Well, unless the situation in a vacuum with zero external influences.
1. We are talking about the difficulty of applying Newton's law to determine the motion of a lump at the domestic level.
2. 2. At first you sneered that it was a piece of cake.
And now suddenly it turns out that you have to take into account the movement of air masses and all Newtonian forces in general. What kind of life is it with you that the use of a data center to calculate air masses is commonplace? You must be building rockets, or some other spacecraft.
4 And what kind of Newtonian forces are these? The force used by Isaac Newton to revolve once every 1 second? (Just kidding)
1. You're the one talking about the unknown, and SanSanych, for whom you're interjecting, was saying that "on a domestic level" laws don't apply to fluff. But I assure you, down is just as subject to the laws of physics as any other body.
2. Easier than a dime a dozen? - This is your fantasy, I did not say that it is easier or simple.
3) If you do not take into account all the forces acting on the body, it becomes impossible to determine the further trajectory of motion. And some may even think that the body is beyond the laws of physics. Let me repeat once again, in simple terms - if "something" does not work for someone, then this person does not take into account all the factors acting on "something".
Familiarize yourself first with Newton's three laws and the consequences of these laws. Then you can proceed to an in-depth study of kinematics of a solid body, and then, if you wish, to the mechanics of a solid deformable body.
To reinforce what you have read, try drawing on paper a metal ball with a specific density of 7.8 g/cm3 and a similarly sized fluffball with a density of 0.00001 g/cm3. Schematically apply to these bodies and mark the force vectors. It will become clear the reasons for the different behavior of the ball and the fluff if they are sent in free fall. At once the desire to approach the study of processes in the market "at the household level" will disappear.
"Don't talk nonsense" (c) Reshetov, I don't remember the year.
Systems analysis declares an error of the first kind:
"Applying the right methods to the wrong problems."
My example.
Applying Newton's law to the fall of a steel ball is an example of applying the correct method to a correctly posed problem, because one can be guided only by Newton's law andcan neglect all other factors affecting the fall of the ball.
Applying Newton's law to a lump is an example of applying the correct method to a NOT correct problem, because in the case of a lumpyou can neglect Newton's law and need to consider other forces that determine the motion of the lump.
In Forex the problem of selection of factors (predictors), relevant to the target variable, from noise is extremely acute, because the presence of noise predictors in the model leads to overtraining of this model. It is the retraining of the model (trading system) that is the main issue.
To me the picture is as follows.
On sufficiently large amount of observations (5000 - 10 000 bars) we select among several hundreds of predictors 20-30 predictors that are relevant to the target variable. For a particular target variable the selection will be individual.
Then, in a smaller window of 100-300 bars, we re-select predictors from this set, e.g. by rfe. I was surprised to see that as the window moves, the list of predictors and their number change. In my case: initial list = 170 predictors. I selected 27 predictors, and by rfe I get from 5 to 15 predictors as the window moves.
With this approach, the rf prediction error is close to 20%. Ada looks better, and gbm even better. I will especially note that the error on the training sample is about equal to the error outside that sample. The model is NOT retrained.
This is what "applying the right methods to the right problem" looks like in forex.
And what does "applying the correct rf method to the wrong set of predictors look like?
We take a set of predictors out of the blue. At the same time it is very important to consider the experience of TA - they are very fond of trend indicators - various scales.
With such a list of predictors on the training set rf can easily be trained with an error within 5%.
And if we take the set outside of this training set, it will immediately become clear that the trees found by the rf algorithm have nothing to do with the new kotir segment - the model is retrained, i.e. it has memorized the specifics of the training set and is completely unusable outside of this set. A familiar picture, isn't it?
At the link the guy talks about interesting ideas on how to do it based on Bayesian probabilities.
Respectively, we have two opposing tendencies: on the one hand, we have a training sample that we would like to predict as accurately as possible, all other things being equal, while on the other hand, we have the complexity of found regularities, the complexity of forecasting algorithm, and we would like this complexity to be lower. These requirements contradict each other, so we need to find a compromise somehow, but in order to find it, we need to express both complexity and accuracy in some unified terms.
https://postnauka.ru/video/55303
Unfortunately Dmitry Vetrov can't tell the difference between the learning capability of the models, and the generalizing capability. Therefore he has no compromise. However, if we look at the graph (taken from HERE), we see that the compromise can be easily found:
That is, if we follow D. Vetorov's reasoning, there is no learning capability for the trade-off (the blue line is the learning capability. However, if we look at the dependence of the model's generalizability error on complexity, we see that the trade-off is reached at the M value of model complexity (the extremum of generalizability).
jPrediction, starting with version 9 and finds this compromise, namely it complicates the models to a value of M and outputs the model found at M as the result.
By model complexity in jPrediction we mean a gradual increase in the number of predictors. Because in jPrediction the number of neurons in the hidden layer equals 2^(2*n+1), where n is the number of predictors. Accordingly, as the number of predictors increases, the complexity of the model (the number of neurons in the hidden layer) increases. Thus, by gradually increasing the complexity of the models, jPrediction will sooner or later reach the value M, after which further complication of the models will lead to a further decrease in the generalizability (increase in errors in the generalizability).
Thus, jPrediction kills two birds with one stone:
And there are no problems, as voiced by D. Vetrov. Newton has nothing to do with it either.
1. you are the one talking about something unknown, and SanSanych, for whom you are interjecting, was saying that "on a domestic level" laws do not apply to down. But I assure you, down is just as subject to the laws of physics as any other body.
I didn't like the trollish, sarcastic comments that came down on CC either. Don't do your favorite thing in this thread. It seems that the "underprivileged engineers of machine learning in the MQL language" here are having a good time with their fellows who do research using other tools.
Let's put it this way: CC said that if you take just poplitee fluff, their extremely noisy landing trajectories would be much harder to derive Newton's laws. This - I agree with him - echoes the problems we have. The extremely noisy signal from which we are trying to pick up the true laws.
And all other considerations sound like trolling. CC has given us a metaphor to comprehend. That's it!
I didn't like the trollish, sarcastic comments that came down on CC either. Don't do your favorite thing in this thread. It seems that the "underprivileged engineers of machine learning in the MQL language" here are having a lot of fun with their fellows who do research using other tools.
Let's put it this way: CC said that if you take just poplitee fluff, their extremely noisy landing trajectories would be much harder to derive Newton's laws. This - I agree with him - echoes the problems we have. An extremely noisy signal from which we are trying to fumble for true laws.
And all other considerations sound like trolling. CC has given us a metaphor to comprehend. That's right!
SS just gave a very unfortunate example, while continuing to persist in his ignorance. It would be ok to say like "well, yes, overdid with the example", but no, persists, look what he writes even after my recommendations to try to draw the scheme of the acting forces on the ball and the fluff:
Applying Newton's law to the fluff is an example of applying the correct method to a NOT correct problem, because in the case of the fluffyou can neglect Newton's law and need toconsider other forces that determine the movement of the fluff.
What do you mean by "other forces"? The same forces act on the ball and the fluff - the force of gravity (weight) and the force of the wind flow distributed over half of the body area. So there are only 2 forces in both cases. The wind flow is the same, but the weight is thousands of times different. Construct the sum of the vectors of the applied forces and see what happens.
So the metaphor doesn't work for SS.
Otherwise he is right about "applying the right methods to the wrong predictors", but there is nothing very wise here, yes - the sky is blue, the sun is shining, the bus #16 passes by the house at exactly 1:07 pm... So what's next, what's the wisdom or at least practical value of this saying?
SZY. I probably have less experience in the MOE than you do, and if I say something heresy about the MOE, you are unlikely to pass by and try to help me eliminate my prejudices (I believe that you are a sympathetic person not indifferent to flagrant cases of excessive enthusiasm for stepping on the same rake). I, too, can't get past the blatant "Nyuto's powers don't work on a fluffer" bloopers.