Machine learning in trading: theory, models, practice and algo-trading - page 1288
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
i.e., nonstationarity is not killed by all this crap and the patterns are not better found
You don't have to kill nonstationarity.) It is impossible to kill it, because you, by definition, cannot absolutely precisely isolate anything from any BP or isolate anything that moves at all from BP, but only part of it, and only by your criteria, a significant part will always remain in BP, and will generate non-stationarity.
In general, stationarity-unsteadiness is not a very good criterion.
Don't kill non-stationarity.) It is impossible to kill it, because you, by definition, cannot absolutely precisely isolate anything from any BP or isolate from BP anything that moves at all, but only a part of it, and only by your criteria, a significant part will always remain in BP, and will generate nonstationarity.
In general, stationarity-non-stationarity is not much of a criterion.
In naked BP, regularity is only in cyclicality. This is an axiom. If cycles cannot be singled out, nothing works by definition.
The same attempt to lead to stationarity is an attempt to isolate a constant signal, which, as Alexander would say, does not existIn naked BP, the pattern is only cyclical. This is an axiom. If cycles cannot be singled out, then nothing works by definition.
I think the axiom is so-so). Take sound, music, say. There is no cycle there, only a short-term one, the rest is unpredictable. And even in a short-term cyclicity you don't understand very well - a whole orchestra plays, and everyone has his own part.)
I think the axiom is so-so). Take sound, music, say. There is no cyclicity there, only short-term cyclicity, the rest is unpredictable. And even the short-term cyclicity is not really understandable - a whole orchestra plays, and everyone has his own part.)
Well, it's not like anyone has tried to trade music or sound. There is a cyclic one, beats of some kind, and there is an acyclic unpredictable one.
There are predictable BPs, there are unpredictable onesI counted permutation and removal of predictors by 1 on the valid plot. Full randomness, as well as on the training plot.
, feature, absolute value, related value * 100
1) 23 0.05544194501249716 100
2) 53 0.04867290288234849 87
3) 32 0.03782135076252724 68
4) 37 0.03541102077687447 63
5) 26 0.03532324097876799 63
6) 33 0.03362697736099274 60
7) 40 0.03278533635676495 59
8) 13 0.03230890464933017 58
9) 60 0.03111487121639406 56
10) 24 0.03067918054294078 55
11) 8 0.02900490852298082 52
12) 10 0.02877257422711971 51
13) 49 0.02715383847459318 48
14) 64 0.02681691125087354 48
15) 38 0.02662037037037041 48
16) 35 0.02532532532532533 45
17) 1 0.02212475633528266 39
18) 57 0.02151192288178594 38
19) 28 0.02077687443541104 37
20) 12 0.01949317738791423 35
21) 11 0.01935357107770902 34
22) 56 0.01921172509407804 34
23) 19 0.01870370370370372 33
24) 27 0.01806684733514002 32
25) 46 0.01805450097021855 32
26) 3 0.0175925925925926 31
27) 42 0.01603966170895305 28
28) 44 0.01603966170895305 28
29) 4 0.01568141958114105 28
30) 54 0.01553166069295103 28
31) 36 0.01553166069295103 28
32) 25 0.01440866453921286 25
33) 63 0.01370370370370372 24
34) 41 0.01329274479959414 23
35) 55 0.01322751322751325 23
36) 15 0.01322751322751325 23
37) 17 0.01289590426080678 23
38) 39 0.01284348864994028 23
39) 7 0.01260422726391314 22
40) 9 0.012243648607285 22
41) 43 0.01221434200157606 22
42) 50 0.01074595722483046 19
43) 62 0.0106090745476935 19
44) 52 0.01058201058201058 19
45) 21 0.009986426216792743 18
46) 59 0.009936766034327027 17
47) 47 0.009652712202287306 17
48) 14 0.009616300104732023 17
49) 58 0.009333730513355176 16
50) 0 0.009109109109109115 16
51) 22 0.008516537928302648 15
52) 5 0.008285913946291301 14
53) 51 0.008285913946291301 14
54) 16 0.007571107018620848 13
55) 6 0.007467144563918782 13
56) 18 0.00722673893405601 13
57) 20 0.006734006734006759 12
58) 45 0.005037037037037062 9
59) 30 0.004840067340067367 8
60) 48 0.003703703703703709 6
61) 29 0.002872678772955772 5
62) 31 0.002849002849002857 5
63) 61 0.001154128632882168 2
64) 34 0.0003138731952291307 0
65) 2 -0.0009033423667569873 -1
, feature, absolute value, related value * 100
1) 14 0.04838455476753351 99
2) 28 0.04332634521313766 89
3) 40 0.03703703703703703 76
4) 48 0.0356709168184578 73
5) 37 0.03461279461279465 71
6) 26 0.03151827324012757 65
7) 3 0.02880658436213995 59
8) 39 0.02445842068483578 50
9) 34 0.02417848115177496 49
10) 51 0.0228526398739165 47
11) 6 0.02062678062678064 42
12) 52 0.01807496118873364 37
13) 19 0.01765719207579675 36
14) 17 0.01600654282042296 33
15) 50 0.01582491582491585 32
16) 25 0.01527640400043961 31
17) 36 0.01527640400043961 31
18) 44 0.01488195143784271 30
19) 1 0.01475021533161069 30
20) 47 0.01404853128991063 29
21) 33 0.01257220523275571 25
22) 22 0.01227513227513227 25
23) 41 0.01095008051529794 22
24) 7 0.0109137350516661 22
25) 16 0.01020525169131981 21
26) 43 0.009586056644880214 19
27) 4 0.009417989417989436 19
28) 49 0.008301404853129024 17
29) 35 0.007797270955165692 16
30) 27 0.007680976430976427 15
31) 29 0.00753851196329075 15
32) 23 0.00753851196329075 15
33) 59 0.006652765365902091 13
34) 24 0.006644880174291934 13
35) 15 0.006374326849104328 13
36) 13 0.006297363646066811 13
37) 38 0.006224712107065045 12
38) 55 0.005901505901505899 12
39) 10 0.005698005698005715 11
40) 61 0.005642761875448876 11
41) 9 0.005427841634738195 11
42) 42 0.005152979066022578 10
43) 0 0.00490852298081218 10
44) 2 0.003703703703703709 7
45) 30 0.003406967798659233 7
46) 62 0.003122308354866488 6
47) 31 0.003122308354866488 6
48) 64 0.002295252999478359 4
49) 21 0.0008465608465608732 1
50) 11 0.0006224712107065211 1
51) 53 0.0005336748852599049 1
52) 12 0.0005336748852599049 1
53) 58 0.0002916302128900816 0
54) 5 0.0002153316106804914 0
55) 8 -0.0001086130118387874 0
56) 18 -0.0007739082365947891 -1
57) 20 -0.0008417508417508102 -1
58) 54 -0.0009746588693956837 -2 (30)
59) 46 -0.002010582010582018 -4 (25)
60) 32 -0.002348169495143548 -4 (3)
61) 57 -0.003145611364789413 -6 (18)
62) 56 -0.004743162781309929 -9 (22)
63) 45 -0.00597371565113497 -12 (58)
64) 60 -0.007107107107107102 -14 (9)
65) 63 -0.008547008547008517 -17 (33)
in () for the bottom 7 - the position of this predictor when removed by 1 - you can see that the position is random
And in the article about this method, everything is very nice.
Why can it be so?
In the article, all the predictors (6 in total) are important and this method does a good job of eliminating one noisy predictor. Of my 65 predictors, half or most may be noisy.
Plus the forest is still random and when calculating the error by removing 1, random deviations could also be introduced, which can shift the importance of the predictor on the scale of importance.
Plus what makes the permutation method unstable is that in practice the tree will always find another predictor with almost the same good separation, and permutation kind of removes this node (making the result of its work random).
In general, on my data permutation (in my version, i.e. by rearranging the lines of the predictor being checked) does not work.
Maxim, you implemented it differently (something with a normal distribution). You didn't do a comparison with removal by 1? Or did you take for granted the results from the article?
Well, it's not like anyone tried to trade music or sound. There is cyclic exactly the same way, beats of some kind, there is acyclic unpredictable.
There is a lot more cyclicity in sound, music, than in the market. Let's trek on music! I'm sure, that the results will be not better than on the market).
I've counted permutation and deletion of predictors by 1 on a valid plot. Total randomness, just like on the training plot.
And in the article about this method, everything is very nice.
Why may it be so?
In the article all predictors (6 in total) are important and this method eliminates one noisy predictor very well. Of my 65 predictors, half or most may be noisy.
Plus, the forest is still random, and when calculating the error by deleting 1, random deviations could also be introduced, which can shift the importance of the predictor on the scale of importance.
Plus what makes the permutation method unstable is that in practice the tree will always find another predictor with almost the same good separation, and permutation kind of removes this node (making the result of its work random).
In general, on my data permutation (in my version, i.e. by rearranging the lines of checked predictor) does not work.
Maxim, you implemented it differently (something with a normal distribution). You didn't do a comparison with removal by 1? Or did you take the results from the article on faith?
First of all, you need to decorrelate (if you haven't already done so), i.e. remove all correlates, say, above 0.9 at least. Otherwise the permutation doesn't work.
I didn't really go into comparisons, I just saw that it reduces error this time, throws out unnecessary things and makes the model easier (almost without losses) that's two
I may find some examples in python for other models and compare with what I got through the alglib, but I'm bored.
to begin with, you need to decorrelate (if you haven't already done so), i.e. remove all correlates, say, above 0.9 at least. Otherwise the permutation doesn't work
I didn't go deep into comparisons, I just saw that it reduces error this time, throws out unnecessary things and simplifies model (almost without losses) that's twoI tried it with Spearman 0.9 removal, no improvement.
Well, what are the final errors? for both models and how it works on the new data
Man, it's a lot of deep analysis to be done here
There's a lot more in sound, music, cyclicality than in the marketplace. Let's try it on music! I'm sure that the results will be no better than on the market).
You can simply run around in a circle, waving your hands, the result will be about the same).