Bayesian regression - Has anyone made an EA using this algorithm? - page 51
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
And how do you determine the "best" combination?
The basic problem is that of the list of predictors. Having justified the list of predictors we can move on to the rest.
Thank you. I'm leaning towards adding more predictors too.
Do you think the number of predictors you use is insufficient?
I don't know for sure.
I do not know for sure.
I've already written, I'll say it again.
I performed the work on selection of predictors several times, including on demand. The results are given below
So.
Let's take some set of predictors, not less than 50, and better than a hundred.
All sets of predictors I dealt with (i.e. I do not claim to generalize) can be divided into two parts:
I write "relation" very carefully and quite deliberately do not use any terms.
Example of predictors:
Please note that I am specifying the target variable. For the other target variable, it may be the other way around
The problem with having these two groups of predictors in the original set of predictors is that the standard tools for determining IMPORTANCE do not work. Therefore, some tools are needed, and I have developed and use them, which allow for the coarse sifting of noise predictors. It should be noted that there is no unambiguity here. The algorithm quantifies separately for valid and nominal predictors. Less than 2 (some relative value) is noise for sure. Between 2 and 3: can be used, but better not....
The problem with noise is that predictors relevant to noise clobber predictors not relevant to them. For example, randomforest, ada, and svm algorithms for some reason build the model more on these noise predictors.
Having screened out the noisy predictors, and in my sets there were about 80%(!), we take the rest of the list of predictors and to it start applying the tools from R to determine the importance of the variables. The actual number of predictors used to train the model is about half of the NOT noise predictors, i.e. about 10% of the original set.
I determine the importance of predictors in the window. As the window moves, the list of predictors from the basic 20% changes all the time. I.e. 12-15 predictors are used to build the model, but they are different as the window moves following the quote.
What's the reason?
Well, the point is that clearing the set of predictors from noise ones leads to creation of models which are NOT retrained.
In numbers.
On full set of predictors it is possible to build models with 3%-5% prediction error! And any algorithms that divide the sample into parts, the so-called "out-of-sample" - OOV, confirm this result. This is very well seen in raatle, which always divides the original sample into parts and is very happy with the results.
But.
If the initial sample contains noise predictors, then if we take a real "out-of-sample", i.e. for example the sample for training from 01.06.2015 to 01.01.2016, and then calculate on the sample after January 1, we can easily obtain an error of 50% and 70% instead of 3%-5%! Moreover, the further away from 1 January, the worse the result.
MODEL IS RETRAINED
If I cleanse the original set of noise predictors, the results are as follows and the same for randomforest, ada SVM as well as several other models - i.e. the model solved nothing in my cases, the results are: prediction error is about 30% on any set. By applying R's predictor importance tools we can further reduce the error to around 25%. It was not possible to improve this result for the target variable ZZ.
I've already written, I'll say it again.
I have done the work of selecting predictors several times, including on commission. The results are given below
So.
Let's take some set of predictors, not less than 50, and preferably more than a hundred.
All sets of predictors I dealt with (i.e. I do not claim to generalize) can be divided into two parts:
I write "relation" very carefully and quite deliberately do not use any terms.
Example of predictors:
Please note that I am specifying the target variable. For the other target variable, it may be the other way around
The problem with having these two groups of predictors in the original set of predictors is that the standard tools for determining IMPORTANCE do not work. Therefore, some tools are needed, and I have developed and use them, which allow for the coarse sifting of noise predictors. It should be noted that there is no unambiguity here. The algorithm quantifies separately for valid and nominal predictors. Less than 2 (some relative value) is noise for sure. Between 2 and 3: can be used, but better not....
The problem with noise is that predictors relevant to noise clobber predictors not relevant to them. For example, randomforest, ada, and svm algorithms for some reason build the model more on these noise predictors.
Having screened out the noisy predictors, and in my sets there were about 80%(!), we take the rest of the list of predictors and to it start applying the tools from R to determine the importance of the variables. The actual number of predictors used to train the model is about half of the NOT noise predictors, i.e. about 10% of the original set.
I determine the importance of predictors in the window. As the window moves, the list of predictors from the basic 20% changes all the time. I.e. 12-15 predictors are used to build the model, but they are different as the window moves following the quote.
What's the reason?
Well, the point is that clearing the set of predictors from noise ones leads to creation of models which are NOT retrained.
In numbers.
On full set of predictors it is possible to build models with 3%-5% prediction error! And any algorithms that divide the sample into parts, the so-called "out-of-sample" - OOV, confirm this result. This is very well seen in raatle, which always divides the original sample into parts and is very happy with the results.
But.
If the initial sample contains noise predictors, then if we take a real "out-of-sample", i.e. for example the sample for training from 01.06.2015 to 01.01.2016, and then calculate on the sample after January 1, we can easily obtain an error of 50% and 70% instead of 3%-5%! Moreover, the further away from 1 January, the worse the result.
MODEL IS RETRAINED
If I cleanse the original set of noise predictors, the results are as follows and the same for randomforest, ada SVM as well as several other models - i.e. the model solved nothing in my cases, the results are as follows: the prediction error is about 30% on any set. By applying R's predictor importance tools we can further reduce the error to around 25%. It was not possible to improve this result for the target variable ZZ.
Thank you.
I see what you are thinking. From all of the above I saw a possibility to calculate the importance of predictors on several parts of the training sample, then compare the lists and select duplicates.
I can't say anything about manual selection, I prefer to use the machine right away.
SZZ: I'll try to apply my homebrew method, based on the mutual information function, in addition to the importance of the variables from the decision forest. I'll show you the results later.
The problem with noise is that predictors relating to noise clog up predictors not relating to it. For example, randomforest, ada, and svm algorithms for some reason build the model more on these noise predictors.