Machine learning in trading: theory, models, practice and algo-trading - page 3356

New comment
 
Aleksey Vyazmikin #:

Is that exactly what you want to calibrate - shifting the point of class division to the point of income generation?

No.
 
La. You can ask 30 times, but you can't google it.
 
Maxim Dmitrievsky #:
No.

Then what's the purpose?

 

I think everyone has heard about calibration, but it is of no practical use, precisely because the sample is not representative.

Probabilistic estimation of individual leaves, in my opinion, gives a more reasonable result than reweighting the sum of leaves of the model.

 
Aleksey Vyazmikin #:

I think everyone has heard about calibration, but there is no practical use in it, just because the sample is not representative.

Probabilistic estimation of individual leaves, in my opinion, gives a more reasonable result than reweighting the sum of leaves of the model.

Everyone has heard everything, but no one has responded to anything. Not to mention other nuances that are not disclosed, but only guessed that it turns out to be the case.

And if you have a weak (with low expectation), but stable on OOS model, there is no sense to calibrate too? And if you think about it.
 
Maxim Dmitrievsky #:
Everybody heard everything, but nobody answered anything. Not to mention other nuances that are not revealed, but only guessed that it turns out to be this.

And if you have a weak (with low expectation) but stable on OOS model, there is no sense to calibrate it too? And if you think about it.

Now I came up with the idea of constant calibration, with some weight - something like EMA for each interval. Then at least there will be an effect of adaptation to market volatility and model obsolescence.

I don't see any sense in static calibration on some separate data. On my predictors I investigated the issue of stability of statistical indicators, and there are few such indicators, and the model is full of such erratic predictors. That's why I'm looking for stability to which something like this can be applied.....

In the screenshot above I showed the model in section - you can see how low Recall at the edges is usually, which already speaks about not equal statistical measures for the same weighting, and often they will not be enough to talk, even in theory, about stability in this range of "probability". So from this point of view, too, calibrating the total looks like a dubious idea.

I am more interested in the idea of reweighting values in leaves, however, I have written about it earlier, but I did not get any feedback here - so it's all on my own....

 
Some new definitions again.
For the last time: the classifier is calibrated because it outputs incorrect probabilities. They are meaningless in their original form. Get over it.
 
Catbusta has an open code - you can look at it to know exactly what is being given away.
 
To explain the picture in simple language: for the classifier, the first and the second case on the histogram are identical, because class labels are used. Both there and there is a unit for the most probable class. After training, it will give not the class probability, but a one minus the prediction error passed through sigmoid or softmax.

This is completely inconsistent with what you would expect when setting a prediction threshold.
1...334933503351335233533354335533563357335833593360336133623363...3629
New comment