Machine learning in trading: theory, models, practice and algo-trading - page 2566

 
Renat Akhtyamov #:

https://www.mql5.com/ru/forum/375928/page2

if 0 ≤ H < 0,5 - prices are fractals, the validity of FMH is confirmed, there are "heavy tails" in distribution of variables, antipersistent series, i.e. negative correlation in price changes, pink noise with frequent changes of price direction;
👍
 
If you take SB realizations and calculate Hearst for them, it will always be different and will differ from 0.5 and sometimes greatly. In order to make any sense in such calculations, it is always necessary to calculate the p-value - the probability that the calculations were carried out on SB.
 
Aleksey Nikolayev #:

Vorontsov is probably the best expert on MO in Russia. The course is therefore bound to be good, but since it is for IT people, it omits basic and important mathematics for us. I`ve noticed more than once, that for the application of mathematical methods in trading their basic, simplified form is not very suitable.

MO is based (see for example Tibshirani) on the assumption that there is a constant joint distribution of predictors and responses P(X,Y). From it, the conditional probability Py(Y|X) can be calculated, from which the regression Y=f(X) can be calculated. Eventually, this regression is approximated by some MO models. In the physical world, this theory more or less works. But not in trading. It turns out that P(X,Y) changes unpredictably with time (non-stationarity) and the whole theory collapses a bit.

The most common approach is just to ignore non-stationarity and then be surprised with the results and complain about the MO).

It is important to understand exactly: "non-stationarity" of what? and not to descend to non-stationarity of the time series itself. You can probably ignore the non-stationarity of the quotient itself.

The cornerstone is the conditional probability Py(Y|X).

Instead of conditional probability, it is more convenient to use the "predictive power" of the predictor with respect to a particular teacher.

I introduced a measure of such predictive ability and ran a window on BP, typing in a statistic of 2000 examples. I will especially note that there is no mention of models at all. Looking for a predictor-teacher pair.


Here are some of the results: the column is an individual predictor, and the summary lines are given: the mean predictive ability, standard deviation and % for convenience.


We see that among the predictors there is a predictor with an sd/mean ratio of about 10%. But remarkably, I have NOT encountered any predictors that have this percentage greater than 100%.

So the design challenge is to find a set of predictors for a particular teacher that would be limited to an sd/mean ratio of 10%, or preferably 5%, which can be neglected. Stability of predictive ability is the cornerstone of a trading system.

 

Google the sentence verbatim

"

It turns out that the optimal prediction of the future value of the level of the series is its current value

"

 
SanSanych Fomenko #:

It is important to understand exactly: "non-stationarity" of what? and not to jump to non-stationarity of the time series itself. Most likely, we can ignore the non-stationarity of the quotient itself.

The cornerstone is the conditional probability Py(Y|X).

Instead of conditional probability, it is more convenient to use the "predictive power" of the predictor with respect to a particular teacher.

I introduced a measure of such predictive ability and ran a window on BP, typing a statistic of 2000 examples. I will especially note that there is no mention of models at all. Looking for a predictor-teacher pair.


Here are some of the results: the column is an individual predictor, and the summary lines are given: the mean predictive ability, standard deviation and % for convenience.


We see that among the predictors there is a predictor with an sd/mean ratio of about 10%. But remarkably, I have NOT encountered any predictors that have this percentage greater than 100%.

So the design challenge is to find a set of predictors for a particular teacher that would be limited to an sd/mean ratio of 10%, or preferably 5%, which can be neglected. Stability of predictive ability is the cornerstone of a trading system.

It is just an element of the art of the master in the MO, the search for such connections. I do something similar on the basis of my own work. Often, we look for dependencies through cross-entropy, which is costly in terms of resources. Is it faster for you?
 
SanSanych Fomenko #:

It is important to understand exactly: "non-stationarity" of what? and not to jump to non-stationarity of the time series itself. Most likely, we can ignore the non-stationarity of the quotient itself.

The cornerstone is the conditional probability Py(Y|X).

Instead of conditional probability, it is more convenient to use the "predictive power" of the predictor with respect to a particular teacher.

I introduced a measure of such predictive ability and ran a window on BP, typing in a statistic of 2000 examples. I will especially note that there is no mention of models at all. Looking for a predictor-teacher pair.


Here are some of the results: the column is a separate predictor, the summary lines are given: the average predictive ability, standard deviation and % for convenience.


We see that among the predictors there is a predictor with an sd/mean ratio of about 10%. But remarkably, I have NOT encountered any predictors that have this percentage greater than 100%.

So the design challenge is to find a set of predictors for a particular teacher that would be limited to an sd/mean ratio of 10%, or preferably 5%, which can be neglected. The stability of predictive ability is the cornerstone of a trading system.

Without series stationarity, statistical calculations such as yours may be meaningless - there may be no convergence of sample values to true values. For example, it's not hard to think of an example where the sample correlation of neighboring increments would be nonzero, but the true correlation would be zero.

PS.

Stationarity is understood "in the narrow sense" - the time independence of joint distributions.

Stationarity can be incomplete - for example, it can refer only to joint distributions of increments (processes with stationary increments).

It is of course correct to talk about the stationarity of a slope rather than a series which is only one of the realizations of a given process. But we are not at the exam, so it does not matter.)

Often by stationarity they mean its variant "in the broad sense". They remember only the constancy of the mean and variance, forgetting about the condition on the ACF. In any case, such stationarity is not sufficient in MO (it will be sufficient for linear models).

 
There are no true values in the marketplace. There are only realizations.
 
Maxim Dmitrievsky #:
It's just an element of the art of the master in MO, finding such connections. I do something similar on the basis of my experience. Often dependencies are searched through cross-entropy, which is resource-consuming. Is it faster for you?

About a second per predictor (XEON-1620).

 
Aleksey Nikolayev #:

Without stationarity of the series, statistical calculations like yours may be meaningless - there may be no convergence of the sampled values to the true values. For example, it is not hard to think of an example where the sample correlation of neighboring increments is nonzero, but the true correlation is zero.

PS.

Stationarity is understood "in the narrow sense" - the time independence of joint distributions.

Stationarity can be incomplete - for example, it can refer only to joint distributions of increments (processes with stationary increments).

It is of course correct to talk about the stationarity of a slope and not of a series which is only one of the realizations of a given process. But we are not at the exam, so it does not matter.)

Often by stationarity they mean its variant "in the broad sense". They remember only the constancy of the mean and variance, forgetting about the condition on the ACF. In any case, such stationarity is not sufficient in MO (it will be sufficient for linear models).

I'm not interested in the quotient itself. I'm interested in the predictor's ability to predict the teacher. To me the biggest mistake of the vast majority of traders is in their attempts to solve the problems of the quotient itself. And we need the prediction of the teacher. This is a completely different problem.

 
SanSanych Fomenko #:

I am not interested in the cotier itself. I am interested in the predictor's ability to predict the teacher. To me the biggest mistake of the vast majority of traders is in their attempts to solve the problems of the kotir itself. And we need the prediction of the teacher. That's a completely different problem.

what is this "teacher's prediction" ?

Reason: