Machine learning in trading: theory, models, practice and algo-trading - page 2733

[Maxim Kuznetsov]

mytarmailS #:

because you don't hit stop learning before learning is complete .

Hurrah! It worked... 3 hours of learning, I almost forgot about it.

Now the cockroach runs as it should (almost) tries to avoid green ones and eats red ones.

Apparently it so happened that one "feeler" less feeler trained :-) That is, if a penalty green is on a certain side, it will be eaten immediately. Conversely reds on a certain traverse are avoided

But this is probably a system ogrich.

[Aleksey Vyazmikin]

Aleksey Nikolayev #:

This works only in the case of independent features, and since they are counted at the same price, it is not possible. In the case of dependence it is much more complicated - we can take copulas as an example, where univariate distributions are always the same uniform, but at the same time bivariate distributions can be very different.

Maybe there is a solution on the same R, but you just need to look for it?

Aleksey Nikolayev #:

You have a predilection for heavy enumeration calculations) We will have to add (to the already considerable amount of enumeration) enumeration by feature types and, for sure, by feature parameters.

Nevertheless, it seems to me that there is a rational grain in your approach, there is something to think about.

Yes, apparently I am bad at predicting outcomes without experimentation, so it is better to test and often be disappointed than to build multi-stage logical calculations in which an error may creep in at one of the steps.

Even if you learn to divide a sample into sub-samples, the next question is how to correctly classify the current sample and apply the necessary model on it.

[Aleksey Vyazmikin]

mytarmailS #:
Maybe instead of statistical criteria of sample homogeneity just watch the change of feature importance of the model in dynamics (in a sliding window).

If there is a strong discrepancy between the current state and the previous state, it means that we are already in another sample....

Pros:

1.You don't have to programme the stat. Tests, everything is ready out of the box

2. It takes into account not only the change in sampling time, but also the change in target, which I think is not less important

At what moment to watch, at the moment of training? And how do you suggest to watch in dynamics, how to implement?

I have a script that shows the activation of leaves of the model in dynamics, i.e. you can see which leaves are triggered. Maybe we should calculate the average frequency of their activation on the training segment, and then look at the deviations on the test segments? In this way, we can at least try to monitor the decrease in model efficiency, theoretically.

[mytarmailS]

Aleksey Vyazmikin #:

At what moment to watch, at the moment of training? And how do you propose to watch in dynamics, how to realise?

I have a script that dynamically shows the activation of leaves of the model, i.e. you can see which leaves are activated. Maybe then we should calculate the average frequency of their activation on the training segment, and then look at the deviations on the test segments? In this way, we can at least try to monitor the decrease in model efficiency, theoretically.

There is no point in looking at the old model, it does not capture changes in the market....

I propose to implement as suggested))))))

In a sliding window retrain the model and watch the importance of signs, or just take some determinator of good signs and watch it in a sliding window. window

Here is an example on a randomly generated sample of 5 features and 1 binary target.

forrest and fiche selector

X <- matrix(rnorm(1000),ncol = 5)
Y <- as.factor(sample(0:1,nrow(X),replace = T))

head(X)
head(Y)

#install.packages("randomForest")
library(randomForest)
rf_imp_mat <- matrix(ncol = ncol(X),nrow = nrow(X))
for(i in 30:nrow(X)){
  ii <- (i-29):i
  rf <- randomForest(Y[ii]~.,X[ii,],ntree=100)
  rf_imp_mat[i,] <- importance(rf)[,1]
}

#install.packages("FSelectorRcpp")
library(FSelectorRcpp)
fs_imp_mat <- matrix(ncol = ncol(X),nrow = nrow(X))
for(i in 30:nrow(X)){
  ii <- (i-29):i
  infg <- information_gain(y = Y[ii],x = as.data.frame(X[ii,]))
  fs_imp_mat[i,] <-  infg$importance
}

par(mfrow=c(1,2))
matplot(rf_imp_mat , t="l",lty=1,main="rf_var_imp")
matplot(fs_imp_mat , t="l",lty=1,main="fs_var_imp")

In R-ka of course it is not customary to write in cycles if it is not necessary, but this style confuses newcomers, and middlemen like me also confuses....

but you can write like this, the code is 3 times less, and the result is the same.

X <- matrix(rnorm(1000),ncol = 5)
Y <- as.factor(sample(0:1,nrow(X),replace = T))
idx <- embed(1:nrow(X),dimension = 30)[,30:1]

library(randomForest)
rf_imp_mat <- t(apply(idx,1,function(i) importance(randomForest(Y[i]~.,X[i,]))[,1]))
library(FSelectorRcpp)
fs_imp_mat <- t(apply(idx,1,function(i) information_gain(y=Y[i],x=as.data.frame(X[i,]))$importance))

par(mfrow=c(1,2))
matplot(rf_imp_mat , t="l",lty=1,main="rf_var_imp")
matplot(fs_imp_mat , t="l",lty=1,main="fs_var_imp")

Also different selectors for any taste, probably 5% of what is available in R-ka.

[Maxim Kuznetsov]

or look at windows on two or three currencies, and trade with a delay in the third.

i.e. look at the second half of the day EUR,CHF to USD CAD and open early in the morning JPY (AUD,NZD)...so that the system learnt to determine the trend of USD and had time to buy before "it is not rotten".

I'm just afraid that in a single quote "all paths have been travelled" and it's impossible to get fish out of it using available methods.

[Maxim Dmitrievsky]

To invent something you need to define the object of research, then determine its properties, then it will not be like a monkey and glasses

[Evgeny Dyuka]

Maxim Dmitrievsky #:
To invent something you need to define the object of research, then define its properties, then it will not be like a monkey and glasses

It is too early to define the object of research and its properties, the forum is only 6 years old. Don't be so nasty!

[Maxim Dmitrievsky]

Evgeny Dyuka #:

It is too early to define the object of research and its properties, the forum is only 6 years old. Don't be so nasty!

If you start to study quotes as a time series, you may notice some peculiarities that are not found in other time series. Perhaps there are patterns in these features. And yes, not everything can be pulled out by autoregression and classification directly using lag features, but with the addition of ingenuity you can

[Renat Akhtyamov]

Maxim Kuznetsov #:

(I hope they don't shoot you for the CodeBase link : https://www.mql5.com/ru/code/36558

may come in handy in predicting signs - you are welcome to predict :-) the indicator just shows (and summarises) the "black/white" signs.

I'll show the bar signs here, it's not a pity.

I'll dig through my modest archive of developments and post them.

It's interesting, of course, I don't argue, but there is non-stationarity there too.

that's why it's predicted with a probability of a little less than one.

but basically, if it's up now, the next bar will be down.

and this kind of movement of the younger TFMs is reflected in the older ones.

so the trend is not a straight line, but with a lot of pullbacks, multiple in duration and bar size of the younger ones.

However, it would seem that you can apply Fourier and find those waves, but it's not that way, because

the time scale is modulated in the same way, backwards and forwards.

It is not easy to understand this miracle, and if you do it head-on, there are a lot of questions.

For example, we can consider the price movement from left to right or from right to left, starting not necessarily from the right edge.

so you get forward and backward, up and down.

[Renat Akhtyamov]

I couldn't find it.

Very much and therefore tedious to dig in it

But once posted one of my indicators and said grail type, which at that time was already about 7 years old, as I posted on the forum

Throw it into neuron, maybe it will work....