Machine learning in trading: theory, models, practice and algo-trading - page 755

[Mihail Marchukajtes]

Dr. Trader:

the tables can be joined together using the rbind() function
MLP1 <- rbind(Train1, Test)

What is Test(Train2)? And why combine the training and test table? The test should be separate, to test on it already trained model.

It will not come out as easy and fast as you want. In Reshetov's model, you can just feed the data, and the model itself will sift out the predictors, determine the weights, the number of neurons, and so on. With regular neuronics you can't do that, the neuron parameters must be selected through crossvalidation, like in the example I wrote here today, or as in the articles by Vladimir Perervenko.

Look again what I wrote there -https://www.mql5.com/ru/forum/86386/page753#comment_6821981
Just paste into that code your data as described, and run it. If predictors are taken all in a row and not sifted out, then from the first time it will probably come out bad. After running the code, run themax(gaResult@fitness) command additionally, this will show the R2 score of the best model found by genetics. If the estimation is close to 1 or a little less, good, if it is close to 0 or even negative, that is bad.

Then inPREDICTOR_COLUMNS specify specific numbers of columns that you identified as good, and once again run the code, the model will be trained only on these predictors. Ideally the best model score found by genetics should be close to 1, and the accuracy on the training and on the test table should be 1.00.

The test table - should be in time strictly after the training data, as if this model trades already in the market.

If nothing good comes out of that code (even withKFOLDS=10) - then it's no good. If it works, then there is a good alternative to Reshetov's model and we can further deal with that code and port the model to mql.

Actually there are only two parts of Train1 and Test1.

NetworkA learns from Train1 and is sampled by Test1

GridB learns on Test1 and is sampled on Train1.

When we add up the two plots. For GridA is Test1, for GridB is Trine1, then we will get the learning results for the entire learning plot, but that plot will be a test plot. That's what Reshetov's point is. I don't know, as you suggested I check..... kFold divides somehow by sections.... it is not clear.....

[Mihail Marchukajtes]

I don't want to repeat Reshetov's result, I want to organize AI like his. The way he trains the committee. It is clear that the training itself will be already available means, but the organization may play an important role....

I'll try to run your script on my data, see what comes out of it...

[Dr. Trader]

I see, it's almost the same thing. Reshetov's lines are taken one by one (training on even lines, testing on odd lines), and in my example the lines are divided in groups in a row (training on lines 1-100, testing on lines 101-200). This is better for forex, at least some check for the fact that the model may work in another time interval, unknown to it.

Reshetov-style splitting can also be done, thenerase in the functionelemnn_kfold

split2 <- function(x,n) split(x, cut(seq_along(x), n, labels = FALSE)) #функция  которая поделит строки таблицы на число кусков согласно KFOLDS
folds <- split2(1:nrow(x), KFOLDS) #номера  строк для обучения в каждом фолде

And instead of that, insert

folds <- list()
folds[[1]] <- c(which(y==1)[c(T,F)], which(y==0)[c(T,F)])
folds[[2]] <- c(which(y==1)[c(F,T)], which(y==0)[c(F,T)])

and be sure to change the number of folds to 2 -KFOLDS=2, otherwise you'll get some errors

[Dr. Trader]

Only we need to understand that "training" and "test" inside the crossvalidation fold are kind of nested checks. There is one train table, which is divided into several parts, and these parts will be either training or test, and models in the cycle learn from one part and predict others, to make sure that with these parameters the model is able to learn and predict something on new data for it.

And there is a separate test table that is no longer involved in this crossvalidation. It just waits until the model is fully trained to check it, like a test in a walk-forward that simulates real trading.
And if the roll forward gives normal results in each test when shifting the training and test windows by time, then the last trained model is not tested on a separate test table, and goes to trade in the terminal.

[Mihail Marchukajtes]

That's right, the Control sample, which can be a section of the OOS. Trained, tested. Thrown in the trade. I personally think so......

[Mihail Marchukajtes]

Mihail Marchukajtes:
All right, the control sampling, which can be a section of the OOS. Trained, tested. They threw the trade. I personally think so......

But here comes the question. How to choose a model without a control plot. Trained, received the results on crossvalidation, said this is the model that will score, put on the real!!!!!

As a rule I make a test plot within a day, it's 3-4 signals..... taking into account that it has to work for two weeks, it's not a big loss......

[Mihail Marchukajtes]

I ran your script with the above changes. It does not give errors, and even trains something there, but does not display any graphs in the process of training. One way or another, I see the result in the form of an error. I will remake my models, and then return to this script. Just when training sets will be formed, there we will continue....

[Vladimir Perervenko]

TO: Dr. Trader.

The script to understand in detail, is very "ragged" writing. Well, everyone has his own style. But there are a few important points. ELM has a number of features in the application:

• the more examples you give ELM in training, the less sensitive it is to noise (from experience at least 2000)
• in ensemble of ELM neural networks it must be 100+ for sane result (from experience it's better to take some redundancy and sift it out later)
• Combining by simple voting gives worse results than averaging. Although this is at the developer's discretion.
• seet.seed does not provide the necessary repeatability and diversity of hyperparameter optimization. Take a look at the comment to the article. There elibrarius tested this idea by experiment. Only by forcing RNG into a controlled state at every initialization of a neural network you'll get true optimization results. For me it is better to use Bayesian optimization. Gives a good choice of hyperparameters.

Good luck

[Dr. Trader]

Thank you for your comments, I will take it into account.

I added a link to your last article there at the end of the text, who wants to understand elmNN better go straight to the article.

Vladimir Perervenko:

the aggregation you use by simple voting gives worse results than averaging. It's up to the developer, though.

there are actually two ways of ensemble prediction

1) the elemnn_kfold_predict() function - the arithmetic mean for the regression results of each model is considered

2) function elemnn_kfold_predict_reshetovstyle() - I added this for Mikhail for his experiments, this way of combining ensemble results is used in Reshetov neuronics. Classification only. It compares results of all models, and gives an answer with a particular class only if all models gave the same prediction. If at least one model gave a different prediction than the others, the function returns a "don't know" prediction.

[[Deleted]]

Well, have you got the result yet? Where is the advisor?