Machine learning in trading: theory, models, practice and algo-trading - page 2957

 
Aleksey Nikolayev #:

If someone solves such a difficult problem, it is unlikely that they will share the solution.

I've shared it so many times already, that at some point I got bored....

People start thinking and talking about what I discussed here years ago, but no one understands
 
Aleksey Nikolayev #:

If you remove the ONNX_NO_CONVERSION switch, you can input a double vector. The output should still be a float vector.

After compilation the model remains as a separate file or is sewn into .ex5 ?
 
Evgeny Dyuka #:
After compilation, the model remains as a separate file or is sewn into .ex5 ?

If it is not "stitched", how to sell it? I assume that it becomes a resource like other files.

 
Vladimir Perervenko dimensionality reduction and other engineering tricks. The main rule of any predictor and target transformation at this stage is that all transformation parameters (such as mean, median, sd, mad, etc.) are calculated on the training set. The test set and new data are processed using the parameters obtained on the train set.

So in the example it was necessary to calculate mean/sd on the train, pass this data to the part where the predicate is made, process the new data with these parameters. After that, denormalise and get the real values of the predicted price. This sequence is important.

It is clear that nobody feeds real prices to the input of the model, but this is already specific.

Good luck

In our onnx-model example, input prices are normalised using mean and std on the whole series. The obtained result is then varnormalised using the same rules

 
Evgeny Dyuka #:
After compilation, the model remains a separate file or is sewn into .ex5 ?

In my example no, it remains a separate file. In Renate's example (public project ONNX.Price.Prediction) it is written into .ex5.

 
Slava #:

In our onnx-model example, the input prices are normalised using mean and std over the whole series. The obtained result is then varnormalised using the same rules

Let me explain on my fingers. In ONNX.Price.Prediction.mq5 you get 10 OHLC. Then on this data you determine mean and sd, and normalise these 10 values with them. This is not correct.

For this new data, you should use mean and sd obtained on the training set. That is, in the previous script. Is that clear?

 
Vladimir Perervenko #:

I'll explain on my fingers. In ONNX.Price.Prediction.mq5 you get 10 OHLC. Then on this data you determine mean and sd, and normalise these 10 values with them. That's not right.

For this new data, you should use mean and sd obtained on the training set. That is, in the previous script. Is that clear?

Of course it is clear and it was done deliberately.

The example was made for the sake of testing onnx model loading, not for extracting a reasonable model result.

 
Evgeny Dyuka #:
After compilation, the model remains a separate file or is sewn into .ex5 ?

Use *.mqproj projects instead of single files, include your onnx models and other files as resources. This is now the preferred option for writing programmes.

Especially as it has more configurability and we will only in *.mqproj files increase configurability of programmes. Soon we will enable AVX/AVX2/AVX512 command set management when optimising robot code.

Resources are automatically embedded in EX5 file, compressed and encrypted for protection.

Check out the public project ONNX.Price.Prediction for an example.

Создание и работа с проектом - Проекты и MQL5 Storage - Справка по MetaEditor
  • www.metatrader5.com
MetaEditor позволяет удобно работать над большими проектами: объединять множество файлов в одну структуру, управлять настройками проекта и вести...
 
Renat Fatkhullin #:

Certainly understandable and done deliberately.

The example was done for the sake of testing onnx model loading, not to extract a reasonable model result.

Yes, of course I understood that. But did those using the example understand it?

Maybe I'm being picky.

 
Vladimir Perervenko #:

I'll explain on my fingers. In ONNX.Price.Prediction.mq5 you get 10 OHLC. Then on this data you determine mean and sd, and normalise these 10 values with them. That's not right.

For this new data, you should use mean and sd obtained on the training set. That is, in the previous script. Is that clear?

Of course you do. No problem. This model was made very quickly. Just as an example.

We needed to check the correctness of the inference in MQL5 very quickly.

And of course, we checked the effectiveness of the prediction on the current data. About 52 per cent. I wrote a verification script on Friday with detailed comments concerning technical details. Rashid wrote an Expert Advisor based on this model to run in the tester. Apparently it hasn't been published yet