Discussing the article: "Developing a robot in Python and MQL5 (Part 1): Data preprocessing" - page 6

New comment
 
About stationary signs, by the way, another observation.

The XGBoost model on tests, no matter how many times I run it, on different dates, shows one after another profitable forwards, dozens of times on different dates with current signs. Although I am a beginner, but not an idiot, I cut off training for 2007-2016, and then pure forward test. Accuracy of tags with risk reward trades 1:8 - 66% on forwards is average, sometimes XGB kicks out 72-74%. All other models, neural nets, all other bousting, random forests - pour ungodly.

I wouldn't even publish the first article of a cycle with current traits if they've been pouring in a used model on forwards for 10+ years.

There's a reason the XGB model beats a lot of forecasting and data science competitions. Apparently it somehow knows how to work with raw data. And it has cross validation built in. I was told about it by an acquaintance scientist, doctor of sciences, he uses it for forecasting, well, and trades too, he threw me a trading report with Profit factor of 55, but he disappeared from the Internet, he had a lot of developments, it is not clear what could have happened to him. Actually it was him and also super articles by Maxim Dmitrievsky that brought me to MO a couple of years ago.

But XGB takes a very long time to learn. The last model was trained for two days, I'm already tired of waiting, while a normal boolean usually learns in a couple of minutes on the server. But for me this is to some extent an indicator of the complexity and efficiency of the algorithm. I will go to rent a more powerful server for research.

I hope the ONNX version will not weigh too much and have too many lines. In MQL5 there is a limit on the number of rows of ONNX model data. I once trained a model with 100 million rows and was very upset about it, mcl does not let me use it) I think it will weigh a lot, and anticipating this I made a model for online trading directly through Python, it will appear in the next articles of the cycle, on Monday I will put it to test. The draft of the series is already done. I wanted to publish everything in one sheet at first, but the administration didn't let me, and it's probably right, it would be boring to read such a mess).

The idea of a scalper for Sber in Finam MT 5 has appeared, the point is that there is a function in the Python library mkle to get the history of the price stack, what if we train a model on it, and let it scalpit, the guys in the propahs are trading on the stack, and quite successfully. Judging by the experience of prop traders, I am not the first to have such an idea, and there have been a lot of such scalping algorithms on MO for a long time. Scalping is attractive because the profitability can be covered day in and day out. I have a Finam account as a foreign investor in Russia, albeit a small one) So maybe the last article of the series will be devoted to such scalper on MosBirch or CME, via Finam or AMP Futures Europe.

I also have a draft of computer vision on Python, I will make an article about it after the current cycle.

There are plenty of ideas, really plenty, every day a new idea appears and I sit down to code. Although my wife says, let's already make money, take the US Prop USA with what you have, make money. There are already dozens of trained models. But I'm more interested in research. I guess it's true, I should take an account and calm down. I took an account in February, and as luck would have it, there were problems with the props with meta quotas, I understand the props did not want to buy licences and cheated. As luck would have it, my prop removed MT 5, I used to trade with my hands through another terminal that they put instead of MT 5, and I ended up losing. I will take a new account in another prop with MT 5, and write news about it, how trading is going.

About the fact that we should limit the choice of the best predictors to the FORWARD date, it is a very good idea, I somehow missed this point).
 

Forward sample from 2010, training before 2010.

Example, I also implemented EXAMWARD sample to test the model on it separately, here is the result. ALL other models and neural networks have been pouring on these features since day one.

 

And this is how a simple forward differs:

And a forward with chips like cross-validation, model bagging (yes, this is all sewn into XGB by default I think, but I decided to implement it), grid hyperparameter enumeration, etc.:


 
Yevgeniy Koshtenko #:
If class labels are not reset, then the best feature for predicting labels will be the labels themselves, no?

You don't reset labels (which means reset - clear - as a synonym), but exclude the columns containing labels, and feed the labels themselves separately into the model as targets, i.e. information about them is not reset and does not disappear irrevocably, but is used when training the model.

 
Yevgeniy Koshtenko #:
The XGBoost model on tests, no matter how many times I run it, on different dates, shows one after another profitable forwards, dozens of times on different dates with current signs. Although I am a beginner, but not an idiot, I cut off training for 2007-2016, and then pure forward test. Accuracy of tags with risk reward trades 1:8 - 66% on forwards is average, sometimes XGB kicks out 72-74%. All other models, neural networks, all other bousting, random forests - pour ungodly.

There are a lot of rookie mistakes in the article - I wrote earlier, if you use the same code - there can be miracles.

Try trading your solution for a month on a demo, then compare entry points by adding a sample to test the model.

It is certainly interesting to read about XGB miracles, especially how you found hyperparameters - I read that it is very sensitive to them.

 
Rashid Umarov #:

Didn't mean to make that assumption so as not to offend you :)

Henceforth, check your source before rebuking

I have the terminal in portable mode installed, do I need to write the "portable" key in some way?

If the terminal is running - the code doesn't work, and if you switch it off, it tries to boot without the key, but it doesn't work either.

 
Aleksey Vyazmikin #:

I have the terminal in portable mode installed, do I have to write the "portable" key in some way?

If the terminal is running, the code doesn't work, and if I switch it off, it tries to boot without the key, but it doesn't work either.

Try to run it directly from the terminal. You just throw the script on the chart and it prints the results in the "Experts" tab

Perhaps you need to specify the path to the python folder in the meta-editor.

It works like that for me. But after a lot of dancing))))

 
Aleksandr Slavskii #:

Try to run it directly from the terminal. Just throw the script on the chart and it prints out the results in the "Experts" tab

Maybe you need to specify the path to the python folder in the meta-editor.

It works like that for me. But after a lot of dancing))))

Are you using terminal in portable mode?

In ME the path is prescribed (appeared automatically).

 
Aleksey Vyazmikin #:

Are you in portable mode using the terminal?

In ME the path is prescribed (appeared automatically).

I checked it in the portable version, everything works.

If two terminals are running and the path to the terminal is not specified in the script, an error occurs in one of the terminals when trying to compile.

 
Aleksandr Slavskii #:

Checked in the portable version, everything works.

If two terminals are running and the path to the terminal is not specified in the script, an error occurs in one of the terminals when trying to compile.

I ran it from the terminal

2024.04.01 17:22:57.397 synergy_ml_bot (EURUSD,Monthly) Number of instruments in the terminal: 140
2024.04.01 17:22:57.397 synergy_ml_bot (EURUSD,Monthly) Data for EURUSD not available (attempt 1)
2024.04.01 17:22:58.416 synergy_ml_bot (EURUSD,Monthly) Number of instruments in the terminal: 140
2024.04.01 17:22:58.416 synergy_ml_bot (EURUSD,Monthly) Data for EURUSD not available (attempt 2)
2024.04.01 17:22:59.416 synergy_ml_bot (EURUSD,Monthly) Number of instruments in the terminal: 140
2024.04.01 17:22:59.416 synergy_ml_bot (EURUSD,Monthly) Data for EURUSD not available (attempt 3)
2024.04.01 17:23:00.418 synergy_ml_bot (EURUSD,Monthly) Number of instruments in the terminal: 140
2024.04.01 17:23:00.418 synergy_ml_bot (EURUSD,Monthly) Data for EURUSD not available (attempt 4)
2024.04.01 17:23:01.421 synergy_ml_bot (EURUSD,Monthly) Number of instruments in the terminal: 140
2024.04.01 17:23:01.421 synergy_ml_bot (EURUSD,Monthly) Data for EURUSD not available (attempt 5)

The path was specified both ways to the terminal.

terminal_path = "C:/FX/MT5_02/terminal64.exe"
#terminal_path = "C:\\FX\\MT5_02\\terminal64.exe"

Somewhere it is looking in the wrong place - there is history in the terminal.

1234567
New comment