Machine learning in trading: theory, models, practice and algo-trading - page 1638

 
elibrarius:
Too short an explanation.
It's not clear whether 1 predictor splits into several and is already being fed as 5 predictors on the outside. Rather it is still done internally as pre-calculated split values. And they divide by sectors.
I agree that this is more efficient than the half division in the classical tree algorithm.

What do you mean outside or inside? As I understand it, they take a predictor and try to divide its indicators into segments in order to maintain a sufficient number of activations and to give each segment some predictive ability, for this use different methods - with a given step or with a linear union of small steps (this if to simplify), we get such cells with ranges. When building all the trees in training, just sets of these cells are used. But, this is not exact :)

I, on the other hand, am trying to combine these cells into one. When watching this post yesterday, there's a mention that they do something similar for categorical predictors.

In my case there's a risk of overtraining - I'll check that on a sample a little later when the models are ready, and I'll do a sample at the same time to test it.

 

The almost complete ignoring of the non-stationarity problem in this thread is quite disconcerting. For some reason, it is assumed that the patterns found in the past will work in the future, and if they do not work, then retraining has occurred. But, it is quite possible that some patterns simply stop working over time - gradually or even by leaps and bounds (for example, as a result of a crisis such as the current one).

The problem I see is that MO patterns are complex and not well interpreted by humans. If they start to work badly, it is impossible to distinguish (within the models) the variant of overtraining from the variant of non-stationarity. In conventional thechanalysis, you can always say: "change of trend", "level/channel breakdown", etc.

 
Aleksey Vyazmikin:

With such a lady, chances are you will have to do laundry and eat cooking :)

Well it depends on how to fry it. A squeezed-out woman cooks better :-)
 
Mihail Marchukajtes:
It depends on how you fry her. A squeezed-out woman cooks better :-)

You won't get away with poetry with a lady like that, you'll have to learn math at the MHU level.)

 
Aleksey Nikolayev:

The almost complete ignoring of the non-stationarity problem in this thread is quite disconcerting. For some reason, it is assumed that the patterns found in the past will work in the future, and if they do not work, then retraining has occurred. But, it is quite possible that some patterns simply stop working over time - gradually or even by leaps and bounds (for example, as a result of a crisis such as the current one).

The problem I see is that MO patterns are complex and poorly interpreted by humans. If they start to work badly, it is impossible to distinguish (within the models) the variant of overtraining from the variant of non-stationarity. In conventional thechanalysis it is always possible to say: "change of trend", "level/channel breakdown" etc.

I'm not guessing... Practice shows 1% error in the training plot and 50% error in the new plot. I.e. we need significant predictors, and it is possible to train even with one tree or regression of some kind.

One tree, by the way, will be very easy to interpret.

 
Aleksey Nikolayev:

The almost complete ignoring of the non-stationarity problem in this thread is quite disconcerting. For some reason, it is assumed that the patterns found in the past will work in the future, and if they do not work, then retraining has occurred. But, it is quite possible that some patterns simply stop working over time - gradually or even by leaps and bounds (for example, as a result of a crisis like the current one).

The problem I see is that MO patterns are complex and not well interpreted by humans. If they start to work badly, it is impossible to distinguish (within the models) the overtraining variant from the non-stationarity variant. In conventional thechanalysis, you can always say: "trend change", "level/channel breakdown", etc.

I totally agree.

I have repeatedly wondered about this question and I believe it is necessary to compare the results of the system with its potential in a certain area.

I was just thinking about it today, how to do it better and more universally. I imagine the learning process to consist of several stages, the first of which is the partitioning of a sample, and you can partition it based on some signal strategies. These strategies should be primitive but have potential, for example, MA crossing by the price will generate a signal to enter in the direction of such crossover or vice versa. Then training is just a way to filter out false signals. If such assumptions are accepted, we can calculate the percentage value of efficiency of such filtration at each time section. The simplest one is to calculate the accuracy and completeness of classification relatively to the basic strategy. There are other options - metrics. Then we can see how the effectiveness of the model changes, even if it starts to lose money.

 
Aleksey Nikolayev:

The almost complete ignoring of the non-stationarity problem in this thread is quite disconcerting. For some reason, it is assumed that the patterns found in the past will work in the future, and if they do not work, then retraining has occurred. But, it is quite possible that some patterns simply stop working over time - gradually or even by leaps and bounds (for example, as a result of a crisis such as the current one).

The problem I see is that MO patterns are complex and poorly interpreted by humans. If they start to work badly, it is impossible to distinguish (within the models) the overtraining variant from the non-stationarity variant. In conventional thechanalysis it is always possible to say: "change of trend", "level/channel breakdown" etc.

I have some practice. I have not noticed any changes within a month since the last training, even after a strong plummet of the bitcoin. The only thing that affects it is the period immediately after the manipulative movement of the asset, at this time the neural network is completely lost and talks all sorts of nonsense, the further away from such a storm the more adequate the predicates become.
 
Evgeny Dyuka:
There is practice. I do not see any changes within a month since the last training, even after the bitcoin is heavily flushed. The only thing that affects me is the period right after the manipulative movement of an asset, during this time the neuronet is completely lost and displays all sorts of nonsense, the further away from such a storm the more adequate the predicates become.

- interesting to see... can i get a link to it (the channel) in a personal note?

- were you able to create in the end?

 
onedollarusd:

- interesting to see... can I get a link to it (the channel) in my personal?

- was it possible to create in the end?

- no, utopia, lots of time and effort, eventually on the backtest the bot makes up to X5 a year on one pair, but on average 1 time a year it all pours out. In the real market this "once a year" is bound to happen quickly, especially during storms like now. I don't believe in fully automated bots anymore, it just can't work, the market will adjust and cheat anyway)

- this one worked better, there is a working prototype at the moment.
Neuro estimates for BTCUSD pair for the next 10-30 minutes, it depends on "confidence" of the network. The higher the confidence, the higher the probability of repetition around 15 minutes, the lower the confidence, the more blurred the prediction. There is no binding to the candles, the prediction comes out every minute.

There is an Expert Advisor for MT5, which visualizes beautiful predictions, you can download here (those who downloaded it earlier can update it using this link, bug fixes were made).
It works only on BTCUSD and only on M1 timeframe, read the instruction.

This visualization shows that predictions are not ideal yet, but training is only at an early stage, everything is on its knees. There is an understanding of where to go next...

 
elibrarius:
I'm not assuming... Practice shows a 1% error in the training plot and 50% in the new plot. I.e. you need significant predictors, and you can train even with a single tree or some regression.

One tree, by the way, would be very easy to interpret.

Any predictors have a tendency to change their significance over time. It is not about their uselessness, but about the fact that it is necessary to constantly look for new ones and prepare for the loss of significance of the ones found earlier.

Reason: