Machine learning in trading: theory, models, practice and algo-trading - page 321

 
Maxim Dmitrievsky:

https://www.mql5.com/ru/code/127

There's just some error in the code, in my opinion, the indicator is normalized incorrectly

Everything is correct, it is just normalized not in [0;1] but in another interval.


For simplicity you can leave the same CalcRegression function that you have (just take into account that b already returns with opposite sign*, i.e. you don't need to change the sign for the second time). Find experimentally, by what value you need to multiply the result to get it in the range [-0.5;0.5]. And then add +0.5 to get the midpoint from 0 to 0.5.


*This formula will give you b with the right sign if the oldest prices have the highest index in the array.

 
Dr. Trader:

Everything is correct, it is just normalized not in [0;1] but in another interval.


For simplicity you can leave the same CalcRegression function that you have (just take into account that b in it already returns with opposite sign, i.e. you don't need to change the sign for the second time). Find experimentally, by what value you need to multiply the result, for it to fall in the range [-0.5;0.5]. And then add +0.5 to shift the middle from 0 to 0.5.


for different time scales have different max and min, not very convenient, it seems to me it is better to normalize on a large part of history
 
Dr. Trader:

*This formula will give you b with the right sign if the oldest prices have the highest index in the array.


just add another normalized buffer to the indicator over the existing buffer :)
 

The regression slope could be interesting, too. Backtest and forward. In short, it looks like you can work with this logic and experiment with different predictors.


Files:
 
Maxim Dmitrievsky:

The regression slope could be interesting, too. Backtest and forward. In short it looks like you can work with this logic and experiment with different predictors.


Thanks for the code, I tested it. I got worse. But I took only 1 month for optimization instead of year, I wanted to check it quickly. The fronttest shows my Expert Advisor is good at first, then it starts flattening balance and then it runs out of balance. It is still living a little on new data, not bad.

I would change this.

1) double a3 = regr[0]; - zero index is the oldest by time data. The most recent (new) bar here would be 49 (since 50 was taken), it's better. But the last one can be overdrawn, so 48 is better. It is also necessary to check if the indicator is overdrawn.
The same is with the rsi, you have removed the one that was newer of the three. It makes sense to try to remove the one with a zero index, as the most outdated.

2) The third version of Yuri's EA has the RNN function with four parameters which allows to take both rx and trend. In general, the function can be extended with new parameters, as written in Yuri's article, but each +1 parameter will double the number of coefficients.

3) I do not like stops and takeoffs, their optimum values are constantly changing on new bars. In fact, there are no optimal values, there are just some "convenient" values for the Expert Advisor on a particular time interval. Plus brokers can draw candlesticks before the stops. I would try to make a forecast at every new bar and just hold the trade in that direction until the next bar and forecast, and there flip if necessary.

4)

   int handle=iRSI(_Symbol,0,9,PRICE_OPEN);
   double rsi[30];
   CopyBuffer(handle,0,0,28,rsi);

The Handle should be initialized in OnInit, not every time you create an EA. And also the array has 30 elements, and only 28 are copied. I.e., you can't access indexes 28 and 29 (no one does, but you never know), they are not filled.

 
Dr. Trader:


Thanks for the code, I tried it out. I got worse. But I've taken only 1 month instead of year for optimization, I wanted to test it quickly. The fronttest shows how the EA is initially normal, then slowly starts to flatten the balance and then plummets. It is still living a little on new data, not bad.

I would change this.

1) double a3 = regr[0]; - zero index is the oldest by time data. The most recent (new) bar here would be 49 (since 50 was taken), it's better. But the last one can be overdrawn, so 48 is better. It is also necessary to check if the indicator is overdrawn.
The same is with the rsi, you have removed the one that was newer of the three. It makes sense to try to remove the one with zero index as the most outdated.

2) In the third version of Yuri's EA the RNN function contains four parameters, you can take with it 3 rsi and the trend at once. In general, the function can be extended with new parameters, as written in Yuri's article, but each +1 parameter will double the number of coefficients.

3) I do not like stops and takeoffs, their optimum values are constantly changing on new bars. In fact, there are no optimal values, there are just some "convenient" values for the Expert Advisor on a particular time interval. Plus brokers can draw candlesticks before the stops. I would try to make a forecast at every new bar and just hold the trade in that direction until the next bar and forecast, and there flip if necessary.

4)

The Handle should be initialized in OnInit, not every time you create an EA. And also the array has 30 elements, and only 28 are copied. I.e., indexes 28 and 29 can't be accessed (no one does, but you never know), they are not filled.


Hmm, yes? I thought the null when copying the buffer returned the very first value. I'll have to check...

https://www.mql5.com/ru/docs/series/copybuffer

I added trailing instead of sl and so on, the results are not much better

about the handle, yes, for regression i put it in the init, for ppi i forgot to do it

There i need to choose the forward with the highest profit factor and the backtest should be about the same, so with these parameters the grid is able to prognosticate ok. If I wanted to check if there is a trend there and it has changed on a forward... I would need a larger sample, yes... The shorter the period the more buy or sell orders prevail, it is optimized mainly for buy or sell, when the larger the period the number of buy and sell is equalized.

And RNN3 gives much less signals for some reason.

Also I don't quite understand, is it better for the normalization function to take 5000 bars array instead of 50, so it would find more correct max and min from the beginning and not update them with time, because in the beginning of testing we'll get not quite correctly normalized values for entry, but later more and more accurately

Plus, I would still detrend the graphs and feed more digestible values for the grid, I'm not sure yet how to better, say, regression slope and autocorrelation on a stationary series, since I'm not very good at econometrics, I'm watching clips now

And I also want to compare later this RNN with regular MLP, but there will be not quite correct comparison, because mlp will have to feed something to the output. Another option is to make a committee of 3 RNNs and feed their results into RNN :D, or into some convolutional NS. It will take a long time to optimize even through claud. Better yet, feed 3 RNN into MLP and give it an output of price increment, thus getting rid of overtraining, in theory. I.e. RNN will play a role of autoencoder http://cyberleninka.ru/article/n/avtoenkoder-podhod-k-ponizheniyu-razmernosti-vektornogo-prostranstva-s-kontroliruemoy-poterey-informatsii


 
Maxim Dmitrievsky:




To find a quote for which ARIMA can be applied is almost impossible. But that was the end of the lecture.

AND GARCH? It is much more promising and widely used on the timeframe less than a day, even for high-frequency trading.

 
SanSanych Fomenko:


To find a quote that can be applied to ARIMA is almost impossible. But that was the end of the lecture.

AND GARCH? It is much more promising and widely used on the TF less than a day, even for high-frequency trading.


I just started to get acquainted with these things, I haven't even experimented yet. And I've only heard about garbage :)

What do you think about the idea of using RNN as an autoencoder for MLP?

As far as I understood it, the recurrent network is an autoencoder of this type with a set of logical rules:


 
Maxim Dmitrievsky:

Well, what about the subtraction of trends and then the application of autoregression to them, and then shoving the values into the NS, together with the slope of the regression? I just started to get acquainted with these things, I haven't even experimented yet

better experiment right away!

You will save a lot of time and get rid of unnecessary knowledge

 
SanSanych Fomenko:


It's almost impossible to find a quote on which ARIMA can be applied. But that was the end of the lecture.

AND GARCH? It is much more promising and widely used in the timeframe less than a day, even for high-frequency trading.

I've seen it too. Imho, it is not our subject area.