Machine learning in trading: theory, models, practice and algo-trading - page 444

 
Yuriy Asaulenko:

That's right, the futures are traded for the last 3 months - immediately before and after the expiration of the previous one. It is useless to look before and try to do something with it.

Maxim (or his system) has noticed the obvious.

:)) I don't mean the futures, which have not yet started, but the cyclic behavior, probably related to the contract life, expressed in the fact that the patterns change in a new contract, such as volatility and liquidity, the market reactions change. It is invisible to the eye, by the way, it can be detected by statanalysis. But that is not the main thing, the market is full of all sorts of other cyclicities. The problem is that the system would rebuild itself without any loss of profitability on the automatic mode. That is, I would not have to do anything manually, I would be lazy...
 
Maxim Dmitrievsky:
:)) I don't mean the futures, which have not yet started, but the cyclic nature, probably associated with the lifetime of contracts, expressed in the fact that the patterns change in a new contract, such as volatility and liquidity, as well as the market reactions. It is invisible to the eye, by the way, it can be detected by statanalysis. But that is not the main thing, the market is full of all sorts of other cyclicities. The problem is that the system would rebuild itself without any loss of profitability on the automatic mode. That is, I would not have to do anything manually, I would be lazy...

Enter a parameter that will determine the quality of your EA, if its value deviates below the limit you set, retrain the model. All this is possible to do in R without leaving the working state.

What is the problem?

 

Warning. There may be errors in R.

As recently as yesterday I was building histograms. I've been doing it for hours, I didn't write any scripts, everything is on the command line. I have no desire to repeat hours of experiments just to post the codes.

So, let's make a histogram. According to the histogram the probability of finding the values of the series in a certain range is 0.22. This is fine, but according to the histogram itself there is something unlikely. Summing everything up, we get 0.59, not 1, as it should be.

Then we plot the same histograms for a sample of uniformly distributed random processes. Everything converges here, and the sum is 1, as it should be. How's that?

Okay, there are many ways to do the same thing in R. Let's do it another way. This time we get the probability 0.43. This can't be, because there is no way to get 0.43 from 0.22 (with the sum of 0.59).

Summing up the values of the histogram, we obtain 0.991. Recall, d.b. = 1. Even if everything is correct, we still get 1% error. This is an outrageous error - it simply can't be.

After this, the credibility of R somehow, you know, falls down. And if the same with other calculations and more complex algorithms, where to check is not so easy, and often impossible.

 
Yuriy Asaulenko:

Warning. There may be errors in R.

As recently as yesterday I was building histograms. I've been doing it for hours, I didn't write any scripts, everything is on the command line. I have no desire to repeat hours of experiments just to post the codes.

So, let's make a histogram. According to the histogram the probability of finding the values of the series in a certain range is 0.22. This is fine, but according to the histogram itself there is something unlikely. Summing everything up, we get 0.59, not 1, as it should be.

Then we plot the same histograms for a sample of uniformly distributed random processes. Everything converges here, and the sum is 1, as it should be. How's that?

Okay, there are many ways to do the same thing in R. Let's do it another way. This time we get the probability 0.43. This can't be, because there is no way to get 0.43 from 0.22 (with the sum of 0.59).

Summing up the values of the histogram, we obtain 0.991. Let me remind you, d.b. = 1. Even if everything is correct, we still get 1% error. This is an outrageous error - it simply can't be.

After this, the credibility of R somehow, you know, falls down. And if the same with other calculations and more complex algorithms, where it is not so easy to check, and often impossible.


I'd like to see something.

I build histograms regularly, but haven't noticed anything.

 
Yuriy Asaulenko:

Warning. There may be errors in R.

As recently as yesterday I was building histograms. I've been doing it for hours, I didn't write any scripts, everything is on the command line. I have no desire to repeat hours of experiments just to post the codes.

So, let's make a histogram. According to the histogram the probability of finding the values of the series in a certain range is 0.22. This is fine, but according to the histogram itself there is something that does not fit. Summing everything up, we get 0.59, not 1, as it should be.

Then we plot the same histograms for a sample of uniformly distributed random processes. Everything converges here, and the sum is 1, as it should be. How's that?

Okay, there are many ways to do the same thing in R. Let's do it another way. This time we get the probability 0.43. This can't be, because there is no way to get 0.43 from 0.22 (with the sum of 0.59).

Summing up the values of the histogram, we obtain 0.991. Let me remind you, d.b. = 1. Even if everything is correct, we still get 1% error. This is an outrageous error - it simply can't be.

After this, the credibility of R somehow, you know, falls down. And if the same with other calculations and more complex algorithms, where to check is not so easy, and often impossible.

Without reproducible example an empty sound. Obviously, the error probability in the basic functions is less than 0. Can't you draw the command sequence from history? Or you're not practicing in Rstudio?

 
Vladimir Perervenko:

Without a reproducible example there is nothing. The error probability in the basic functions is less than 0. Obviously, hours of experimentation have had an effect. Can't you draw the sequence of commands from the History? Or aren't you practicing in Rstudio?

In R 3.4.1 c https://www.r-project.org/

It takes at least an hour and a half just to reproduce a sequence (which I don't need anymore). Of course, I won't do it just for the sake of posting the results on the forum. Yesterday, when everything was, unfortunately, I did not think to write in the thread.

So I wrote as accurately as possible and without stating anything -Warning! There may be errorsin the R . Not have, but exactly may be.) In general, I do not insist. If you think it's good, then it is.

It is interesting that for some series in the construction of histograms, they are clearly there, and for other series, they do not seem. In the end, I had to go to Excel and finish it up there.

PS Yes, thanks for reminding me about RStudio. Already on it. Imho, it will be more convenient than using R from the CAD.

 
Yuriy Asaulenko:It is interesting, that for some rows when drawing the histograms, they clearly exist, but for other rows they seem to be absent.


setwd("D:/") # set the working directory
x <- read.csv("x.csv", head=T) # load data file
sapply(x, class) # which class the variable contents belong to

 
Vladimir Perervenko:

Enter a parameter that will determine the quality of your EA, if its value deviates below the limit you set, retrain the model. All this is possible to do in R without leaving the working state.

What's the problem?

I don't use R for its complete uselessness in developing bots :) If you need some kind of statistic you can, everything else is done in mt5, including neuronet libs you can use directly as a dll, what for do we need R here?
 
Maxim Dmitrievsky:
I don't use R for its complete uselessness in developing bots :) If you need some sort of statistic, you can, everything else is done in mt5, including neuronet libs you can use directly as a dll, what for is R here?

Obviously you don't need R. Good luck

-----------------------

SanSanych:

Since you switched to regression problems look at this

Good luck

Neural networks for algorithmic trading. Multimodal and multitask deep learning
Neural networks for algorithmic trading. Multimodal and multitask deep learning
  • 2017.07.09
  • Alex Honchar
  • medium.com
Here we are again! We already have four tutorials on financial forecasting with artificial neural networks where we compared different…
 
Vladimir Perervenko:

Obviously you don't need R. Good luck

-----------------------

SanSanych:

Since you switched to regression problems look at this

Good luck


I'm trying to master a very specific model - GARCH. It attracts me by the fact that the original series is decomposed into components and then these components are modeled separately. Decomposition into components is intuitively understandable and directly connected to retraining of model. Since I'm interested only in the model's retraining capability (I can create retrained Expert Advisors in TA with lifetime up to 6 months), that's what determined the choice of GARCH.

I am not aware of any approaches in NS that would allow to account for thick tails, kinks... It seems to me that the NS model itself has nothing at all to do with the problems of the original quotient.

In GARCH, while fitting the model, I can run tests that serve as a basis that in the future the resulting model will behave exactly as it does on the training data. I'm not able to fitting GARCH with parameters greater than 90% probability.