Machine learning in trading: theory, models, practice and algo-trading - page 127

 
Alexey Burnakov:


How did you determine this? Have you tracked the performance of your predictors on the then unknown future?

Why not an unknown future? Everything is known on history. I learn it on the 1-10 chunk, and use them to check crossvalidation. Then I obtain statistics on 11, 12, etc. and compare with what I obtained on 1-10 segments. I.e. a complete analogy of real trading. I've described it a lot on this thread.
 
SanSanych Fomenko:
Why not a known future? Everything is known on history. I learn it on the 1-10 chunk, then I check it on crossvalidation. And then I get statistics on 11, 12, etc. and compare with what I got on 1-10 chunks. I.e. a complete analogy of real trading. I've described it a lot on this thread.

Yeah, well, that's what I figured.

And how many different variations of the model do you go through this way? Does it happen that on the 11th, 12th, etc. chunks the model performs worse than on the training chunk and you start over?

 
Alexey Burnakov:

Yeah, well, that's what I figured.

And how many different variations of the model do you go through this way? Does it happen that on the 11, 12, etc. chunks the model performs worse than on the training chunk and you start over?

The variation is about 2-3% with an average of 27-28%. It is the variation as a deviation from the average.

What you are writing about is supposed to reduce the learning error. I haven't done that. For my predictors, GBM is the best, then ada. I use rf. Next are SVM and worst of all nnet. As you can see I'm not using the best model.

 
SanSanych Fomenko:

The variation is about 2-3% with an average of 27-28%. It is the variation as a deviation from the average.

What you write about is that it should reduce the learning error. I have not done that. For my predictors, GBM is the best, then ada. I use rf. Next are SVM and worst of all nnet. As you can see I'm not using the best model.

Well, ok. SanSanych.

If all of your models are about equally well validated, there's no need to dance with tambourine.

I laid out the information to make sense of it. I use it, I test myself and save myself from mistakes.

Whether it will help you or not - I do not know.

I think the information is not the most useless.

 
Alexey Burnakov:

Well, okay. SanSanych.

If all of your models are about equally well validated, there's no need to dance with tambourine.

I laid out the information to make sense of it. I use it, I test myself and save myself from mistakes.

Whether it will help you or not - I do not know.

I think the information is not the most useless.

I have been following all your posts with great interest.

So, good luck.

 
SanSanych Fomenko:

I have followed all your posts with great interest.

Good luck.

Thank you.

I am, however, under the impression that you are predicting something of little relevance to trading. It can't be that your models get 60-70% so easily and still give accurate signals. We are fighting with overtraining here and you just looked at the data and said, no signal so no signal. Your work looks like Fisher's Iris prediction. I removed a bit of noise and everything is stationary. You, colleague, have not tried to predict complex processes. Where there is a lot of noise, you immediately refuse. And what is the benefit to you, except for customers' money, it is not clear at all.

Moreover, you do not know how your predictions worked in the future, although you talk about it confidently.
 
Alexey Burnakov:
Thank you.

I am, however, under the impression that you are predicting something of little relevance to trading. It can't be that your models get 60-70% so easily and still give accurate signals. We are fighting with overtraining here and you just looked at the data and said, no signal so no signal. Your work looks like Fisher's Iris prediction. I removed a bit of noise and everything is stationary. You, colleague, have not tried to predict complex processes. Where there is a lot of noise, you immediately refuse. And what is the benefit to you, except for customers' money, it is not clear at all.

Moreover, you do not know how your predictions worked in the future, although you confidently talk about it.

What is "complex processes" I do not understand. I have a real EA hooked up to a quote in my terminal... What they give me is what we chew.

I already wrote that I use prediction in TS on the major TF, which has three windows. Senior TF sets the direction. But apart from that the EA has quite a big set of tools, the purpose of which is risk management. That is why I cannot say for sure what percentage of errors in predicting that the next candle belongs to the direction of ZZ influences the total result, and I am not interested in that.

I repeat my previous post. The R tools, in particular, machine learning, I try to use to solve specific problems of a certain EA. Not in general, but a specific one. It is quite possible that in other TS my approaches will not work.

About the future...

It works since December and has a good flight. Unlike the indicator that was in this place in the Expert Advisor, there is a tool that has been retrained. The lag has been eliminated. Not enough? A colossal step forward. Is this the truth in the last resort? No, of course not.

 
Alexey Burnakov:

Yes.

One cycle of training with M1 crossvalidation on 10 fouls, you understand correctly. For each combination of training parameters: on 9 fouls we teach, on a delayed one we check. So 10 times. We obtain the average value of the quality metric at 10 folds deferred. Let's call this value m1.

I am doing a similar thing, but I am moving not a test window, but a training window. In the middle I will always have a moving training window, and test data surrounds it on both sides. I think that if the dependencies are not constant, and some "predominant periods" are constantly floating back and forth (according to mytarmailS, I myself in frequencies and Fourier can not) - then there is no point in training on extreme (in time) situations, and test on something in the middle that could be approximated by the model from extreme options. For training I take even much less than 50% of data, that's ok too (at least not worse)

I use something like outer loop from this picture, but I train on blue, and I test it on gray. I make sure that whatever 30% of sequential data I take for training, the model should be able to profitably trade on the rest of the data. There's a theory that if I can train a model that trades well during the whole period, using any 10% of successive data for training, then it will also trade well on new data.

 
Andrey Dik:

The method is as follows (all of the following applies equally to both ML and classic TS) .............

I've read it three times, but I don't understand it at least (

1) I guess this is not pure ML, it is kind of an improvement of some existing TS, that has signals for entry, and only on these inputs we enter and then we analyze ML, right?

2) When there is a profit we close the trade. When the trade is at a loss we hold the position, why would we do that?

3) When to buy, when to sell?

I can play the fool, I'm like me), but I don't really understand what you've written, maybe you can show me a picture or a scheme...

 

Came across the blog and almost cried, the man did almost the same idea that I once came up with and decided to implement, and that's the idea and I started learning programming about a year ago

https://www.r-bloggers.com/time-series-matching-with-dynamic-time-warping/

And I thought that my idea was unique))) youth, naivety... Of course I didn't use DTW, because I didn't know anything about it at the time

Time Series Matching with Dynamic Time Warping
  • systematicinvestor
  • www.r-bloggers.com
THIS IS NOT INVESTMENT ADVICE. The information is provided for informational purposes only. In the Time Series Matching post, I used one to one mapping to the compute distance between the query(cur…
Reason: