Machine learning in trading: theory, models, practice and algo-trading - page 111

 
Mihail Marchukajtes:
And you keep trying to curb the market on minutes in 5 years?????? These 71 observations, two weeks of trading on 5 minutes if anything...... And only buying. So go for it..... Or are you deflated?
what's he talking about... that's crazy.
 
Dr.Trader:

As I said before, this metric is useless.

The data is randomly divided into 2 roughly equal parts, then the model is trained on the first part only, and tested on both at once. A generalizability of ~75% means that the model at the end correctly predicts 75% of all the examples on file at all.
There are several ways that the model can reach 75%:
1) The model has trained to 100% accuracy on the data used for training, and failed at all on the new data from the second part of the file, where it got 50% (the same as flipping a coin). The average would be exactly 75%. This is a very bad scenario, and it will be bad in the trade.
2) The model is trained to 75% accuracy on the training data and shows the same 75% on the test data, that is 75% on average again. In this situation, this is the best scenario, there is a chance to earn something.
3) Any option in between these two.

Your option is probably closer to the first. One has to rely very heavily on luck in order to trade with such a result, I assume that you have not lost the deposit only thanks to the indicator, which serves as your main signal (sequent, or whatever). I think an EA simply based on this one indicator will give you the same result as an indicator + jPrediction.

How do you know how to calculate the ability to generalize? This is known to Reshetov. I think that the calculation is based solely on test data, as he said before...... If the classical sequendas were workable I wouldn't use the predictor, but alas... it's as draining as anything. But the addition of the classifier quite improves it. Once again I will write that 71 observations is the work of TS for two weeks on 5 minutes. For this timeframe this is quite an acceptable interval. I am not used to making grids in half a year or more. I trained for 2 weeks, earned in a day and was done, while you are still looking for the grail. And yes, I train models every day. In the morning. The time of optimization is VERY acceptable now.....
 
mytarmailS:
What's he talking about... That's a load of crap.
Well, what don't you understand???? Or is it beyond your comprehension????
 
Mihail Marchukajtes:
Well, what don't you understand???? Or is it beyond your comprehension????

I ask you how you measure total capacity and you tell me about years of history and some other nonsense...

I can't measure general ability in one way and you in another, but you have no idea how to measure it, all you can do is look at the numbers injPrediction without the slightest idea where and how they come from, so when they ask you specific questions you start talking crap about years of history, etc. So stop it... Please...

 
mytarmailS:

I ask you how you measure total capacity and you tell me about years of history and some other nonsense...

I can't measure general ability in one way and you in another, but you have no idea how to measure it, all you can do is look at the numbers injPrediction without the slightest idea where and how they come from, so when they ask you specific questions you start talking nonsense about years of history, etc. So stop it... please...

I may have already explained that I use Reshetov's predictor, how he measures generalizability, these are questions for Yuri. Ask him. Although he gave formulas and in general I remember them, but I do not understand why you ask me. I'm just a user of his program and no more.....
 
Mihail Marchukajtes:
I think the calculation is based only on test data

If that's the case, I'm glad, it's much better.

In any case fronttest shows much better results. I divided your file into 2 parts (without shuffling, just in order), the first part has 50 lines, the second 19. So jPrediction has no access to examples from the second file, and it will be really new data for the model.

In the end, JPrediction gave the answer in only 9 cases in the second file. Right in 5 cases, wrong in 4. Accuracy is about 50%, nothing good in this result.

Files:
 
Dr.Trader:

If that's the case, I'm glad, it's much better.

Anyway fronttest shows much better result. I split your file into 2 parts (without shuffling, just in order), the first part has 50 lines, the second 19. So jPrediction has no access to examples from the second file, and it will be really new data for the model.

In the end, JPrediction gave the answer in only 9 cases in the second file. Right in 5 cases, wrong in 4. The accuracy is about 50%, there is nothing good in this result.

Agreed. We need to work on the input data.....
 
Dr.Trader:

If that's the case, I'm glad, it's much better.

In any case, fronttest shows much better results. Divided your file into 2 parts (no shuffling, just in order), the first part has 50 lines, the second 19. That way jPrediction has no access to the examples from the second file, and it will be really new data for the model.

As a result On the second file, JPrediction only gave an answer in 9 cases. Correct in 5 cases, wrong in 4. Accuracy is about 50%, nothing good about this result.

19, 50, who is more. Take any example from a database of datasets with at least hundreds of lines.

For me this software is not suitable, if only because I myself would prefer to pick up the parameters and break down the data. But as entry level I think it would be interesting.

 

Reshetov!

My offer still stands.

 
mytarmailS:

Hello Yuri !

There are questions )) about the sequential search ...

Let's say we have 10 predictors

1, 2, 3, 4, 5, 6, 7, 8, 9, 10

the green group of predictors is that group which has shown the best generalizing ability exactly to this group will be added other predictors N+1

the red group, it is the group which has shown itself a little bit worse, than the green group, and it(the red group) already will not take part in the tests, all tests are already focused on the green group

Question: what if after all the trials with other predictors one by one N+1 it turns out that in the end the red group has more generalizing ability, is it also quite real, or am I misunderstanding something ???? Please clarify.

If you want to get an unambiguous answer without looking at the data and algorithms, it is best to turn to SanSanych Fomenko, because he with a clever face will admonish you with "precise and valuable" instructions on any subject, regardless of its ambiguity.

And if you want a more accurate answer, then conduct an A/B test, i.e. in one case, try attaching red with black to the green, and in the second, only black. Whichever option gets the best generalizability from the results of the experiment is the most correct one for your problem.

The bottom line is that the results of the experience are always the criteria of truth.

For example, I was testing data centering for jPrediction today. The results were either dismal or slightly better on different samples. For the backpropagation grid, though, centering gives a noticeable improvement. I had to leave linear normalization.

And if I didn't do A/B testing, but instead of experience, took "ready knowledge" from some little book or lecture on machine learning, or asked some know-it-all, I'd get the answer that centering is supposedly "better" than linear normalization. Although experience shows that this is not unambiguously true for all algorithms.

That's the kind of pie.