Machine learning in trading: theory, models, practice and algo-trading - page 78
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
No, guys, the system's gone, it doesn't even make sense to show it.
Could it be because I forgot to warn that the ternary classifier has changed its logic?
Binary ones have the same logic: if the output value is less than zero, then it's like 0 in the sample. If it's more, it's 1 in the sample.
And ternary if the output is: 1 as 1 in the sample, 0 as 0 in the sample, -1 as "-" in jPrediction.
made for interest a committee of 8 models and simulated on validation a sequence of 4000 trades (5.5 years of trading). This is a Monte Carlo simulation to get an idea of the possible scope of deposit growth, without adjusting for initial trade points (which gives a run in the strategy tester). Here the MO on validation is 0.00026 points. The lower quantile is obviously above zero, which is what I wanted.
Next we will analyze the distribution of the recovery factor for these series.
This is the distribution of the recovery factor. The median of 3.7 is sort of the target value (plan). The lower quantile is not bad either...
It remains to wait for the training run on 80 more training samples and make a bigger committee.
Could it be because I forgot to warn that the ternary classifier has changed its logic?
Binary ones have the same logic: if the output value is less than zero, it's like 0 in the sample. If it is more, it is 1 in the sample.
And with ternary if the output is: 1 as 1 in the sample, 0 as 0 in the sample, -1 as "-" in jPrediction.
ҮNot that I understand, there's nothing wrong with that.... There's something else going on here. The point is that the Optimizer is very good at finding data containing output information. Once the data is found with a generalization above 80%. Then they, this data, can be successfully used in any other network topologies and the training result will be on top, without retraining, etc. This is for those who do not want to use the optimizer directly in trading. In my case it turned out that the data used for the output classification was not entirely successful, so the trading results on Friday were not what I would have liked. However, I didn't get depressed and found another set of predicates, which, in case of repeated model building, didn't lead to a state of garbage on entry and exit....... anyway here are some recommendations for using the classifier as a smart signal filter for your TS (not the forecast)
1. Separate the buy and sell signals separately. If you use data of previous values and such separation confuses the order of data, then obligatory one of the predicates should be the signal sign. Buy=1, Sell=-1, i.e. one of the predicates will be the direction of the signal.
2. The predicates must be somehow related to the prediction system. I have it is Zscore of signal, Kelli of signal, the order of the signal (the system is not reversible, so there can be a few consecutive signals in one direction). In general, the information that somehow relates to the system. You can even take the Momentum, separately in the optimizer MT4 picked up the parameter that gives a profit on the test period when trading by the system signals. That is, Momentum is optimized for the system signals on the training area, etc.
3. Output. This is the most important part, because at me the output of 100 pips gave 50% of profit generalization, and changed it to 10 pips and the generalization grew to 65%. In Demark's TS it is quite difficult to do. I often have to look through it manually, because the formalization of the approach in this case does not pass........ In the training area it is better to make mistakes in the signals, but to get a model with a high level of generalization. Let's say, the signal in the training has earned 100 pips, but we still marked it as 0 because the market was prone to the opposite side of the signal. I.e. we deliberately let the TS lose in order to increase the level of generalization and thereby make the TS work better in the future. After all, our output is ideal, let it be a little bit not ideal, but correct....
4. use the trading volumes with CME in predicates. This really works and increases the level of generalization of the model SIGNIFICANTLY...... I take from the delta cluster, so it's like this.....
This is a Monte Carlo simulation to get an idea of the possible range of deposit growth, without fitting the initial points of trades (which gives a run in the strategy tester).
I hope that no matter how you move the starting points of the trades, they are all after the training period. For example, if you trained the model on data from January and March 2015 (removing February data), then you should not do the final validation in February, even though this data was not in the training sample, but such results will not be reliable. Validation will show adequate results expected in real trading only if you start it in April 2015, after all the dates that could be used by the model in training or crossvalidations to select model parameters.
My second attempt (regression rounded to [0;0.5;1]) failed on your data. The model and genetics again chose the "no trade" constant class option.
Apparently, for predicting three classes the neuron must necessarily have 3 outputs, otherwise it is not able to fit three classes into one output with levels. Or I had to remove the "do not trade" class and leave two classes, and work with them.
In order to somehow check something on fronttest I took a list of predictors selected by genetics, trained a neuron (everything is correct, with three outputs and factor on the input) on them through caret with crossvalidations (method = "repeatedcv"). It turned out so-so, the accuracy even on the training data is only 40%.
I've just noticed that validation file has real numbers on its outputs, not classes. I just replaced them with +1 for all positive, and -1 for all negative.
Further, I got two accuracies for the fronttest:
1) given that at prediction=0 the model should not trade, then the correct result does not matter, because the trade will not be opened and will not affect the final statistics. In this case the predictions -1 and 1 are correct 51% of the time (ignoring all examples where prediction=0).
2) take all predictions, compare with all correct results. In that case, accuracy = 14%, even worse than random.
I am wondering what will be the result of trading the model in your trading emulator, could you please make a graph like above but trading with the model I attached?
The RData file was too big and the forum didn't accept it right away. I made two vinrar archives and put them separately in zip archives. You need to extract both zipped archives, then open first zipped archive, extract data, second zipped archive will hook up on its own and most importantly they should be in the same folder.
Or you can run the attached r script, it contains a list of selected predictors and will generate a model with them. You should have the original csv file that you posted, it is used.
The same script can be run after opening RData from the atachment, the script will take the already prepared model instead of creating a new one, in the code an example of prediction and accuracy estimation. The RData file you posted earlier is needed for the fronttest.
Given past mistakes, I'm starting my third attempt.
1) considering that at prediction=0 the model should not trade, then the correct result does not matter, because the trade will not be opened and will not affect the final statistics. In this case the predictions -1 and 1 are correct 51% of the time (ignoring all examples where prediction=0).
For a wrong answer I would score -2 points, for no answer -1, for a correct answer 1. The proportion of points received for the answers can be changed to one side or the other, but it is important that the network does not fails unnecessarily, so you need to punish for "abstention".
The committee of two nets, each giving 1 signal, -1 (or 0) no signal, if the signals were contradictory, then the final signal was 0. Thus the signals from the nets were combined into a common signal [-1, 0, 1].
I hope that no matter how you move the starting points of the trades, they are all after the training period. For example, if you trained the model on data from January and March 2015 (removing February data), then you should not do the final validation in February, even though this data was not in the training sample, but such results will not be reliable. Validation will show adequate results expected in real trading only if you start it in April 2015, after all the dates that could be used by the model in training or crossvalidations to select model parameters.
Yes, of course, you can be ironclad. This is exactly the validation. It's just that the sequences of deals are composed according to the principle of random sampling from a large amount of observations. Monte Carlo is needed to understand where the trade curve may appear. Any shift of trades by a few minutes in one direction or another generates its own curves. In the MT tester I see one curve - one realization. If I build in a condition with a random number generator on every trade and open with a random number, I can run the strategy tester 500 times and get the same curves. But in MT it would take a long time. And I need it in 30 seconds.
Then, when I make the EA on the committee, I will run the validation in the strategy tester. The curve should fit into a fan and I will get the unique value of the recovery factor and mathematical expectation. But it is impossible to draw conclusions from just one run. Not desirable.
I wonder what will be the result of trading model in your emulator trade, could you please make a chart as above, but trading with the model that I have attached?
RData file was too big, the forum didn't accept it right away. I made two vinrar archives and put them separately in zip archives. You need to extract both zipped archives, then open first zipped archive, extract data, second zipped archive will hook up on its own and most importantly they should be in the same folder.
Or you can run the attached r script, it contains a list of selected predictors and will generate a model with them. You should have the original csv file that you posted, it is used.
The same script can be run after opening RData from the atachment, the script will take the already prepared model instead of creating a new one, in the code an example of prediction and accuracy estimation. The RData file you posted earlier is needed for the fronttest.
Given past mistakes, I'm starting a third attempt.
And what model are you learning, the Neural Network?
Let me try that later. I have a model being learned on all cores around the clock.
Actually, I thought you'd be the judge of that. Just substitute the model predictions in the validation file I gave at the link. It should be directions like +1 / -1 / 0. There will be a new prediction column. Multiply the prediction column by the target column. Subtract the 0.0001 spread. Remove the entries where the model output is 0. Take the cumsum function:
plot(cumsum(your_data$trades_simulated_after_spread), type = 'l')
mean(your_data$trades_simulated_after_spread)
Roughly this will graph the trade over all observations and calculate the MO. It's simple.
And what model are you learning, the Neural Network?
...
Multiply the prediction column by the target column.
Yes, I'm learning a neural network, the nnet package. In this case, I'm training the model with caret, which picks up a couple of model parameters.
I looked again at the results column in your RData file, now I immediately saw that it was price gains. Last time I thought it was some kind of probabilities.
I liked how you store the price increment in the target variable, you can either easily convert them to classes for training if needed, or calculate profits. I'll be doing that as well.
Got it all, thanks for the clarification, counted the profit, took into account the spread, drew a graph. Too bad, MO = -2.052423e-05, slow but sure loss.
About the spread, it should only be taken into account in 4 cases:
1, 2) Forecast for the last bar = 0; forecast for the new bar = -1 or 1. Opening a new position.
3) Past bar prediction = -1; new bar prediction = 1. Reverse, the old position is closed, the new position is opened.
4) Forecast for the previous bar = 1; forecast for the new bar = -1. Reverse, old position closes, new position opens.
In all other cases, either the previous position from the last bar remains or the old position is closed, or no position at all. This all does not require a spread.
(More precisely, buy trades are opened at bid+spread, and are closed at bid. Sell deals are opened inversely on bid, and are closed on bid+spread. Bid is the price that is usually shown in the terminal. But, for the convenience, with a constant spread, it is enough to simply consider it once at the opening of any new position.)
This all should also be considered in your EA in order for it not to close the position that it will reopen immediately in the same direction on a new bar. I made such a mistake once, thinking that I will get rebates from my broker. But the frequency of transactions for rebates in this case does not pay off, everything is worse.
Yes, I'm learning a neural network, the nnet package. In this case, I train the model with caret, which picks up a couple of model parameters.
I looked again at the results column in your RData file, now I immediately saw that it was price gains. Last time I thought it was some kind of probabilities.
I liked how you store the price increment in the target variable, you can either easily convert them to classes for training if needed, or calculate profits. I'll be doing that as well.
Got it all, thanks for the clarification, counted the profit, took into account the spread, drew a graph. Too bad, MO = -2.052423e-05, slow but sure loss.
About the spread, it only needs to be taken into account in 4 cases:
1, 2) Forecast for the last bar = 0; forecast for the new bar = -1 or 1. Opening a new position.
3) Past bar prediction = -1; new bar prediction = 1. Reverse, the old position is closed, the new position is opened.
4) Forecast for the previous bar = 1; forecast for the new bar = -1. Reverse, old position closes, new position opens.
In all other cases, either the previous position from the last bar remains or the old position is closed, or no position at all. This all does not require a spread.
(More precisely, buy trades are opened at bid+spread, and are closed at bid. Sell deals are opened inversely on bid, and are closed on bid+spread. Bid is the price that is usually shown in the terminal. But, for the convenience, with a constant spread, it is enough to simply consider it once at the opening of any new position.)
This all should also be considered in your EA in order for it not to close the position that it will reopen immediately in the same direction on a new bar. I made such a mistake once, thinking that I will get rebates from my broker. But the frequency of transactions for rebates in this case does not pay off, everything is worse.
In order.
MO IS 0.00002. It is more than -spread. That is, the model has learned something from the noise, but the strength of the prediction is insufficient.
Second. Have you somehow specified which part of the data to slip into a certain fold? If you just default to CV (repeated CV) the model will take randomly selected observations for the test. Not the best option in case of time series.
Next, trading simulation. I have a simple signal for opening and closing after a certain number of minutes. And then it is possible to make it more complex. Therefore, I have a positive MO when taking into account the spread on each transaction. And this is real to get.
Do some work with crossvalidation. It is possible to create sets of indices to include certain parts of the time series in the fords.
And besides neuronics, there are a lot of other methods. One of the best ones today is either GBM or XGBOOST. I have chosen the first one so far. I like it.
I used the "repeatedcv" method in trainControl, with the default split. Recently I wrote code for crossvalidation myself, tried crossvalidation with just randomly taken bars, and with consecutively taken chunks without gaps. I didn't see any difference on the fronttest, it gave about the same result in both cases. I divided the data for training/crossvalidation 50%/50%, maybe at that ratio it doesn't matter anymore.
I'll experiment with that in caret...
I remember in that article you posted a while ago, the leading comparison was boosted trees with Platt's method (something like that). All I found on google about this method is that you have to pass the model output to sigmoid and take its result. Can gbm or xgboost do that? This approach seems to be better than forest, neuronka, and some "bagged trees", which come in second place.