Machine learning in trading: theory, models, practice and algo-trading - page 120
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
The right function to assess the quality of the model is very important, because what we are looking for and trying to maximize is what we end up with. If you use model accuracy, you can easily slip into a local maximum where the model always returns the class with the largest number of examples. For training data with unbalanced classes it's better to use F-score or Cohen's Kappa. But even with them I had problems, because price growth per bar can be different, and even having a positive estimate, you can be in the negative - the number of winning trades will be more than losing ones, but a few losing trades can have large, irrecoverable losses. Trading simulation, and evaluating the model based on trading results is the right thing to do, I agree with you, I've recently started doing it too. You need profitability with a small drawdown - so model evaluation should use these concepts.
I think that MO is not a good choice because it does not take into account the drawdown which is important. I like Recovery Factor and Sharpe ratio the best of all functions available for estimating EAs in MT5.
Recovery Factor - final profit divided by maximal drawdown for the whole period. It is quite simple, but effective.
Sharpe Ratio - many descriptions on the Internet, but I didn't find many formulas. There is a code for MT4, I think we should transfer it to R and try it.
The parameter riskFreeYearlyIncome - annual return for risk-free investment (bank deposit), in the example it is 1%. Sharpe Ratio takes into account how much better the strategy is than such an investment.
double seconds = double(OrderCloseTime() - OrderOpenTime()) - how long a trade was opened in seconds. For bar trading it is respectively how many seconds in a bar.
1) Properly chosen function for assessing the quality of the model is very important, because what we are looking for and trying to maximize, that is what we get in the end.
2) But even with them I had problems, because the increase in price per bar can be different, and even having a positive assessment, you can be in the negative - the number of winning trades will be more than losing, but at the same time a few losing trades can have large, irrecoverable losses.
3) Trading simulation, and evaluating the model based on trading results is the right thing to do, I agree with you, I've recently started doing it too. You need profitability with a small drawdown - so model evaluation should use these concepts.
4)
I think MO is not a good choice because it does not take into account drawdown, which is important. From those functions that are available in MT5 for estimating EAs, I like Recovery Factor and Sharpe ratio the most.
Recovery Factor - final profit divided by maximal drawdown for the whole period. It is quite simple, but effective.
Sharpe Ratio - many descriptions on the Internet, but I didn't find many formulas. There is such a code for MT4, I think it is worth to transfer it to R and try.
1) Absolutely
2) exactly.
3) Yes, it helps to get rid of the starting-point bias in evaluation.
4) FS is the recovery factor. I have a lot of respect for it in general. I'll put it in the estimation function, it's easy.
Sharpe. It's easy! Let me explain.
a) You have got the vector of trade outcomes in pips, right?
b) We need to simulate trading with REINVESTMENT and convert the vector of points into a vector of percentage increments of the deposit. Example. We start with $100,000. We trade with the lot 0.1 and obtain 0.001 (ten points). This is 10 dollars per lot. So this value turns into 10/100,000. And so on, taking into account the increase or decrease of the lot.
c) On the resulting vector of percentage increments count: the average, st.deviation.
d) Enter a certain risk-free income per year (3%) - the number is conditional, a small number.
e) You must either translate your trades into average annual income, or divide the risk-free return per year by the average return per trade. Done.
f) (your average return per trade (per period) - risk free return) / st.deviation
I don't have the code for R yet. But it shouldn't be hard to write such a function.
Everything is retrained in the financial markets! Even what they say is not retrained is retrained.
Robust linear regression is retrained to a lesser extent, but gives such a shitty quality of learning that it also expands to new data in a shitty way.
Your idea of categorizing price increases has met with purely emotional rejection from the very beginning. If we predict trends, we look at ZZ and it is clear: here is one class, and here is another class. If we look at the graph of price increases, it is chaotic jerking around zero. Where are the clearly visible classes here?
This is pure emotion, which I attach great importance to statistics - everything should be clear and obvious without mathematics.
Besides, for many years ARIMA-ARCH has been developed and works quite well for price increases on financial markets. This opera even has models that consider Hearst. What is better than a classification?
Classification is the patterns - "head and shoulders", the search for which we have automated and which predict a clearly visible class. With classification, we find a pattern (in the form of a tree, for example) and say, "after this pattern there will be growth. If we formulated the target variable correctly, it will. Here ZZ is not the correct target variable. The idea here was to predict future growth of 100 pips. Such a target variable seems much more correct to me.
Conclusion. We should start from the target variable, the target is always our all, false targets lead to "swamp".
Your idea of categorizing price increases has met with purely emotional rejection from the very beginning. If we predict trends, we look at ZZ and it is clear: here is one class, and here is another class. If we look at the graph of price increases, it is chaotic jerking around zero. Where are the clearly visible classes here?
This is pure emotion, which I attach great importance to statistics - everything should be clear and obvious without mathematics.
Besides, for many years ARIMA-ARCH has been developed and works quite well for price increases on financial markets. This opera even has models that consider Hearst. What is better than a classification?
Classification is the patterns - "head and shoulders", the search for which we have automated and which predict a clearly visible class. With classification, we find a pattern (in the form of a tree, for example) and say, "after this pattern there will be growth. If we formulated the target variable correctly, it will. Here ZZ is not the correct target variable. The idea here was to predict future growth of 100 pips. Such a target variable seems much more correct to me.
Conclusion. You have to start with the target variable, as always the target is our everything, false targets lead to the swamp
I don't know what's emotionally wrong with that. That's one way to look at it. The color of the candle is also called that.
Let me remind you that the regression and classification tasks were a fitting before the stage of building a profitable system.
Next, I came to the point where I predict not just the sign of growth, but growth with a larger modulus. That is, entering buy only where the system predicts strong gains, not just gains. Also with the sell. Technically, the machine learns a regression. And then I interpret its output as strong up, strong down, dangling in a channel.
Now that's pretty close to predicting a future 100 pips rise. It's just not clear to me myself how you see the formalization of such a targeting. What does it mean if before the growth there was a 90 points drop, and if this growth is reached only in a week?
I have a strong growth of about 20 pips on the horizon of 9 hours. After 9 o'clock the trade is closed.
And in fact, I am not predicting the class at all right now (up / down / dangling / strong rise). I'm actually building a system whose evaluation at the training stage is measured by the maturity expectation of the trade. That is, classification accuracy may be low (53% for me), but hitting in the right direction gives a slightly higher gain than not hitting. Do you feel the difference in the problem statement? Classification accuracy correlates but does not fully coincide with trade profitability.
By the way, the recovery factor as a function of model quality assessment is also one-sided. I can get 10 trades and 1 of them unprofitable, or 500 trades and 10 unprofitable. In the second case the FS may be lower and the profit 50 times higher.
Alternatively, you can take the total profit and subtract from it the max drawdown. Here at least the profit will be maximized.
Or the traditional profitability factor: total profit / total loss.
In the second case, the FS may be lower, but the profit is 50 times higher.
It's all about the risk you want to take. For example, there are two signals: the first brings 100% profit per year with a maximum drawdown of 10%. The second signal is 300% of profit with a drawdown of 40%. So the second signal is the leader in terms of profit.
But if you calculate PV, the leader will be the first signal, and it is right, because you can simply increase the size of transactions (in lots) of the first signal in four times, which will give the same drawdown as the second signal, but with greater profits. Or if you don't want to risk a large percentage of the account balance, you can reduce the size of lots of the second signal by 4 times, which will give the desired 10% risk of the balance, but the profit will also be less than the first signal
That is, the trading plan for the strategy may be as follows: Determine the maximum possible drawdown when trading on the strategy by a walk-forward; calculate how many times this drawdown is greater/lower than the value you allow for the balance in your account; calculate the lot size so that these numbers are convergent.
Using FS implies that before you start trading, you will calculate all risks and determine a suitable lot size. The smaller the drawdown the strategy has when trading, the larger the amount of trades you can use.
Or the traditional profitability factor: total profit / total loss.
IMHO, more acceptable is:
criterion_evaluation_TC = sum_of_profitable_transactions / (sum_of_profitable_transactions + sum_of_loss_transactions)
on condition of trading with a fixed lot.
In this case, if the criterion is higher than 0.5, then the TS shows the final profit, otherwise it shows a loss.
The advantage of the criterion is that it is strictly normalized and its value cannot exceed the range from 0 to 1.0 inclusive.
Drawdowns, even in spite of the final profit of the trade, have a negative impact on the value of this criterion, i.e. they are considered as a negative factor.
IMHO, more acceptable is:
criterion_evaluation_TC = sum_of_profitable_transactions / (sum_of_profitable_transactions + sum_of_losing_transactions)
on condition of trading with a fixed lot.
In this case, if the criterion is higher than 0.5, then the TS has shown the final profit, otherwise - the loss.
The advantage of the criterion is that it is strictly normalized and its value cannot exceed the range from 0 to 1.0 inclusive.
Drawdowns, even in spite of the final profit of the trade, have a negative impact on the value of this criterion, i.e. they are considered as a negative factor.
IMHO, more acceptable is:
criterion_evaluation_TC = sum_of_profitable_transactions / (sum_of_profitable_transactions + sum_of_losing_transactions)
on condition of trading with a fixed lot.
In this case, if the criterion is higher than 0.5, then the TS has shown the final profit, otherwise - the loss.
The advantage of the criterion is that it is strictly normalized and its value cannot exceed the range from 0 to 1.0 inclusive.
Drawdowns, even in spite of the final profit of the trade, have a negative effect on the value of this criterion, i.e. they are considered as a negative factor.