Machine learning in trading: theory, models, practice and algo-trading - page 318

 
kaus_bonus:


are you serious?)) the left hand is fighting the right hand?) then who will sponsor elections, etc.

http://www.rbc.ru/finances/19/10/2015/5624cf299a79472c1c14ac85

etc.


Well they ended up making their own losses, and it's not the only big player in the market. Random can have different investment horizons, and the distribution of profits and losses on each investment horizon is about equal, the probability of shaving off the black swan is the same, as E. Peters wrote about, for example. And the fight against monopolies is the key role of the state and the competitive market, otherwise we would be buying bread at $1000
 
Oleg avtomat:

If the problem is formulated as the problem of determining the control signal

Thanks for the wording. I haven't yet set new tasks relying on the manageability of forex, but this one sounds like something I could set myself in a few steps.


SanSanych Fomenko:

So what's the solution?

My first thought was to walk away from forex, to trade cryptocurrency since it's all so free and independent. But since crypto price is usually expressed in dollars - then btc/usd price will also get bad unpredictable properties from usd, too bad. I think it's worth trying to trade crypto crosses - btc/ltc for example (bitcoin-lightcoin). I've seen some broker advertisement that they have btc/usd in MT4, I should look for btc/ltc in MT5. In worst case you can go directly to bitcoin exchange, but you have to write your own program to trade robot through their rest api.

There are also a couple of ideas how to make classification models work in forex. Classify first the current control type (according to my experience - the type changes randomly to some previous one. There don't even seem to be many of them, only four pieces. But this is not exact). There should also be a trained trade classifier for each type of control. And then to make a buy/sell prediction with the corresponding classifier. Somehow it is too complicated, I need a lot of new bikes :)

 
Dr.Trader:

Thanks for the wording. I haven't yet set new goals based on the manageability of forex, but this one sounds like something I could set myself in a few steps.

A properly set goal is already halfway done.

I wish you success!

 
Dr.Trader:

Thanks for the wording. I haven't yet set new goals based on the manageability of forex, but this one sounds like something I could set myself in a few steps.


My first thought was to get away from forex, to trade cryptocurrency since it's all so free and independent. But since the price of crypto is usually expressed in dollars - then the price of btc/usd will also get bad unpredictable properties from usd, too bad. I think it's worth trying to trade crypto crosses - btc/ltc for example (bitcoin-lightcoin). I've seen some broker advertisement that they have btc/usd in MT4, I should look for btc/ltc in MT5. In worst case you can go directly to bitcoin exchange, but you have to write your own program to trade robot through their rest api.

There are also a couple of ideas how to make classification models work in forex. Classify first the current control type (according to my experience - the type changes randomly to some previous one. There don't even seem to be many of them, only four pieces. But this is not exact). There should also be a trained trade classifier for each type of control. And then to make a buy/sell prediction with the corresponding classifier. Somehow everything comes out too complicated, you need a lot of new bicycles :)

If we talk about classification models, we need to look for predictors that have approximately equal predictive power over large time intervals.

At the moment we are stuck with all sorts of derivatives of the most predicted currency pair.

And if we take other currency pairs. I've been doing this. There are currency pairs that have a predictive ability for EURUSD, and there are pairs that don't predict this currency pair at all.

But there is a more serious thought.

The thing is that the currency rate, for example USD, directly depends on different macroeconomic indicators: the Fed rate, GDP..... It is quite possible that the pattern changes you have identified are due to changes in macroeconomic indicators. They have a periodicity of a month is very fast. Usually a quarter or a season.

 

Greetings to all interested and participating in the discussion of this thread! I read the topic from beginning to end, with a few gaps. There are interesting ideas and thoughts here. I want to express my own point of view and approach to the problem, in particular how to approach the market and what to try to get from it.

I have enough trading experience to position myself in the market without any indicators or other technical means. The idea of turning my vision/experience into a mathematical model emerged. I realized at once that I would not be able to translate all the nuances into numbers, first of all due to my lack of professional programming skills and, most importantly, knowledge in mathematics, statistics, physics, etc. I took this job anyway.

I finally created an indicator (Fig.), which has reflected my approach to the market, to its prediction. I will briefly describe its operation. First we preprocessed quotes and then created so-called "ideal" model with good smoothness and minimal kinks. But this model, having these advantages, has a great disadvantage - it lags behind by a decent (10-12) number of bars. The next task was to compensate for the lag of the "ideal" model. This task is still being solved. But there are results. In Fig. you can see predicted values of the "ideal" model, the yellow one is 5 bars ahead and the red one is 7 bars ahead. I cannot move forward even 1 bar using past data, as there are a lot of false positives and affect smoothness. Thus, I have drawn the maximum from the history. In my opinion, a very complex mixture of predictors from volumes to correlation relations between a traded symbol and dozens of other symbols are involved in forming the next bars. (And I want to specify that correlations are very short-term, I have not been able to identify a more or less long term relationship).

From all this work, I have a definite opinion about how the market moves. We ordinary traders do not have complete information about the state of the market at the current moment, those who move the market, we will constantly lag behind. But it seems to me that it is possible to approach the zero bar, but it will require such resources and knowledge, that it is almost impossible for an ordinary trader to do.


 
Egor Manakhov:

Greetings to all interested and participating in the discussion of this thread! I read the topic from beginning to end, with a few gaps. There are interesting ideas and thoughts here. I want to express my own point of view and approach to the problem, in particular how to approach the market and what to try to get from it.

I have enough trading experience to position myself in the market without any indicators or other technical means. The idea of turning my vision/experience into a mathematical model emerged. I realized at once that I would not be able to translate all the nuances into numbers, first of all due to my lack of professional programming skills and, most importantly, knowledge in mathematics, statistics, physics, etc. I took this job anyway.

I finally created an indicator (Fig.), which has reflected my approach to the market, to its prediction. I will briefly describe its operation. First, pre-processing of quotes is performed and then the so-called "ideal" model is created, which has good smoothness and minimal kinks. But this model, having these advantages, has a great disadvantage - it lags behind by a decent (10-12) number of bars. The next task was to compensate for the lag of the "ideal" model. This task is still being solved. But there are results. In Fig. you can see predicted values of the "ideal" model, the yellow one is 5 bars ahead and the red one is 7 bars ahead. I cannot move forward even 1 bar using past data, as there are a lot of false positives and affect smoothness. Thus, I have drawn the maximum from the history. In my opinion, a very complex mixture of predictors from volumes to correlation relations between a traded symbol and dozens of other symbols are involved in forming the next bars. (And I want to clarify that correlations are very short-term, I was not able to identify a more or less long term relationship).

From all this work, I have a definite opinion about how the market moves. We ordinary traders do not have complete information about the state of the market at the current moment, those who move the market, we will constantly lag behind. But, in my opinion, it is possible to approach the zero bar, but it requires such resources and knowledge, which are almost impossible for an ordinary trader.



Interesting!!!!! Your indicator smoothes collars well, but the line itself has no forecast. That is, it follows the principle where the fig goes, there goes the smoke. As a rule these TS are sensitive to the number of false signals. I wonder how you got the forecast values????
 

the guy just decided to sell his junk .... Mashka some ... on the sly ... like really wants to keep the conversation) ... well ...

 
SanSanych Fomenko:

The fact is that the exchange rate, for example USD, directly depends on various macroeconomic indicators: the Fed rate, GDP.... It is quite possible that the pattern changes you have identified are related to changes in macroeconomic indicators. They have a periodicity of a month is very fast. Usually a quarter or a season.

I made some more experiments with pattern recognition. The essence of the model is the following: take increase/decrease of the price in tens of bars (pattern), find similar patterns from the previous weeks, look how the price behaved after the similar patterns before and trade according to these observations. The model has a lot of different parameters for optimization, such as the length of the pattern (in bars), how far to go in the history when searching for similar patterns, different coefficients, etc.
The "similarity" of the patterns is defined through Cartesian distance, as mytarmailS suggested in this thread.

If you take a small period for training, say a week, then by adjusting parameters of the model you can make profits go up during this whole period. But, as I wrote earlier, this model will generate profits and losses not randomly on new data, but periodically. A week on the plus side, a week on the minus side, a couple of weeks just a slow drain on the spread. And these cycles of profitability or strong loss will sometimes appear in the future, even months later. This is very different from conventional models like neuronka or scaffold, which will evenly and slowly drain the spread on new data. I liked something about this model, it is as if it shows the hidden cycles of forex, you can see how the price reaction to the same patterns changes drastically. Instead of randomness of results (like neuronics) their cyclic (but uneven) deterioration and improvement appears. Unusual.

The new experiments present an even more puzzling situation - the model parameters allow me to obtain its profitability on data of any time interval, for example on a week or a month. But, no matter how long a training interval is taken, there will be no stable profit on the new data. If we take a week of data for training, the periods of profit and loss will also be weekly, and you can't know in advance if the next week will be profitable. If we take a month of training, the periods of profit and loss will also be months. This is bullshit :) About the classification of the current type of forex management I think I was wrong, that's not the point, it can not depend on how long the data interval I took for training. You have to leave sense and logic to understand how it works at all :)

For now the only idea is to increase the number of bars in the pattern.
By analogy, if for example "head and shoulders" pattern was profitable in March and was losing in April, it is obviously not enough to make a decision. It is necessary to look at the figures that preceded it, and as a result it may turn out that for making a decision we should find the three previous figures on the history and make a decision by their combination.
It may work.
But here appears the paradox - the Occam's principle states that if I can teach the model to trade on the profit using the patterns of ten bars, then I should not use one hundred bars. And my conclusions suggest that I should.

No conclusions. I keep on picking forex.

 
Mihail Marchukajtes:

Interesting!!!!! Your indicator smoothes collisions well, but the line itself has no predictions. That is, it follows the principle "where the fig goes, there goes the smoke. As a rule these TS are sensitive to the number of false signals. I wonder how you got the predicted values????


Yes, the smoothing is decent, but the phase delay is not small. Again, the main idea is to get maximum smoothness and minimum fractures. Then by training the neural network and linear regression try to recover this model gradually moving to zero bar keeping smoothness. (I need smoothness mainly for algorithmic trading and minimum number of breaks in order to have something to "bite" from the market even if I recovered the phase for 7 bars from 10-12)

Predicted values in the form of yellow and red line are obtained by neural network training, using "ideal" line as a target function, and predictors are polynomials overlapping the sample by amplitude and by phase. But training "yellow" and "red" model is a little bit different, let's say even because I used "yellow" model as a predictor for training "red" model. I trained it on the one-minute chart of AUDJPY, the sample of 1500-2000 bars. The obtained models operate on all timeframes and on the whole history despite the considerable amplitude difference in quotes.

There were many variants of market forecasting put forward here but many of them could not decide on the predicates, on the target function, on what to train the neural network. In this post I wanted to show how I was solving this complicated problem.


 

Profit charts on "pattern pattern vs. neural network".

Both models were trained to trade eurusd on the plus side in October 2016; constant lot, no stops or takeaways; always in the trade long or short; trade on H1 at opening prices. Trading on the chart - last 5 years, including one month of training data.

Learning models without crossvaluation, they just squeezed the maximum profit they could out of the price.

There is a place on the charts where the server did not give normal ticks, there's some kind of drain, then ignore that place.


Here is the neuron. You can clearly see the time interval at which it was trained, this is the only place with a stable profit.


And here is the model with pattern recognition. The result is negative, but still better than neuronics. And there are plenty of times when it was profitable for weeks. But then it was lost.
It's cool, but it is not clear what to do with it.