Cluster methods of market forecasting. - page 6

 
Aleksey Ivanov:

Candlestick patterns, which logically belong to such clusters, of course, no one has clustered in this way (statistically correct). They were identified as a result of many years of observing the market by traders. But, nevertheless, this is the first and most widely known type of clusters by traders, which is what I wanted to discuss for now - at the first stage (without involving volumes).

And, in fact, I suggested to those forum participants who do candlestick analysis,

To find typical (most common without volumes) patterns on the following charts.

Fig. 1.

Fig. 2.

Fig. 3.

Fig. 4.

Fig. 5.

Alexei, I look at Fig.1: I see the 9th and 10th candle - this reversal pattern is called rails. On Fig.2: the 2nd and 3rd candle are internal, after them the movement can be towards the 1st candle. In Fig.3: The 3rd candle is a pinbar (with a long nose, like Pinocchio), after it usually moves towards the short shadow.

Fig.4: I see the 21st candlestick is a hammer (but the hammer handle is short), after the hammer there is usually a move opposite to the handle. In Fig. 5, after the 71st candlestick, I see four inside candlesticks; after that there should be some movement towards the 71st candlestick, but for some reason the market decided to go up.

And now, Alexei, let us add volumes to these patterns ? Is that your intention ?

 
Veniamin Skrepkov:


The discussion of patterns without volumes is Japanese patterns or Price action, the highlighted area ( hanged - shooting star - hammer ) where the market has implemented a movement pattern = up and down and up through the bottom.

That's good, thanks. But on those that I have posted in the zip on the first post (12 pcs.), there (at least on one) is it possible to see something?
 
Aleksey Ivanov:

So, I would like to get this thread back on track. Here I would like, with your help, gentlemen,to identify the strengths and weaknesses of existing cluster approaches to market forecasting and outline new, perhaps more promising, approaches.

I will explain on my fingers (for those who do not know) what the cluster approach is in relation to the market.

But first, about market dynamics.

Price can experience large and indeed unpredictable (for most people) spikes (1) in strong events (important news about: economic decrees, cataclysms, major business and political events, etc.). In this case there is a relaxation of the fluctuations caused by this with time proportional to ~1/N. The market, however, "lives its own life" (where self-organization processes take place), experiencing (2) its own (not caused by external influence) and sometimes even the smallest jumps, which are characterized by another law of relaxation. characterised by another law of relaxationSqrt(1/N), which, we note, happens much more often than relaxation ~1/N, so, no matter how unusual it sounds to us,the market functions mainly according to its own laws .

First type of jump does not happen immediately (because many people are involved in its forming), which imposes some specific characteristics on the interval of history of the quote, which is sandwiched between the moment when a strong event happens and the surge caused by it. Moreover, the part of history prior to the second type of jump should contain some specific characteristics (delayed swing of the market and its fall from the next state of unstable equilibrium).

Now clustering.

So, the initial hypothesis is that there is a small part of quotes history preceding the price jump (plus volume history that goes there) where the information about the next jump is encoded.

Further, there is a purely technical part. A space of certain parameters or states is introduced, such as: (1) a trivial geometric image in the form of a candlestick pattern, or (2) the space of different frequency modes obtained by Fourier decomposition of this plot (time series), or (3) a spectrum expansion by orthogonal velvet functions (which is much better, since the plot is short) or (4) a spectrum expansion by some other orthogonal functions, etc.

Then a huge - statistically significant set of such (preceding jumps) plots are taken and analyzed for their occupancy of this space of states. And if they are significantly concentrated in some parts of this space (and the other parts of history - not preceding the jumps - do not get there), then this will be the cluster (or set of clusters of types 1 and 2), which allows to make a prediction.


What you describe in a naive way is called CLASSIFICATION, which comes in two kinds:

  • without a teacher, when a source set is divided into some groups
  • with a teacher, when the original set is partitioned under the teacher's values.

Clustering in its classical sense refers to learning without a teacher. For rank, a very interesting algorithm is the support vector method - SVM.

You, on the other hand, started with learning without a teacher and added searching for sites that are of interest to you, which is already learning with a teacher, when you don't just split into subsets, but under some condition.


I have to disappoint you that all this is so well-developed, there are mountains of publications and ready-made software that it is basically unaffordable for one person. Moreover, the corresponding materials are available on this site and are widely discussed.


It seems to me that you have already tried to reinvent the wheel, and this is another one.

Make a choice for yourself, on the crudest of two fronts:


  • You study the statistical characteristics of quotes in order to get rid of non-stationarity and try to get rid of that non-stationarity by modelling different nuances
  • You form a trading system target for yourself, for example, you trade news, as you wrote above. You mark the initial chart with these very news, and then look for raw data that can predict this news. And a ready-made algorithm, of which there are tons, will do a "clustering with a teacher" and match the subtleties in the raw data to this very news of yours.

The main thing is to make up your mind, and remembering that hundreds of thousands of very educated people have tried to solve these questions before you, study the available literature, the available software, and there....


And just a wish: stop reinventing the wheel.

 
Victor Ziborov:

Alexei, I look at Fig.1: I see the 9th and 10th candle - this reversal pattern is called rails. Fig.2: the 2nd and 3rd candle are internal, after them the movement can be towards the 1st candle. In Fig.3: The 3rd candle is a pinbar (with a long nose, like Pinocchio), after it usually moves towards the short shadow.

Fig.4: I see the 21st candlestick is a hammer (but the hammer handle is short), after the hammer there is usually a move opposite to the handle. In Fig. 5, after the 71st candlestick, I see four inside candlesticks; after that there should be some movement towards the 71st candlestick, but here, for some reason, the market decided to go up.

And now, Alexei, let us add volumes to these patterns ? Is that your intention ?

There are no volumes. This is the next step. So can you take a zip and roughly trace these characteristic places (with pattern names)?
 
СанСаныч Фоменко:

What you describe in a naive way is called CLASSIFICATION, which comes in two kinds:

  • Without a teacher, when the original set is divided into some groups
  • with a teacher, when the original set is partitioned under the teacher's values.

Clustering in its classical sense refers to learning without a teacher. For rank, a very interesting algorithm is the support vector method - SVM.

You, on the other hand, started with learning without a teacher and added searching for sites that are of interest to you, which is already learning with a teacher, when you don't just split into subsets, but under some condition.


I have to disappoint you that all this is so well-developed, there are mountains of publications and ready-made software that it is basically unaffordable for one person. Moreover, the corresponding materials are available on this site and are widely discussed.


It seems to me that you have already tried to reinvent the wheel, and this is another one.

Make a choice for yourself, on the crudest of two fronts:


  • You study the statistical characteristics of quotes in order to get rid of non-stationarity and try to get rid of that non-stationarity by modelling different nuances
  • You form a trading system target for yourself, for example, you trade news, as you wrote above. You mark the initial chart with these very news, and then look for raw data that can predict this news. And a ready-made algorithm, of which there are tons, will do a "clustering with a teacher" and match the subtleties in the raw data to this very news of yours.

The main thing is to make up your mind, and remembering that hundreds of thousands of very educated people have tried to solve these questions before you, study the available literature, the available software, and there....


And just a wish: stop reinventing the wheel.

I'll think about it, you can't digest it all at once. It's just that I always follow my own logic, and maybe even bicycles are invented. That's what the discussion is for.

 
Victor Ziborov:

Alexei, I look at Fig.1: I see the 9th and 10th candle - this reversal pattern is called rails. Fig.2: the 2nd and 3rd candle are internal, after them the movement can be towards the 1st candle. In Fig.3: The 3rd candle is a pinbar (with a long nose, like Pinocchio), after it usually moves towards the short shadow.

At Fig.4: I see the 21st candle is a hammer (though the hammer handle is short), after the hammer there is usually a move opposite to the handle. In Fig.5, after the 71st candle, I see four inside candles; after that there should be a move towards the 71st candle, but here, for some reason, the market decided to go up.


Thanks, that's enough information. I don't want to bore you anymore.

 
Aleksey Ivanov:

I'll think about it, you can't digest it all at once. It's just that I always follow my own logic, maybe bicycles at the seed turn out. That's what discussion is for.

I wonder how you would react if economists, or even worse, traders started getting involved in controlling nuclear reactors.

The consequences here are only for pockets, but the gist is the same.

Put R. It's the standard in statistics today, and there's everything for mathematical models in trading. There is code in huge quantities, the code MUST be accompanied by links to the algorithm, there is academic literature, specials, periodicals - in general everything is smart with a great search.

1. If you are interested in statistics, you can find ARMA-ARIMA-AFRIMA-ARCH-GARCH models, of which there are over a hundred.

2. If you are interested in classification, then put rattle, which requires no preparation at all, but gives you a systematic view of classification, i.e. preparation of input data (datamining), models (there are 6 of them) and evaluation of classification results. To make it easier, I can recommend my article, which is an extract from the documentation, but is tied to forex.


There is no grail in either direction. In both directions there are solved problems which have opened up more unsolved problems.

 
СанСаныч Фоменко:

I wonder how you would react if economists, or even worse, traders started getting involved in controlling nuclear reactors.

The consequences here are only for pockets, but the gist is the same.

Put R. It's the standard in statistics today, and there's everything for mathematical models in trading. There is code in huge quantities, the code MUST be accompanied by links to the algorithm, there is academic literature, specials, periodicals - in general everything is smart with a great search.

1. If you are interested in statistics, you can find ARMA-ARIMA-AFRIMA-ARCH-GARCH models, of which there are over a hundred.

2. If you are interested in classification, then put rattle, which requires no preparation at all, but gives you a systematic view of classification, i.e. preparation of input data (datamining), models (there are 6 of them) and evaluation of classification results. To make it easier, I can recommend my article, which is an extract from the documentation, but is tied to forex.


There is no grail in either direction. There are solved problems in both directions which have opened up even more unsolved problems.

Long time ago I was invited to R (back in 2007 the admin on investor_ru advised me, when I was making up forecast "bicycles", like modifications of ARIMA). Will you instruct me on R, if at last I decide?

By the way, in one exeq file mql4-5 can be used for processing of data produced in R and, if so, how difficult is it?

Oh, and by the way, about the reactors - they are now ruled by illiterate managers along with their mistresses. We live on a volcano, by the way. Our economy will not survive a second Chernobyl.
 
СанСаныч Фоменко:



There is no grail in either direction.

So why are you bringing this bullshit into all the normal topics?
 
Aleksey Ivanov:

They have been calling me to R for a long time (back in 2007 the admin on investor_ru advised me, when I was making up forecast "bicycles" there, like ARIMA modifications). Will you instruct me on R, if at last I decide?

By the way, the processing of data produced in R may be put into one exeC file mql4-5 and, if so, how difficult it would be.


If possible. I have very limited knowledge by my needs. The main problem is not with R, which for our needs is mastered in an hour, but with the content of the packages that implement the mathematical methods.


There is a DLL for communicating the terminal with R. It works without any problems. Primitive, but works fast - everything is through memory. There is a debugger for communication between terminal and R.

Takes Excel files. You can test your clustering ideas the same day you install R at your place.

Functional division in operation is obvious:

  • MT - trading functions + money management
  • R - models

MT is not needed in development at all: R is a developer's paradise - heaps of models + wonderful graphics