Machine learning in trading: theory, models, practice and algo-trading - page 1433

 
Aleksey Vyazmikin:

You don't understand the main point, which is simply the implementation of a plan for price behavior. Who says the plan has to be too complicated, and who says it has to be the same all the time?

If you could comment on what the "plan of price behavior" is. Regarding statistics, well, we only have it in the armory, if something is always different and there is no way to detect a pattern (sameness) then we deal with randomness, like a simple CRS like x(t+1) = (a*x(t)+b)%c with more{a,b,c} is "random" because statistically it is not predictable, the same with BP, no statistics = no trading advantage = dumping

 
govich:

If you can comment on what the "price behavior plan" is. As for statistics, well it's the only one we have in the armory, if something is always different and there is no way to detect a pattern (sameness) then we deal with randomness, for example a simple CRS like x(t+1) = (a*x(t)+b)%c with large{a,b,c} is "random" because statistically it is not predictable, as well as with BP, no statistics = no trading advantage = a loss

Let me quote a letter I recently wrote to a friend (the man is not a trader and not a mathematician) - it contains the essence of my vision of the market today and a description of trying to work with it.

"

...

Why I compare it with alchemy, it's simple - expectations of the result always exceed the reality, and these expectations have no scientific basis, only hypothesis. It's a constant search and test of ideas. There have been many attempts to understand and describe the market for the last century - the scientific world tends to the hypothesis that the price for assets is random - the so-called random walk, but they justify it by the fact that they cannot reveal obvious laws affecting the formation of prices - some admit their multiplicity and therefore it is impossible to take them all into account, others believe that every price is the balance of supply and demand. In the community of automated traders there are a lot of people with different education backgrounds and this, of course, affects their judgments - mathematicians, physicists, chemists, programmers, economists, there are also sportsmen, pilots, designers, in general there are many different people with different experience and knowledge backgrounds, but all this helps little to achieve the goal.

The theory that the price of an asset is determined only by supply and demand and the current price is fair and the history of price changes does not matter - I reject it immediately for two reasons - it is useless and you can not derive profit from it and there are always those who have remained in the hands of chips instead of assets - ie a fair price was always absent for them and their volume must have influenced the price.

Regarding the theory of random walk - it's like taking dice, writing on them, say, deltas of price change (in some range) with different sign(vector), at some time interval, putting then in a black bag and taking them out of this bag and drawing a graph based on their values. I am not a mathematician myself, so I draw images for myself in order to operate with knowledge. Oddly enough, indeed, it is often difficult to distinguish a real graph from a random walk if you look at a limited number of its elements - 100-200, but if you study the graph in detail, you will find the absence of any pronounced regularities.

In my opinion, this is not the case - the market is ruled by people, and people are moved by emotions, there are also people, who protect themselves from emotions with trading algorithms. Emotions are not allowed for executor traders - employees of banks and investment companies, so they try to follow certain algorithms, albeit in manual mode, but approved by the management. I would imagine a random wandering price movement from target to target, the target being a certain price. The process looks like a herd of sheep moving in the necessary direction - sometimes the flock of sheep forms a harmonious line (fast movement - trend), sometimes sheep run apart (range fluctuation - flat) and the shepherd has to take efforts to direct the sheep to the necessary direction. If the price were moving in one direction, everyone would earn easy, but there's the trick - for what reason the vector of price movement changes, why dice with opposite vector suddenly start to come out of the bag. Of course, exactly at the reversal points the interested party must take the maximum efforts to change the trend - this is not always possible (there are even methods of psychological pressure, when in the Depth of Market an order is placed for an operation that is hundreds or even thousands of times greater than the average - as if the market participant is given a hint that the further movement will be bought and it is better to close the positions, for such actions the punishment in the States, by the way).

Market participants believe in regularities, and it is this belief, expressed in action, that allows such regularities to exist.

Therefore, I reduce the task to identifying patterns that are actually present due to the belief in them by market participants and the actual application of algorithms by large trading participants. For many years of stock trading there were invented many indicators, which are popular among traders, even the Central Bank of Russia uses them in its prognoses, and promotes the technical analysis (estimation of price movement in the past in order to predict the probable development of events). I prefer to look for short-term manifestations of the market character, within a day or even 1-3 hours at Moex.

Of course, to find a pattern with a high probability is a great success, but often the life of this pattern ends in the window of searching and identifying it, of course, because it is just an approximation from patterns (ratio of the price and its measurement (indicators)).

The task is to create a methodology for identifying patterns (pieces of someone else's algorithm) and evaluating these patterns. In fact we need a lot of different and varied such patterns, the probability of the outcome of which will be profitable in its sum, ie false (random) patterns should be less than those whose patterns continue to manifest themselves beyond observation. And even detected patterns, they are more like omens describing phenomena that coincide with real patterns - some kind of satellites.

All this is exacerbated by the fact that the market is changing - evolving in a natural way and because of this it is difficult to assess the quality of detection of stable patterns that lead to the expected outcome.

And while the processes of identification and evaluation can be automated, the process of creating a predictor (a characteristic that must be taken into account when searching for a pattern - for example, time, day of the week, price level for an option (strike), price position relative to the moving average of the price) is very difficult to automate - going through all the options requires enormous resources, and here we have to fantasize - by analyzing possible relationships with our eyes, which will later be tested by machine learning.

Open methods of machine learning are not very effective in such conditions - where the noise, from erroneous patterns, overlaps grains of valuable information - the trouble is that all these methods use the so-called principle of greed, so I after training the model, and usually it is a variation on the decision tree, try to separately consider each leaf of the tree and give him an estimate.

Now the direct calculations are handled by 6 computers with 6-8 cores each.

And what ends up being the lack of any dogma to lean on, inflated expectations of success, random glitches in the output in the form of excellent algorithm tests, and plum on data outside the training sample - that's the alchemy.

...

"

 
Igor Makanu:

I bought this, but I'm giving it to you, you need it more.


Thank you, I will need it...

 
Igor Makanu:

No problem! Come in! It would be profitable!

You don't understand - it will come in handy in terms of an asset for implementation when there's nothing to eat...

 
Igor Makanu:

Of course I don't understand, from the experience of the 90s such assets were a bag of potatoes, a bag of flour and a bag of sugar, but apparently times have changed - so have values.

Um, I don't remember these assets being stored for sale, but rather for consumption.

 
Igor Makanu:

Hmm, you have a kind of business "in Russian," as in the joke:

Stole a case of vodka, sold the vodka, drank the money

I thought you indicated the purpose of storage of assets - to implement them when there is nothing to eat, without hesitation I gave you a more reliable way to store these assets, so to speak, without costs or overheads

I don't understand you any more, I think that's the end of the discussion process, apparently the joke didn't work.

 

For three years Teacher Ma has been perfecting the art of MO)))


 
Aleksey Vyazmikin:

I'll quote from the letter...

Turn off both the forum and the correspondence. Pay attention to the meaning of the word "pattern"...

 
Aleksey Vyazmikin:

letter to a friend...

All right, I agree with everything, in general, but from such words to business is not so easy to move. Yes, the market is moved by people and they move it not just for fun, but they risk their money, i.e. they risk their comfort, safety, health and their family members' health, each participant certainly risks his own way, but still individual trading decisions cannot be called casual, if you know the layout in each case. But you should understand that most of the liquidity, especially on the foreign exchange market is NOT SPECULATIVE, it is a stupid currency exchange, hedging, etc.. And the main thing is that the performing traders who need to buy/sell "a lot" (% of the daily turnover), they now play all sorts of manipulative games, different every time to confuse the "bait fish", everything goes into the arsenal here, deceptive movements, imbalances in the market and so on.

And we only have to rely on statistics...

 
Igor Makanu:

conspiracy?

Why did Maxim drain the account? - How long did the account live?

There is a demo, but still a shame.

Reason: