Machine learning in trading: theory, models, practice and algo-trading - page 3100

 

You all have a very narrow definition of a model...


a model is a replacement of a real phenomenon with something simpler and more understandable, which preserves the necessary properties and discards the unnecessary ones.


If you're a navigator, you don't have to go into space and look at the earth to navigate, you just have a globe....

A globe is a model of the earth, it has the features you need to orientate yourself, and gets rid of the unnecessary ones.

A globe is not the earth, there are no living things on a globe, no water, no winds, no temperatures, no pressure... but you don't need that, so there's a model.


A model is a useful simplification, when it's an approximation, when it's a linearisation, when it's a neuron, when it's a dimensionality reduction....


What is price? It's ticks, but if we look at a clockwork, what is price? It's candles, what are candles? It's a price model because it's a simplified representation of price.

What a TS is is an attempt to describe a market process by a set of rules or functions. A TS is a model of the market process.


but a model is a useful simplification.

and a market model doesn't necessarily have to be some formula that describes market relationships.

 
Maxim Dmitrievsky #:
Well, thick tails are due to uneven volatility (wave clustering), depending on the trading session. They don't seem to carry any other information. I look at markets as more or less efficient. Accordingly, the task is to search for inefficiencies. It is hard to say what model it can be put into. Because they appear here and there.
.
Either some complicated econometric model, but these are usually investment strategies. And for scalping, either arbitrage or over-optimised TSs.
A trained MOSHka itself already contains both useful information and noise. You can train it once on quotes and then study this model. This is also an option. Without fanaticism and hooting like IO has created a grail. Just explore and pull out the unexplored.

You may be right. Though to explain non-stationarity by uneven volatility is a bit of a simplification in my opinion. Clustering of volatility
is quite simple to model and then any heteroskedasticity would be easily eliminated by correlation with volatility. But in reality there are a lot of tricky arches and garches, which also do not work very well.
The market is probably efficient, and if it is not, you can find out only by finding your alpha, with the help of which you can constantly beat it. Where to look for this alpha, in what direction to dig, in my opinion, the pricing model can suggest and push. Otherwise, the curse of dimensionality can lead to the fact that it will take millions of hours of work of computers to find it.

 
mytarmailS dimensionality reduction....


What is price? It's ticks, but if we look at a clockwork, what is price? It's candles, what are candles? It's a price model because it's a simplified representation of price.

What a TS is is an attempt to describe a market process by a set of rules or functions. A TS is a model of the market process.


A model is a useful simplification.

and a market model does not necessarily have to be some formula that describes market interactions.

I would add to the above notion of a model one more extremely important property in our case: the mentioned globe does not exist, but there is a set of globes that are located in some channel with some probability.


As soon as we forget about this, we slip into determinism and start believing in the obtained figure, even though it is on paper, it is not there. I.e. when we say that classification error = 30%, we should NOT take it literally, but we should understand that- a classification error of 30% is within some limits, e.g. 2% with a probability of 95%. And then 30% is 30%. But does someone calculate the channel width?, Maybe not 2% but 22%? Or is the probability of the channel width 60%? What's a decent classification error of 30% worth?


This is not our complete happiness.

The above reasoning about probabilities and channel width is valid if the error distribution is normal. And who has checked the distribution of classification error for normality?

 
sibirqk #:

Otherwise, the curse of dimensionality may result in millions of hours of compute work to find it.

The curse of dimensionality has been around for a long time.

because of it.

1) the market doesn't repeat itself - because of it.

2) there are so many price options - because of it.

3) MOs that all work according to the scheme: "tomorrow will be like yesterday" do not work - because of it.


That's why we need a model, for simplicity, for repeatability.

 
mytarmailS dimensionality reduction....

What is price? It's ticks, but if we look at a clockwork, what is price? It's candles, what are candles? It's a price model because it's a simplified representation of price.

What a TS is is an attempt to describe a market process by a set of rules or functions. A TS is a model of the market process.

A model is a useful simplification.

and a market model doesn't necessarily have to be some formula that describes market relationships.

I completely agree with you - a model is a useful simplification. But I was referring to a pricing model for a financial asset.

Let's say there is such a model - let the price change at a given moment is determined by the excess of bids, to sell or buy. Let's take for simplicity, a large number of independent participants, let's say a million. And let them place orders with the same volume, simultaneously with some strobe, and let the price change be proportional to the aggregate position unbalanced at a given step. It is clear that in such a model the price of an asset will be a classical pure random walk. Indeed, if traders independently make trading decisions, then about half of them will buy and the other half will sell. Their difference is very likely to be at the root of the number of participants. At the next step the balance may change and the price will shift in the other direction.
In this sense it is declared that the SB is the most accepted and coarse model of pricing. But this model needs to be refined to explain some empirics and at the same time to understand where to look for inefficiencies.

 
sibirqk #:

I completely agree with you - a model is a useful simplification. But I was referring to the pricing model of a financial asset.

Let's say there is such a model - let the price change at the moment is determined by the excess of bids to sell or buy. Let's take for simplicity, a large number of independent participants, let's say a million. And let them place orders with the same volume, simultaneously with some strobe, and let the price change be proportional to the unbalanced aggregate position at a given step. It is clear that in such a model the price of the asset will be a classical pure random walk. Indeed, if traders independently make trading decisions, then about half of them will buy and the other half will sell. Their difference is very likely to be at the root of the number of participants. At the next step the balance may change and the price will shift in the other direction.
In this sense it is declared that the SB is the most accepted and coarse model of pricing. But this model needs to be refined to explain some empirics and at the same time to understand where to look for inefficiencies.

Alexei Nikolaev in his blogs on R implemented a model of the game Cafe, or the victory of the minority, similar in terms of the market, if the position of the player is in a society with fewer participants, he wins (in the cafe, according to the date, players who came on the day with the least number of visitors win, and with a large number of visitors lose), but this is too simple model, in the real world there are still a lot of types of players, ranging from the state and other large players and small players, which are a large number. The model is not even roughly created yet)

But the graphs there are even very similar to tick wandering.

 
sibirqk #:

I completely agree with you - a model is a useful simplification. But I was referring to the pricing model of a financial asset.

Let's say there is such a model - let the price change at the moment is determined by the excess of bids to sell or buy. Let's take for simplicity, a large number of independent participants, let's say a million. And let them place orders with the same volume, simultaneously with some strobe, and let the price change be proportional to the unbalanced aggregate position at a given step. It is clear that in such a model the price of an asset will be a classical pure random walk. Indeed, if traders independently make trading decisions, then about half of them will buy and the other half will sell. Their difference is very likely to be at the root of the number of participants. At the next step the balance may change and the price will shift in the other direction.
In this sense it is declared that the SB is the most accepted and coarse model of pricing. But this model needs to be refined to explain some empirics and at the same time to understand where to look for inefficiencies.

Personally, I don't see any use for the SB model at all.

It doesn't do anything, it doesn't highlight good properties, it doesn't suppress bad properties, it doesn't simplify...

Yeah, the graph looks like prices, so what?

 
sibirqk #:

You may be right. Though to explain non-stationarity by non-uniform volatility is a bit of a simplification in my opinion. Clustering of volatility
is quite simple to model and then any heteroskedasticity would be easily eliminated by correlation with volatility. But in reality there are a lot of tricky arches and garches, which also do not work very well.
The market is probably efficient, and if it is not, you can find out only by finding your alpha, with the help of which you can constantly beat it. Where to look for this alpha, in what direction to dig, in my opinion, the pricing model can suggest and push. Otherwise, the curse of dimensionality can lead to the fact that it will take millions of hours of work of computers to find it.

I call stationarity the usual econometric thing: constancy of mean and variance. In markets there is naturally no such thing, they are not a "monument". Heteroskedasticity is removed, the rest is close to SB.

In general, the type of distribution says little about predictability. It is such a mathematical game, distant from trading. Add some fluffiness to the quote that covers the spread. Or a steady return to the mean at certain times of the day. The spread won't change, and it will be possible to make money. Roughly this and stuff like this can be called inefficiency. To do this, write algorithms taking into account the fact that you can't predict everything, and you don't need to. I would not say that there is such a curse there, just that there are really efficient tools from which you can't get anything out.

 
Teach the AI how to be an info-giant.
 

If we measure the percentage of signals of one type in each month of the sample for a separate quantum segment and subtract the average percentage of profitable signals and build a balance on the data, we can see the following.

This is the selected quantum segment according to my method, and what we see is that from the 38th month to 127th month there was such a stable trend, and then the fluctuations started.

So it turns out that if the sample is divided according to the classical 60+20+20 method, we will learn and everything will be fine till about 100 months, then at 40 months - till 140 we will be in the plus and already at the independent sample for testing we will catch a downward movement with a rebound. At the same time, we can see that there were similar movements on the sample for training before the 38th month, but what model will take them into account and find an "explanation" for these fluctuations? An ordinary wooden model will start pulling out a piece of the whole sample, while it is necessary to pay attention to only a part of it.

That's what I'm thinking about, a way of building a model that would take into account the above described nuances - and make splits not over the whole piece, but as if to take into account separately the changes in each section after the same split.

Maybe I'm reinventing the wheel again and there is already a solution? I've already outlined the system on paper, but the code is still a long way off....