Machine learning in trading: theory, models, practice and algo-trading - page 3352

 
Maxim Dmitrievsky #:

Ordinary increments with arbitrary lag. No logarithms or zero bars. The question was about signs. The main problem there is the low signal/noise ratio. But they contain all the information.

Clearly, all of it. But if we use increments between neighbouring bars, then to understand that there was a trend for the last 100 bars, the model should add up all 100 bars and get the increment from 0 bar. And by feeding from 0 bar at once, - we make its work easier.
Perhaps neural networks can take into account all 100 increments and find a trend, but wooden models are unlikely. You said yourself (and I have the same result) that models work best on several features (up to 10), and a full trend of 100 will not be formed from 10. And besides increments there may be more useful signs.

That's why my arbitrary lag for increments is always from 0 bar, so that the model sees the trend. The increments between neighbouring bars - never used at all, because I consider them noise. For example, an increment of 0.00010 pt between 120 and 121 bars, i.e. 2 hours ago, what influence can it have on the current situation? - None.

 
Forester #:

It is clear that all of them. But if we use increments between neighbouring bars, then to understand that there was a trend for the last 100 bars, the model must add up all 100 bars and get the increment from 0 bar. And by feeding from 0 bar at once, - we make its work easier.
Perhaps neural networks can take into account all 100 increments and find a trend, but wooden models are unlikely. You said yourself (and I have the same result) that models work best on several features (up to 10), and a full trend of 100 will not be formed from 10. And besides increments there may be more useful signs.

That's why my arbitrary lag for increments is always from 0 bar, so that the model sees the trend. The increments between neighbouring bars - never used at all, because I consider them noise. For example, an increment of 0.00010 pt between 120 and 121 bars, i.e. 2 hours ago, what influence can it have on the current situation? - None.

A trend is a shift in the average increments. So it is also on the increment chart. The problem with multiple attributes is explained from the point of view of confounders, i.e. it becomes more difficult to separate the influence of each attribute from all the others. After all, each one introduces additional error into the final estimation. Causal inference becomes more complicated and uncertainty is added. Apparently, there is some limitation on the complexity of the foreground model, which depends on the signal to noise ratio.

The lag of increments is directly related to the duration of the predicted trade in bars. For example, if the duration is 1 bar, it is better to take increments with a small lag and a small window depth. As the duration of trades increases, the lag of increments also increases naturally. And the spread of course affects.
 

Iwrote an Expert Advisor that records the data of each tick and forms a csv file with the data, and next to it I ran a neural network that analyses this file, performs training and, based on the last line in the first file, outputs a forecast of where the price will go and by how many points. Written in python. Who has already done something similar and what data can be learnt to cover the full picture. Now the experience is based on 'Opening', 'Maximum', 'Minimum', 'Closing', 'Volume'.

I don't know how to insert a screen

 
Андрей wrote an Expert Advisor that records the data of each tick and forms a csv file with the data, and next to it I started a neural network that analyses this file, performs training and, based on the last line in the first file, outputs a forecast of where the price will go and by how many points. Written in python. Is there anyone who has already done something similar and what data can be learnt to cover the full picture. Now the experience is based on 'Opening', 'Maximum', 'Minimum', 'Closing', 'Volume'.

I don't know how to insert a screen

record and analyse ticks of ALL currency pairs and metals.

add time of day, day of week

classification of expected news

seconds before the news

classification of past news

seconds after the news

===

in general something to keep yourself and python busy :-)

 
We haven't got through all the MOs yet, so we can't give you a clue :)
 
Maxim Dmitrievsky #:
We haven't mastered all the IOs yet, so we can't tell you right off the bat :)

Why master all the MOs? There are several hundred models alone... And models are clearly less than half the problem.

Maybe master a couple of models, but learn how to consistently achieve approximately the same classification error on OOV and OOS of less than 20%? While having in base proof of no over learning, looking ahead and, most importantly, proof of the resulting classification error in the future?

 
СанСаныч Фоменко #:

Why master all the MoDs? There are several hundred models alone... And the models are clearly less than half the problem.

How about mastering a couple of models, but learning to consistently achieve approximately the same classification error on OOV and OOS of less than 20%? While having in base proof of no over learning, looking ahead and, most importantly, proof of the resulting classification error in the future?

Under all the IOs are common approaches and practices. For example, can you obtain class probabilities for binary classification? If yes, in what way?
 
Maxim Dmitrievsky #:
Under all MOEs are common approaches and practices. For example, can you obtain class probabilities for binary classification? If so, in what way?

I can't immediately think of packages that do NOT produce class probabilities. So I was under the impression that this was the standard.

 
СанСаныч Фоменко #:

I can't immediately think of packages that do NOT issue class probabilities. So it seemed to me that this was the standard.

Batch thinking is in. Those aren't those probabilities, they are so called because of the loess phs applied, for likelihood estimation.

How do you get real class probabilities?
 
Maxim Dmitrievsky #:
Batch thinking is in. These are not those probabilities, they are so called because of the loess fi used, to estimate the likelihood.

How to get real class probabilities?

There is no other way of thinking about it! We use off-the-shelf MO algorithms that are accompanied by a set of additional functions. Everything together is called a "package".

What are"real class probabilities"? For example, the function

predict(object, newdata, type = c("vector", "probs", "both","F"))
returns "probabilityclass estimates". No other probabilities than "estimates" can be contained in the algorithm.