Machine learning in trading: theory, models, practice and algo-trading - page 2959

[СанСаныч Фоменко]

mytarmailS #:

you can...

We need to make the example as simple and reproducible as possible, if there is interest...

you need a little time

It's very interesting to me. My search for a teacher is a long and painful process.

[Elvin Nasirov]

mytarmailS #:

Your problem is an optimisation problem, searching for unknown parameters.

Here is the ONLY article you need to study https://www.mql5.com/ru/articles/2225

If you want to teach AMO to maximise profit and minimise drawdown:

you need

1) create a fitness function, a function that will count profits, losses from trading signals.

2) any MO algorithm that will generate signals for trading, for fitness function (p.1)

3) any optimisation algorithm (genetic, particle swarm, churn) - which will generate signals as targets for AMO (p.2).

algorithm like this

1) AO creates a target for AMO

2) AMO is trained on this target

3) AMO creates a forecast of trade signals

4) trade signals are evaluated by FF and produce a result

5) FF result is evaluated by AO and maximised/minimised further and so on in a circle until an acceptable result is obtained.

==========

AO - optimisation algorithm

AMO - machine learning algorithm

FF - fitness function

=========

ps. if you want to work with neuronka and not with any AMO, you can change weights by means of AO, without learning targeting.

Thank you!

This formulation of the answer immediately led to the question of concretisation of manual trading, where the clarity and concreteness of signals, their variety and excessive flexibility are missing. That is, there is food for thought.

[mytarmailS]

СанСаныч Фоменко #:

It's very interesting to me. My search for a teacher is a long and painful process.

Here is the code of teaching the rendom forest by means of AO,

the fitness function (OUR OBJECTIVE) is to find a beautiful/stable profit growth, namely the maximum correlation between the balance sheet dynamics and the straight growth line

#  install.packages(c("randomForest","GA"))
library(randomForest)
library(GA)


#  создаю фейковые данные для простоты воспроизводимости
price <- cumsum(rnorm(100))
X <- embed(price,dimension = 10)[,10:1]
X <- t(apply(X,1,scale))
price <- tail(price,nrow(X))

#  настройка графики
par(mar=c(2,2,2,2))
layout(1:2)

#  запускаю генетический алгоритм
#  который "придумывает" такой таргет полсе обучения на котором
#  АМО даст самый красивый прирост капитала
best_res <- 0
GA <- ga(type = "real-valued",
         fitness = fitness, 
         lower = rep(-5,nrow(X)),
         upper = rep( 5,nrow(X)),
         popSize = 100,
         maxiter = 50,
         run = 20)

Here is the code of the functions to calculate profit and FF.

#  простая функция которая считает прибыль ( возможно не верно :)  ) 
count_equity <- function(trade_signal, price)  cumsum(c(diff(price),0)*trade_signal)

#  ФФ которая берет выход из АО и обучает АМО
fitness <- function(ga_out){
  
  target <- ga_out
  set.seed(123)
  rf <- randomForest(target~., X, ntree=100)
  pr <- predict(rf,X)
  
  trade_signal <- sign(pr)
  balance <- count_equity(trade_signal = trade_signal,price = price)
  res <- cor(balance, 1:length(balance))
   
  #  просто графика, необезательный елемент
  if(res>best_res){
    best_res <<- res
    plot(price,t="l",main="price")
    color <- ifelse(trade_signal==1,3,2)
    points(price,col=color,lwd=5,pch=20)
    plot(balance,t="l",col=4,main="balance")
  }
  return(res)
}

Here is the result, AO has found such a target for AMO that if we trade its signals we will get a beautiful profit growth.

[mytarmailS]

Elvin Nasirov #:

Thank you!

This answer immediately led to the question of concretisation of manual trading, where the clarity and concreteness of signals, their variety and excessive flexibility are missing. That is, there is food for thought.

I still can't concretise my manual trading....

[Maxim Dmitrievsky]

mytarmailS #:

Here's the code for teaching the Rendom Forrest with AO tools,

fitness function (OUR OBJECTIVE) - to find a beautiful/stable profit growth, namely the maximum correlation between balance sheet dynamics and a straight rising line

Here is the code of the profit and FF calculation functions

Here is the result, AO has found such a target for AMO that if we trade its signals we will get a beautiful profit growth

So? ) How does it affect the result on new data? It's done without ff by simple markup.

[Maxim Dmitrievsky]

Elvin Nasirov #:

I would not say that I am lazy to do any other markups, I try different variants and since I am not a senor in machine learning, when some idea comes to my mind, I try to find at least some variants of examples with attempts to achieve the result.

When I tried to make a parametric version of the solution with its own values of indicators, but it turned out that there are so many variants of the set of values of indicators that with current computing power selection of parameters will be carried out almost 10 years).

I was surprised when I read the phrase "take any profitable TS from the market". I didn't even consider this option, as I thought they are not there.

This is all some kind of complicated perception. You need to find inefficiency. If you can't find it yourself, look for it in others. There are signals with history on the market.

[mytarmailS]

Maxim Dmitrievsky #:
So what? ) It's a simple markup without FF.

If it were that simple, there wouldn't be AO and FF...

It's when we know exactly what we need and understand how to algorithmise it, we can do it with markup.

And there are cases when we just want to say, " I don't know what it should look like, but make it look good".

All we can describe is good/bad, that's what is put into the FF.

Here's an example of a task:

The task is to train such an AMO that it does not make mistakes in its predictions on new data on one of the classes, one of the classes is forbidden to make mistakes at all...

how to make such a markup?

[Maxim Dmitrievsky]

mytarmailS #:

If it were that simple, there wouldn't be AO and FF.

When you know exactly what you need and have an understanding of how to algorithmise it, you can use markup.

And there are cases when we just want to say - I don't know how it should look like, but make it look good.

All we can describe is good/bad, and that's what is built into the FF.

Here is an example of a task:

The task is to train AMO so that it does not make a mistake in its predictions on new data on one of the classes, one of the classes is forbidden to make a mistake at all....

how to do this with markup?

After markup, you throw out the losing trades into the 3rd class, and the same flat curve is obtained.

And how can the selection of optimisation criteria do something for new data? It is a means of improving a ready-made TS. Searching custom losses and seeing what is there on new data? That's not a good idea either.

[mytarmailS]

Maxim Dmitrievsky #:
After marking up, you throw out the losing trades into the 3rd class, and the same smooth curve is obtained.

That's how much code you have to write with these classes, throwing them out, retraining...

In FF I just write 2 lines, penalise for a class error, and reward for a correct answer, and that's it, and then it fantasises how to do it by itself and doesn't need a mountain of code....

OK, that was easy, another example of a task

AMO input is eurodollar, and I want the output to be a series that is

1) cointegrated with the pound

2) if we trade arbitrage AMO row / pound, so that there is a profit.

how can this be done through markup?

Maxim Dmitrievsky #:

optimisation can do something for new data?

the same way as with regular MO, we train and look with one eye on the track, with the other on the test.

[Maxim Dmitrievsky]

mytarmailS #:

How much code do you have to write with these cycles, throwing, retraining.

And in FF I just write 2 lines, penalise for a class error, and reward for a correct answer, and that's all, and then it imagines how to do it itself and doesn't need a mountain of code....

Okay, that was easy, another example of a task

AMO input is eurodollar, on the output I want to get such a series that

1) cointegrated with the pound

2) if we trade arbitrage AMO row / pound to make profit

how can this be done through markup?

the same way as with a regular MO, we learn and look with one eye on the track and the other on the test.

There's no code there, I did it in articles. And correction of losing trades was done, reversed. I just don't understand how it should save on new data.

There is no way to trade arbitrage there, if they are not initially cointegrated. You can only find pieces by time, where there are such moments. And what's the point of distorting symbol quotes, you can't open trades on them