Machine learning in trading: theory, models, practice and algo-trading - page 2385

 
Maxim Dmitrievsky:

I think, Alexei suggested computing power, he likes to calculate something long, maybe you can make a co-op :)

R without vectoring will still be slow. You could use some kind of fast DB.

I have basically vectorized slow parts, the logic of the algorithm should be optimized, I'm rethinking it, I've already thought out how to speed up by 5 times, I've rewritten the code...

There are still so few signs, so primitive, I should do something with it too, I don't like the target...

Oh shit... it's like a repair, ego can only start ...

 
mytarmailS:

Of course you can not see)), and even if I saw it would not affect me in any way))

Try it, but the result will at best be like a Random Forest, i.e. none...


The output of a Forest is the sum of triggered rules, the rules are not filtered out and rejected in any way, and rejected rules are about 100%)

Rules are tested neither for repeatability (there may be only one response), nor for adequacy (does it work); rules are simply stretched to the data (the model is fitted to the data)

The model approximates a training sample randomly, hoping that cross validation will help, but it fails due to objective reasons (there are too few important events in the market)


I tried a different approach, I don't fit the model to the data, but I form hypotheses and check them.

1) I form plausible(already filtered) hypotheses in the form of rules.

2) Hypotheses are tested on small data

3) Hypotheses that have been tested on small data are tested on large data

In fact, only one out of a million plausible rules remains

It is difficult for the untrained reader to understand the difference between the two approaches, but the difference between them is a chasm

Some kind of philosophy. Nothing concrete...
 
elibrarius:
Some kind of philosophy. Nothing concrete...

Well why, the path is announced earlier, and everything is strictly as intended. Generating rules and checking them is not the usual approach where rules are selected based on the logic/objectives of the problem. That is, the rules are usually generated for the solution from the beginning. This leads to the loss of rules that are not logical but produce results.

 
elibrarius:
Some kind of philosophy. Nothing concrete...
  • Many things are incomprehensible to us, not because our concepts are weak; but because these things are not within the circle of our concepts.
Valeriy Yastremskiy:

Why, the path has been announced earlier, and everything is strictly as planned. Generating rules and testing them is not the usual approach where rules are selected based on the logic/objectives of the task. That is, the rules are usually generated for the solution from the beginning. This leads to the loss of rules that are not logical but produce results.

+++

 
mytarmailS:
  • Many things are incomprehensible to us, not because our concepts are weak; but because these things are not within the scope of our concepts.

+++

A little more philosophy))

 
Typical for such approaches are results that can be conventionally called "anti-homster" - which means that the hamster is seen but not seen. This is quite typical for rows similar to SB implementations.
 
Aleksey Nikolayev:
It is typical for such approaches to have results that can be conventionally called "anti hamster" - which means that the hamster is visible, but it is not. This is quite typical for rows similar to SB implementations.

I disagree. The problem with the large number of rules and selection logics is certainly there. The targets in both approaches are the same.

And the fact that the road is completely wrong, well, we check the rules and correct the path))))

 

I have some other MO abstraction that should accurately find real patterns (if there are any), but it's not certain

I couldn't find any Masonic or Pentagon references on google, so I'm keeping it under wraps for now

I came up with the abstraction when I was snapping sunflower seeds.
 
Maxim Dmitrievsky:

I have some other MO abstraction that should accurately find real patterns (if there are any), but it's not certain

I couldn't find any Masonic or Pentagonian references on google, so I'm keeping it under wraps for now

I came up with this abstraction when I was snapping sunflower seeds

I seem to have forgotten: the real patterns of the SB (if they exist))))

 
Maxim Dmitrievsky:

I think Alexei offered computing power, he likes to count something long, you can do a co-op :)

It will still be slow on R without vectorization. You can use some fast database

Moreover, this approach I'm doing for many years.

However, now it is somewhat postponed, because it is really very slow, which makes it impossible to quickly change the target (look for a good one), and individual rules also stop working, no matter how you check them on the history.

Maxim, can you help, make a script in python which will cycle to create a model (or 100 models) from CSV file, then analyze it(them) on predictors validity with standard CatBoost means, then prohibit/allow using of predictors (CatBoost is able to do it) on certain conditions and create again new model. This method will allow to single out important predictors, which will lead to model improvement. I use similar method, but it's really inconvenient for me, because after adding/removing predictors I have to manually restart the loop.