Machine learning in trading: theory, models, practice and algo-trading - page 3620

New comment
 
mytarmailS #:
There's a chat room, chat with others like you.
It's gonna be a full night tonight.)
 
Maxim Dmitrievsky #:
Tonight will be a full night :)

According to the results: no improvement :)

I tried convolutional neuronics too, the result is expectedly worse than scaffolding and bousting

Busting:

106:    learn: 0.5703702        test: 0.3621375 best: 0.3731653 (60)    total: 3.35 s    remaining: 27.9 s
107:    learn: 0.5724813        test: 0.3600689 best: 0.3731653 (60)    total: 3.38 s    remaining: 27.9 s
108:    learn: 0.5732323        test: 0.3605623 best: 0.3731653 (60)    total: 3.4 s     remaining: 27.8 s
109:    learn: 0.5730131        test: 0.3624452 best: 0.3731653 (60)    total: 3.44 s    remaining: 27.8 s
110:    learn: 0.5707056        test: 0.3637406 best: 0.3731653 (60)    total: 3.47 s    remaining: 27.8 s
Stopped by overfitting detector  (50 iterations wait)

bestTest = 0.3731652564
bestIteration = 60

Shrink model to first 61 iterations.
test score:
Accuracy: 0.3730531636209162

Neuronka:

Epoch 45/50
132/132 ━━━━━━━━━━━━━━━━━━━━ 0 s 3 ms/step - accuracy: 0.5362 - loss: 1.2519 - val_accuracy: 0.5055 - val_loss: 1.3550
Epoch 46/50
132/132 ━━━━━━━━━━━━━━━━━━━━ 1 s 3 ms/step - accuracy: 0.5366 - loss: 1.2640 - val_accuracy: 0.5050 - val_loss: 1.3632
Epoch 47/50
132/132 ━━━━━━━━━━━━━━━━━━━━ 0 s 3 ms/step - accuracy: 0.5415 - loss: 1.2693 - val_accuracy: 0.5089 - val_loss: 1.3603
Epoch 48/50
132/132 ━━━━━━━━━━━━━━━━━━━━ 1 s 3 ms/step - accuracy: 0.5477 - loss: 1.2415 - val_accuracy: 0.5017 - val_loss: 1.3649
Epoch 49/50
132/132 ━━━━━━━━━━━━━━━━━━━━ 1 s 3 ms/step - accuracy: 0.5354 - loss: 1.2828 - val_accuracy: 0.5061 - val_loss: 1.3784
Epoch 50/50
132/132 ━━━━━━━━━━━━━━━━━━━━ 1 s 3 ms/step - accuracy: 0.5331 - loss: 1.2766 - val_accuracy: 0.5100 - val_loss: 1.3636
57/57 ━━━━━━━━━━━━━━━━━━━━ 0 s 2 ms/step
Balanced Accuracy: 0.30404839813244877
 

Now it is obvious that the task is for causal inference. The result does not improve much through feature selection and model fitting, and overtraining occurs.

There are characters in discord who trained neurons on gpu for 2 days (I don't know what takes so long to train there) and achieved a speed 43. I think this is still a bad result, scor should be > 50.

Next time I will try to apply causal inference techniques I am familiar with.

 
If you want to show results, generate signs and do fiche selection... All the rest is useless wanking.
 
mytarmailS #:
If you want to show results, generate signs and do fiche selections... Everything else is just pointless wanking.
I think that's the first thing that 6k participants thought of :) the results are in the face.

Is your contest over? Say it's fun, the process.
 
Maxim Dmitrievsky #:
I think that's the first thing that 6k participants thought of :) the results are in the face.

No, the first thing that comes to everyone's mind is the same thing that came to yours - "I'm the smartest, I'll write the code myself and find the best solution".

But to create a code that will write another code and check it yourself is another levelling.

Maxim Dmitrievsky #:
Is your contest over? Say it's cool, the process itself

Yes, it's over, thank God...

At first it was very cool, but in the last few days I pklenal it and just wanted it all to end soon....


The coolest and most mature ideas come from the +- rich experience of solving this problem.

Those generated a dozen ideas, implemented, received an answer and only then begin to be born interesting ideas, but the motivation to write is already about zero (stupidly bored to write), and the time to implement a super idea remain days or even hours, the stress is unreal ...

 
mytarmailS #:

No, the first thing that comes to everyone's mind is the same thing that came to yours - "I'm the smartest, I'll write the code myself, I'll find the best solution."

But to create code that will write other code and check it yourself is another level.

Yeah, it's over, thank God.

In the beginning it was very fun, but in the last few days I've been pecking away at it and just wishing it would all be over soon....


The coolest and most mature ideas come from having +- a lot of experience with the task.

Those generated a dozen ideas, implemented, received an answer and only then begin to be born interesting ideas, but the motivation to write is already about zero (stupidly bored to write), and the time to implement a super idea remain only days or even hours, the stress is unreal ...

I'll check everything if I have time :)

 
I was sitting like this yesterday and a brilliant idea came to me: why do we need features at all, there are graphs themselves. And surely there are algorithms to traverse the whole graph and collect descriptive statistics from it. Find the shortest path and so on.
It's a bullshit question. I find a library, start doing it... and they are acyclic and incompletely connected. Traversing all edges doesn't work on such graphs.
Prado's a beauty.)

Then I'm like, okay. I'll take Markov chains and define graphs through hidden states. There will be signs. It's a bummer again, everything is super on the trayne, but on the test it's bad :)

I'm getting angry. I take regression metrics and generate a lot of them. Not bad! But I can't go beyond 0.43 error, it doesn't go up.

That is, in aggregate I have generated a lot of features, but it doesn't solve the problem.
 
Maxim Dmitrievsky #:
So in aggregate I have generated a lot of signs, but it doesn't solve the problem.

1) try different MOs (for example, in my contest wooden MOs did not work at all, only functional MOs worked: SVM, neurons, ruler).

2) don't feed all features in a heap, but look for the best subset of features, in practice up to 3-7 pieces can be used.


you can get 0.8 on 1000 features and 0.1 on 5 features.

 

Shall we discuss the contest?

Forum on trading, automated trading systems and testing trading strategies

Let's discuss the need for a contest to find patterns. Will you participate in the contest?

Aleksey Vyazmikin, 2024.09.18:08

Such an idea for the contest:
1. The contest is held to identify patterns that the organiser places on the chart in the process of generating quotes. I.e. the source material will be a file with minute bars for, let's say, five years. It is necessary to find one or more patterns. Maybe the probability will not be 100%, to make it more interesting, we can discuss it. Perhaps some other criteria are given.
2. Whoever wants to participate.
3. The result is public, i.e. the pattern should be either described in words, or better to provide the code. Discussable.
4. Contest running time - a week or a couple of weeks - I think it's better to do many contests to keep interest.
5. I don't have a reward - just for interest, or maybe there will be sponsors, or suggestions.

What the contest will provide:
1. An entertaining element to everyday life.
2. An opportunity to actually prove your abilities in working with data.
3. An opportunity to appreciate the difficulty of separating random patterns from non-random patterns.
4. Sharpen your skills.

Regularity can be established through a set of rules, let's say 3 rules described by variables through inequalities between them or constants. Recognising one rule gives a point. We hold five contests, whoever scores the most points wins. Speed bonuses are possible.
The result can be posted in the thread as an archive, protected by a password, at the end of the contest period the participant publishes the password and the organiser checks the results and counts the points. Everything is public.

What do you think? It seems to be an accessible contest for a wide range of people, you can use different methods.

1...3613361436153616361736183619362036213622362336243625
New comment