Machine learning in trading: theory, models, practice and algo-trading - page 1912

New comment
 
Mihail Marchukajtes:

In other words, the algorithm will assign unreasonably high weighting coefficient to these two close vectors....

But why not justified? If something is repeated often, say, 100 times and gives some statistically significant result, then it is adequate to give more weight to it than to an example with one observation and an unclear outcome.


1) you have seen clouds 100 times, it rained 70% of the time

2) your knee itched once and it rained.


why do you think that these two patterns should be given the same weight? the whole world thinks otherwise, and so does neuronics :)

 
mytarmailS:

But why not justified? If something is repeated often, say, 100 times and gives some statistically significant result, it is adequate to give more weight to it than an example with one observation and an unclear outcome.


1) you have seen clouds 100 times, it rained 70% of the time

2) your knee itched once and it rained.


why do you think these two patterns should be given the same weight? the whole world thinks otherwise, and so does neuronics :)

and these statistics don't teach you anything?

you make money 100 times and you win xxx times

;)

 
mytarmailS:

But why not justified? If something is repeated often, say, 100 times and gives some statistically significant result, it is adequate to give more weight to it than an example with one observation and an unclear outcome.


1) you have seen clouds 100 times, it rained 70% of the time

2) your knee itched once and it rained.


why do you think these two patterns should be given the same weight? the whole world thinks otherwise, and so does neuronics :)

If a pattern is repeated 100 times and belongs to the same class, then a neural network is not needed. It is enough to identify this pattern and draw conclusions. The task is not to train the network, but to generalize it. But again, it all depends on the training algorithms and selected network topologies.
 
Colleagues, forgive me generously, but something I am dumb in my statement. Although I have a training set consisting of less rows than columns. However, the training itself takes place on a sample of 11 columns. In general, I did not do it on purpose. I didn't know what I was doing myself :-(
 
Mihail Marchukajtes:
Colleagues, forgive me generously, but I got lost in my statement. Although I have a training set consisting of less rows than columns. However, the training itself takes place on a sample of 11 columns. In general, I did not do it on purpose. I didn't know what I was doing myself :-(

Forgive me) those who thoughtfully watched it understood it...

But I still haven't heard the answer to my simple question - WHY are unique values better than statistically significant ones?

Also if you want to make all rows unique I gave you an idea how to do it, why don't you?


===============================================

Cool video - how and in what ways REAL problems were solved in MO , there is a problem with BP as well, very interesting


 
mytarmailS:

Forgive me) those who have looked thoughtfully have understood...

But I still haven't heard the answer to my simple question - WHY are unique values better than statistically significant ones?

Also if you want to make all rows unique I gave you an idea how to do it, why don't you?


===============================================

Cool video - how and in what ways REAL problems were solved in MO , there is a problem with BP as well, very interesting


Yes thanks for the code, I saved it but haven't really studied it yet... In the week I think I'll look at it...

Because statistically significant vectors can be used head-on, without NS. On learning NS will be better to learn those vectors which are clustered in one area. If in the future the same vectors will appear, everything is OK, but if a vector appears in the vicinity, but is from a different class, the network will make a mistake 100%, because it will not learn that this group belongs to a certain class. IMHO

 
Mihail Marchukajtes:

the same vectors, then everything is OK, but if a vector appears in the vicinity, but is from a different class, the network will make a mistake 100%, because it has notified that this group belongs to a particular class. IMHO

Aaaaah, well, I got your point...

Look, I'm sorry I snapped at you, sometimes I'm not all right.

 

Without encroaching on your grail) there is a question - how does a neural network predict the bank's desire to buy or sell a certain volume (affecting the price) of currency (for example)? Any neuronet can only catch the inertia in the direction when speculative volumes are sold and the price actively starts moving. But a neuronet will never indicate exactly the moment of entering the participant who will change the price (price driver), although it is not needed - in 99% if you know how to identify the price driver and its direction your transactions will always be before the realization and you will have a stable profit. My company also has a department of algorithmic trading, but it is more focused on very fast scalping - the neural network also catches the inertia of the hft algorithms and based on its analysis our robots make deals by fully copying hft (only for certain markets and symbols). Most trading is done by hand in the old way, since it is impossible to predict the price driver (and there is no need to), you can only see it = the car will show you the turn signal before changing direction (you understand that the neural network cannot predict at what moment the turn signal of this or another car will turn on).

 

decided to see how the typical data would look like for 3d training of NS))

the data is 31 indicators, the target is a zigzag

i reduced the dimensionality to three dimensions with three algorithms - pca, t-sne , umap (the last two are considered the most advanced)


what is it anyway -https://en.wikipedia.org/wiki/Dimensionality_reduction

how it can help -https://ru.wikipedia.org/wiki/%D0%9F%D1%80%D0%BE%D0%BA%D0%BB%D1%8F%D1%82%D0%B8%D0%B5_%D1%80%D0%B0%D0%B7%D0%BC%D0%B5%D1%80%D0%BD%D0%BE%D1%81%D1%82%D0%B8


So data 31 indicator target zigzag , first we have PCA

Next t-sne


umap


as we can see we can't divide it by classes, so either the target is rubbish or the signs, or all of them together)))


This is how a surface with good separability should look like, it has three classes but I think you will get the idea


 
Viktar DayTrader:

I have a question - how does a neural network predict the bank's desire to buy or sell a certain volume (affecting the price) of currency (for example)?

Large purchases are not made in a second, it takes time, during this time the price will display a pattern, this pattern can be tried to find with the help of machine learning algorithms

1...190519061907190819091910191119121913191419151916191719181919...3673
New comment