Machine learning in trading: theory, models, practice and algo-trading - page 3250

New comment
 
Maxim Dmitrievsky #:

I was originally interested in how to search for patterns in multidimensional arrays without MO.

Is it correct to say that this is the main task that MO does?

 
Renat Akhtyamov #:

I had to think hard to figure out how to explain it on my fingers.

...

thank you, our opinion is very valuable to you!

 
Maxim Dmitrievsky #:

Well, I'm still counting all the possible pairs at once. There's still a lot of inputs I want to try. That's fine. It's just that in STUMPY it is possible to calculate approximately and then refine. You get a noticeable acceleration, plus paralleling and on the GPU. I will probably switch to that package completely.

The main thing is not to forget to report where the fish are definitely not.

 
fxsaber #:

Is it correct to say that this is the main task that the MoD is engaged in?

Well, essentially yes

 
Renat Fatkhullin #:

The 3980 implemented Conjugate methods for the types complex, vector<complex> and matrix<complex>. They perform conjugation for complex numbers.

They also added processing of ONNX model output of Sequence of maps type. The functionality of ONNX Runtime has been seriously improved.

And it gives hints now, super

It turns out that this is an array of structures

vectorf in = vector<float>::Zeros(SAMPLE_SIZE);
   static vector out(1);
   
   struct abc 
     { 
      long a[];     
      float b[];
     };
   
   abc out2[1];
  
   OnnxRun(ExtHandle,ONNX_DEBUG_LOGS,in,out,out2);

So there are no errors for out2 now. I'll check it again later.

 
Forester #:

I think that the correlation will be influenced by the largest numbers in terms of abs value. For example, a change in volumes of 10000 and 10100, and on their background a change in prices of 0.00040 and 0.00400 will be microscopically small and will have little effect on the correlation of the whole set. I would do a normalisation to test this hypothesis.

I have a smoothly increasing period there, so maybe it doesn't have any effect

I'll give it a try.

 
Maxim Dmitrievsky #:

)) I saw it too

I was originally interested in how to search for patterns in multidimensional arrays without MO. So far I haven't thought of anything better than cramming all measurements into one and calculating through correlation (kinda fast). I guess sometimes the values need to be normalised so that they are not too different.

Following in my footsteps from 3-5 years ago.....

Everything you are doing and thinking about I have already posted here, with graphs, thoughts... funny....


There are two solutions to search for patterns in multivariate data that I've come up with, one without MO and one with MO.

1) (WITHOUT MO)

Reduce dimensionality of data to several dimensions using any dimensionality reduction algorithm PCA, t-sne, umap etc..

So you had 300 features, and you get 2-5...10..., then you compare patterns by proximity or clustering....

It's kind of a recognised practice for working with data.


2) (WITH MO)

(My author's approach) We have multivariate data with say 200 features.

1) We choose the pattern we want.

2) Train a binary classification MO (this pattern/ NOT this pattern) i.e. on the traine we have one observation labelled "pattern" and many observations labelled "NOT pattern".

3) We train the model to distinguish between pattern and NOT pattern.

4) On the test we make a probabilistic inference from the MO by the class "pattern" and watch the probability spikes.

This is how we can elegantly circumvent the problem of multidimensional features and search for sub-patterns that we need.

 
fxsaber #:

Is it correct to say that this is the main task that the MoD is engaged in?

Not correct

 

And so, just a rejoinder.

Correlation does NOT need normalisation, it is not a Euclidean distance, normalisation is already embedded within the correlation

 
mytarmailS #:

And so, just a rejoinder.

Correlation does NOT need normalisation, it is not Euclidean distance , normalisation is already embedded within the correlation

Thank you, Master.
PSA is not needed there, there are few measurements. The more measurements, the fewer instances of patterns.
1...324332443245324632473248324932503251325232533254325532563257...3576
New comment