Machine learning in trading: theory, models, practice and algo-trading - page 371

 
Dimitri:


I still don't get it - inverse correlation or no correlation?

Or, do you think that if two random series have a correlation coefficient of -1, then they "have no correlation"?

Yoeklmn.....


ah... I got it right away )) well, yeah...
 
Dmitry:


1. no one is analyzing correlation-it's about the choice of predictors.

2. You repeated my point three pages earlier -"Dependence is a special case of correlation. If two variables are dependent, then there is definitely a correlation. If there is correlation, then there is not necessarily dependence."

3. cross-entropy just like correlation will not give an answer on the presence of functional dependence


1) First you speak for correlation yourself, then you delete posts, then you don't remember what you wrote about a couple of pages ago.

Dimitri:

All MO is based on the fact that the input variables must correlate with the output variable.


2) no, I said there can be a dependence even where there is no correlation.


3) Cross-entropy. You can estimate a large set of predictors at once in relation to a target. When each of the predictors is not capable of predicting it, but a certain combination of them is. Unfortunately, it is not true for forex, but in general the selection of predictors through cross-entropy is much better for machine learning than selection through correlation.

 
Dr. Trader:


2) No, I said that there can be a correlation even where there is no correlation.



There can be no dependence where there is no correlation. Correlation can be linear or non-linear, but it will be if there is dependence.

There can be correlation where there is no correlation - a false correlation.

I have not deleted a single post in this thread.

 

I can't (I don't know why) download the book to give proof.

Whoever has it, download and post an example picture here to stop the idle bickering.

Bendat J., Pearsol A.

Applied Random Data Analysis: Translated from English: World, 1989.

At pp. 126

EXAMPLE 5.4. UNCORRELATED DEPENDENT RANDOM VARIABLES.


.

ss

example is very telling.

 
Correlation of variables does not mean it is possible to predict. The pairs can be correlated. That is, they are correlated, but you can't predict one through the other because they change simultaneously, and certainly not ahead of time. This is if we talk about correlation!!!!
 
Dimitri:

Two equally correlated predictors - what do we throw out based on lower correlation? Which one is less correlated?

Dmitry, I'm sorry, but I suspect you are either trying to troll me, or fooling around, or just stupid, with all due respect... Can 't you see from a trivial example that two attributes both have zero correlation with the target, BUT both are significant, neither can be dropped, linear dependence is zero, not linear 100%, that is, the correlation can be zero and the dataset is completely predictable, what your statement:

Dimitri:

All MO is based on the fact that the input variables must correlate with the output variable.

Otherwise there is no point in ALL MO models.

Completely refutes it.

 
Dimitri:


There can be no dependence where there is no correlation. Correlation can be linear or nonlinear, but it will be if there is dependence.

False again, there is no non-linear correlation correlation is a STRICTLY defined mathematical structure, like addition or cosine, study at least wikipedia before talking nonsense.
 
Oleg avtomat:

I can't (I don't know why) download the book to give proof.

Whoever has the opportunity, download and post an example picture here to stop the idle bickering.

Bendat J., Pearsol A.

Applied Random Data Analysis: Translated from English: World, 1989.

At pp. 126

EXAMPLE 5.4. UNCORRELATED DEPENDENT RANDOM VARIABLES.


.

ss

example is very telling.


http://sci.alnam.ru/book_dsp.php

only there is no picture on page 126

 

Why are you all so obsessed with correlation?

In machine learning, there is a concept of " importance-importance" of variables, which has nothing to do with correlation at all. The calculation is often built into the machine learning algorithm itself.

For example, in a random forest.

From the entire subset of predictors, there may be several hundred, a few pieces are selected and the internal algorithm is used to see if the values of these predictors predict a particular value of the class. They are either accepted or discarded.

At the end they look through all the nodes of the tree and see how many times a predictor was used in each node of the tree - this gives the importance of predictors.


I keep trying to bring the team together to discuss existing developments in this area, which are much more meaningful than exercises in correlation.

 
SanSanych Fomenko:

Why are you all so obsessed with correlation?

In machine learning there is a concept of "importance - importance" of variables, which has nothing to do with correlation at all. The calculation is often built into the machine learning algorithm itself.

For example, in a random forest.

From the entire subset of predictors, there may be several hundred, a few pieces are selected and the internal algorithm is used to see if the values of these predictors predict a particular value of the class. They are either accepted or discarded.

At the end they look through all the nodes of the tree and see how many times a predictor was used in each node of the tree - this gives the importance of predictors.


I keep trying to get the team to discuss existing developments in this area, which are much more meaningful than exercises in correlation.


there's a dark forest in alglieb too, by the way... you can use it without leaving mt5