Machine learning in trading: theory, models, practice and algo-trading - page 2539

[Andrei Trukhanovich]

Sceptorist #:

My opinion is only Open or tics.

Well, it depends on who needs what. In terms of certainty in time close is the most accurate

[Sceptorist]

And I would be grateful if someone would respond. I just started reading this thread. I'm about 100 pages in. Interesting, thanks to the authors of the early period. It's like a diary. Mistakes, discoveries, disappointments, joys of successes, dashed hopes... A novel, in a good sense of the word. Learned something new, remembered something old, something to laugh about (not without this). Full-fledged everyday life of a prospector, as it were. My question is simple, in this machine learning, "machine" will remain a black box? Have we fed it inputs/predicates and we want to get an answer? Have we looked into the "guts" of what and how it cooks? Maybe they tried to translate the machine into the MQL language they find here?

I'll probably finish reading this thread, it's going well so far, but I'd appreciate spoilers)

[Maxim Kuznetsov]

Andrei Trukhanovich #:

Well, that's a matter of opinion. In terms of certainty in time close is the most accurate

from the point of view of uncertainty in the candlesticks we know exactly either the time or the price... for close the time, for open the price :-)

figuratively, when from 15.58 till 16.03 there are no ticks (and this is a normal situation, there are typical moments of such holes), then close knows the time 16.00 but the wrong price, and for open the wrong time is the correct/relevant price

[Rorschach]

Sceptorist #:

And I would be grateful if someone would respond. I just started reading this thread. I'm about 100 pages in. Interesting, thanks to the authors of the early period. It's like a diary. Mistakes, discoveries, disappointments, joys of successes, dashed hopes... A novel, in a good sense of the word. Learned something new, remembered something old, something to laugh about (not without this). Full-fledged everyday life of a prospector, as it were. My question is simple, in this machine learning, "machine" remains a black box? Have we fed it inputs/precipients and we want to get an answer? Have we looked into the "guts" of what and how the Rrrr cooks? Maybe they tried to translate the machine into the MQL language they find here?

I'll probably finish the thread, it's going well so far, but I'd appreciate spoilers)

To achieve enlightenment start with a branch on Onyx and only then start this one *sarcasm

Read books

[JeeyCi]

I think the philosophy here is simple :

(H+L)/Close. i.e. close. as the fairest (equilibrium) at the current(!) Moment, H/Close+L/Close, as the sum of fractions in the time span, with a total of either + or - i.e. up/down result of momentum... imho

Sceptorist #:

. My question is simple, in this machine learning, will the "machine" remain a black box? You feed it inputs/predicates and want to get an answer?

...and the answer is kind of simple.

Maxim Kuznetsov #:

In theory, yes, but where is the vector of weights or forward+reverse coordinate transformation?

The MNC is an almost universal method, what can I say ... I mean, it's abstract, but for it to work you need a reasonable physics of the process...

.. VMNC - weighted MNC (scales, for example, dispersion)... in general, it seems to me that everything brilliant should be simple...

Renat Akhtyamov #:

I don't know what they're cooking up

predictors for MO most likely (scales are concerned)

I suspect that they make up a function like

price = a1*y1+a2*y2+...aN*yN

a logical trick in principle

it is interesting what will result

only if you break it up into segments, you probably need to multiply each part by something related to the angle

polynomial - for multivariate analysis (and your formula - linear!!! - by the way, coefficient a at your y [though x] is the angle) - ... for single-factor - just a typical straight line equation (y=kx+bias)...

and here begins the most interesting thing about differentiation - the order of the polynomial (=number of its bends-1 and signal propagation from the beginning of training to the finish) -- you certainly shouldn't get carried away, but if reality is such that dy/dx=a^3x ( not a linear dependence) and above. -- then I don't think there's much to do here -- take the derivative of what's there (I think I saw somewhere recommendations for differentiation depending on 1s and 2s differences => choosing degree of polynomial -- can't find it)... OR consider the CDF and throw out the extreme persentiles... imho to find the mean... OR to exploit them as signals in the analysis of deviation from the mean... imho... That's how Maxim Dmitrievsky put it in ML terms

+ and the problem of a damped or increased gradient is always on the agenda too -- here, I suppose, the right weighting comes in handy... Although the philosophy of weighing again can be their own for supporters of "freedom to choose weights"... I hate freedom fighters in the way of Russian language (so they start to distort real correlations and cause-effect relations in formulas too) -- right differentiation (i.e. right variables) will give right weights, and right NN architecture will help to bring significance of weights to the training result... imho

p.s.

all the same to me H and L fractions from close inspire more confidence than just close... - that's the point here - to set scales correctly in black box (that's why it's important what's on the input, what's on the output) -- not to get unreasonably high/low dispersion... and not to lose significant gradient at learning stage - and as Mikhail Mishanin already said (before the thread gets sunk in floods and squabbles) - "let the most important thing to survive and evolve"

in NN - almost like on Titanik

[Mihail Marchukajtes]

I trade with this polynomial and don't bother

double decision = -0.07651082803761469 * sigmoid(x1 + x2 + x3) -0.04311207233300622 * sigmoid(x1 + x3 + x4) -0.05386865347421374 * sigmoid(x0 + x1 + x5) -0.00827465962899799 sigmoid(x1 + x2 + x4 + x5) -0.10576034451547747 * sigmoid(x4 + x6) -0.10960470278870797 * sigmoid(x0 + x2 + x3 + x5 + x6) -0.07378595451557275 * sigmoid(x0 + x1 + x2 + x4 + x5 + x6)  + 0.12026124486704332 * sigmoid(x7)  -0.06493964309873379 * sigmoid(x0 + x1 + x4 + x5 + x7)  -0.019388523137606112 * sigmoid(x1 + x2 + x4 + x5 + x7)  + 0.11097666707952629 * sigmoid(x4 + x6 + x7)  -0.0850998961499252 * sigmoid(x2 + x5 + x6 + x7)  -0.07273316247296563 * sigmoid(x0 + x4 + x8)  -0.2787231204565592 * sigmoid(x1 + x3 + x5 + x8)  -0.05990910736573329 * sigmoid(x0 + x1 + x4 + x6 + x8)  -0.0678407759220267 * sigmoid(x0 + x1 + x5 + x6 + x8)  -0.009089263426671367 * sigmoid(x0 + x2 + x3 + x4 + x7 + x8)  -0.10259720462275745 * sigmoid(1.0 + x3 + x4 + x8);

And as I said before, the polynomial itself is not as important as the method of obtaining it. But everyone becomes deaf when they can not understand an elementary phrase ...

[JeeyCi]

JeeyCi # :

I still have more confidence in the H and L shares of close than I do in close...

I'll add/refute it myself:

And here again the same problem - the recommendationshere- fractions may not be a panacea, but the dynamic range may not be "period difference between 2 close" at all

Khristian Piligrim #:

Of course it's very important for stable operation and effective forecasting to correctly train the model, and for this, first of all, we need to correctly form the input data. For this purpose, I first scaled up the input data by analogy with what I did in my article "The Principle of Superposition and Interference in Financial Instruments" , and then I shifted the scaling grid so that the data were always in the same dynamic range, no matter how the market changed, I abandoned traditional normalization methods, they distort the data too much. In the next phase I tried to ensure that the vector, in relation to which the training was conducted, was completely covered by the input variables, in Fig. 1. - is a bad overlap, while in Fig. 2. - is much better, and, accordingly, the accuracy of training will be significantly higher (the black line is the vector, relative to which the training is conducted, and the other lines are the input signals).

i.e. standard normalization on variance and not particularly suitable... (

? Maybe to add to normalization coefficient for incoming data also WMA, or simply by weight, - after all, it reflects dynamics (though with a lag)

p.s.

1) but maybe "dynamic range" is painfully simple - the point of intersection of 2 MAs - it is important to choose the right periods... only OTFs look at 50 and 200... while for bigData analysis more profitable MA periods may be found by the memory (weights) of the neural network (in case of other related factors)... imho

2) although personally I think that "dynamic range" is the one/those periods, where the price was still normally distributed from Level to Level (I guess you could say the cluster - we did mark-up and again work/classify by weights/characteristics/memory already defined in the market before - before the new OTF arrival)... but how to exploit this logic in input rationing - I don't know yet (except just to make all the same dy/dx in addition to just t-statistics)... It's bad, of course, that the strategy tester doesn't select features (indices) itself, and optimization is only possible of what is given to it (and far from clean source info)... - So people have to go to ML

[BillionerClub]

You have a gold mine, but you can't see underfoot.

[JeeyCi]

BillionerClub #:
You have a gold mine, but you can't see underfoot

It's clear that SVM allows linear separation of non-linear dependencies (only dimensionality reduction has to be somehow adjusted - it has its own nuances)... But first of all, multivariate analysis (with derivation of polynomial of multiple regression) is so-so for me, when all factors influence each other, and I don't know how the library makes its feature_extraction (and there are a lot of nuances in statistics) ... Secondly, to pick up correct hyperparameters for SVM in python - you also need to know the library somehow... many are stomping around here (the library is decent) - if only the nuances that I described the modeling process with this library did not generate the output model with overstated/understated/inaccurate metrics, overstudied or understudied...

to understand this library, if you see it for the first time, you'll have to look under your feet for a long time...

the "golden" part is debatable... I'm still skeptical about the inability to hold trends and jump out of them early because of the robot... But I don't want to tolerate the drawdowns when the robot didn't notice something either... so just a high-quality statistical model will be worth its weight in gold even before ML... to try to increase the probability of 50/50... imho

[JeeyCi]

JeeyCi #:

OR consider the CDF and discard the extreme persentiles... imho to find the middle one... or exploit them

probability of hitting PDF tails(which is essentially a derivative of CDF, i.e. PDF is a differential PDF): 1-P(a<=X<=b) , where [-infinity,a] and [b,+infinity] are tails of the distribution