Machine learning in trading: theory, models, practice and algo-trading - page 1475

 
Alexander_K:
A couple of weeks ago I had a question about why models learn and trade so well on your real ticks from 03_AUDCAD. The answer I've now come to is.
Because the distribution of price gains is symmetric, and that symmetric distribution is preserved in the sliding window.
Something like this is what I need to achieve on M15.
2018.04.16 22:43
Very interesting. I will check it.
2018.04.17 00:31
2018.04.17 00:57
There are 10000 last price increments on real ticks from 03_AUDCAD.xls
The yellow line is a moving average with a window of 100. It is almost perfectly flat.
2018.04.17 00:58

And here is the EURUSD M1 for comparison. 10,000 last bars, no thinning. The average is constantly going way off to the side.

2018.04.17 01:04
2018.04.17 01:04

This is one of the last entries I had in my PM from Doc... Something made me cry, remembering the old days...

And what is the criterion by which some of the ticks are thrown off? Or does the word "thinning" carry a different meaning?

 
elibrarius:
Have you studied AUDCAD including the spread? It is huge there - about 40-50 pips. I looked at the chart - in the last 100 minutes the price was within the spread

Yes.

Despite the fact that Doc's model on thinned ticks was giving great results, the spread was eating up almost all the profits. Therefore, he moved on to thinning, up to obtaining an event (quote) about once every 15 min. Alas, I don't know what happened to him. Disappeared... Maybe killed like Alyosha - who knows...

I stopped at the achieved, and simply applied to the obtained BP formulas from the theory of random processes.

 
Aleksey Vyazmikin:

And what is the criterion by which some of the tics are thrown away? Or does the word "thinning" carry a different meaning?

I thinned by Erlang as the simplest stream of events. There is a series of tick quotes, every 2nd quote is left from it - we study it, it doesn't fit - so every 3rd quote, etc. Until a series with certain properties is obtained.

 
Alexander_K:

I was thinning by Erlang. There is a series of tick events, every 2nd quote is thrown out of it - we study it, it does not fit - so every 3rd quote, etc. As long as we have a series with certain properties.

Let's assume that we have already shuffled this stuff on the history, found the desired distribution, and then what? If we start thinning not from the first tick but from the second, we will have to throw out completely different data, right? I don't understand how it can be done in real time from the desired point.

 
Aleksey Vyazmikin:

Let's say we threw this stuff out on the history, found the desired distribution, and then what? After all, if we start thinning not from the first tick but from the second one, we will have to throw out completely different data, right? I don't understand, how it is possible to thin in real time from the desired point.

:))) Well, it took me 1.5 years to do that. But, without thinning, I have no idea how to solve this problem at all.

And then - at Doc's NS stupidly predicted the sign of the next increment, while I got the Ornstein-Uhlenbeck process with a return to the mean.

 
Alexander_K:

Yes.

Despite the fact that Doc's model on thinned ticks was giving great results, the spread was eating up almost all the profits. Therefore, he moved on to thinning, up to obtaining an event (quote) about once every 15 min. Alas, I don't know what happened to him. Disappeared... Maybe killed like Alyosha - who knows...

I stopped here and simply applied formulas from random processes theory to BP obtained.

Man... I think these jokes about Doc are no longer funny, given that Aliosha did die a violent death, as presumably did Yura Reshetov. And DR_TR, fortunately, is alive and well, working for a salary as a clerk, doing boss's errands, and does not even remember about this whole nightmare with the markets, at least until the spiritual wound is healed, after losing about 3 kilobucks on the crypto exchange, and then, I'm sure, he will return refreshed and with new ideas.

 
Alexander_K:
A couple of weeks ago I had a question about why models learn and trade so well on your real ticks from 03_AUDCAD. The answer I have now come to is.
Because the distribution of price gains is symmetric, and that symmetric distribution is preserved in the sliding window.
Something like this is what I need to achieve on M15.
2018.04.16 22:43
Very interesting. I will check it.
2018.04.17 00:31
2018.04.17 00:57
There are 10000 last price increments on real ticks from 03_AUDCAD.xls
The yellow line is a moving average with a window of 100. It is almost perfectly flat.
2018.04.17 00:58

And here is the EURUSD M1 for comparison. 10,000 last bars, no thinning. The average is constantly going way off to the side.

2018.04.17 01:04
2018.04.17 01:04

This is one of the last entries I had in my PM from Doc... Something made me cry, remembering the old days....

https://www.hindawi.com/journals/tswj/2015/909231/

The Lambert Way to Gaussianize Heavy-Tailed Data with the Inverse of Tukey’s h Transformation as a Special Case
The Lambert Way to Gaussianize Heavy-Tailed Data with the Inverse of Tukey’s h Transformation as a Special Case
  • Hindawi
  • www.hindawi.com
I present a parametric, bijective transformation to generate heavy tail versions of arbitrary random variables. The tail behavior of this heavy tail Lambert random variable depends on a tail parameter : for , , for has heavier tails than . For being Gaussian it reduces to Tukey’s distribution. The Lambert W function provides an explicit inverse...
 
Alexander_K:
A couple of weeks ago I had a question why models learn and trade so well on your real ticks from 03_AUDCAD. The answer I have now come to is.
Because the distribution of price gains is symmetric, and that symmetric distribution is preserved in the sliding window.
Something similar I need to achieve on M15.
2018.04.16 22:43

I have already written about it, and explained that it is nonsense....

You don't need to be a genius to make a series with such properties, you don't need to use some exotic transformations etc. It is enough to make a double/triple differentiation of series....

And yes!

1. We get a super stationary series with symmetric gains and smooth sliding

2. We will get a constant return to zero

We will receive excellent predictability of such series from any classifier, higher than 90%.


But having applied such a signal to the market, we will be squashed at the first trend, because after inverse transformation this signal is not worth a penny!

So come onAlexander_K

Either justify my wrongness with evidence (a screenshot of someone else's trade without nicknames is not proof that you're right)

Or stop spreading this nonsense in the masses, someone may also believe ...

I'm waiting for a substantive discussion.

 
mytarmailS:

Either argue my wrongness with evidence (screenshots of other people's trade without nicknames do not count as proof of your rightness)

Or stop spreading this nonsense to the masses, who can also believe ...

I'm waiting for a substantive conversation.

The problem is that the drawdowns are equivalent to profits, this graph does not show equity. So the proof of the method is very dubious, all due respect.

I agree about the strong moves, there just haven't been any lately
 
mytarmailS:

But applying such a signal to the market, we will be squashed at the first trend, because after the reverse transformation this signal is not worth a penny!

So come onAlexander_K

Justify my wrongness with proofs (a screenshot of someone else's trade without nicknames is not proof that you're right)

Or stop spreading this nonsense in the masses, someone may also believe ...

I'm waiting for a substantive discussion.

I don't have to say anything. Would that make you feel better?

Especially since the Doc disappeared and for his work I can't say anything, except that I saw his signal grow, and then bang - and there's nothing...

About my signal:

I have described everything I can in the TYP branch. And originally it is all built on the thinning of the tick stream. Just on M1, M5, .... No random process theory formulas work. In fact it comes out +0% profit like on SB. On the thinned, nonlinear in time, rows - work. I do not know why it works this way and not otherwise.

Can I simply stick thinned series in VS and gain profit? This question could be answered by Doc ... My personal opinion - no. I said this to Maxim. NS must still know the theory of random processes and independently derive the Einstein-Smoluchowski formula for the variance of the process... To defeat the human genius of NS is not possible. IMHO. I may be wrong...

But, in fact, almost no one in this thread deals with preprocessing of input data. But 1000 pages ago Koldun said that this is the most important and this stage is the main secret of all masters of MI. And you have to listen to Koldun.