Matstat Econometrics Matan - page 31

 
Dmytryi Nazarchuk:

And Shiryaev's two-volume book can be thrown at cockroaches.

I see you've already been hit.

 
Aleksey Nikolayev #:

Well, stability can probably be calculated somehow)

This would be interesting to discuss. Some simple robust metrics, such as percentage of profitable trades, or percentage of new equity haves. Maybe someone knows the best stability metrics?
 
secret #:
Now this would be interesting to discuss. Some simple robust metrics, like percentage of profitable trades, or percentage of new equity haves. Maybe someone knows better stability metrics?

Calculation and interpretation of any metric (including robustness checks) is always possible only within the framework of some models. A metric will always be adequate to the model, but if the model is inadequate to reality, so will the metric)

The notion that one can come up with some absolute, model-free metrics seems to me to be erroneous and harmful. Often it just means unconscious application of some standard model without a clear understanding of the limitations it imposes on the described reality.

For each particular model there is a limit to the number of independent metrics and in too large a set of them they will be expressed one through the other. For example, for SB everything can be expressed through variance. For SB with drift, it can be expressed through drift and volatility, and for the multivariate variant correlations will be added.

Calculating and expressing some metrics through others can be a very complicated mathematical task. This can be easily seen, for example, by trying to calculate the distribution of the recovery factor at least for SB)

 
secret #:
This would have been interesting to discuss. Some simple robust metrics like percentage of profitable trades, or percentage of new equity haves. Maybe someone knows better stability metrics?

There was a German bank which ran a contest in 2010 on demo accounts.

There, the key parameter to select the winners was the return over the three months of the competition, adjusted for the variance of the return curve over those 3 months.

 
Aleksey Nikolayev #:

The notion that you can come up with some absolute, model-free metrics seems to me misguided and damaging.

Let's be more specific) what's wrong with, for example, the percentage of profitable trades?
 
Dmytryi Nazarchuk #:

There was a German bank which ran a contest in 2010 on demo accounts.

There, the key parameter to select the winners was the return over the three months of the competition, adjusted for the variance of the return curve over those 3 months.

Well, that's the Sharpe ratio.
 
secret #:
Let's be more specific) what is wrong with, for example, the percentage of profitable trades?

OK, let's be specific)

Not bad if it corresponds to a system - it really makes a sequence of deals, not overlapping in time, with invariable (from opening to closing) volumes and with the same stop-loss/take profit ratio for all deals.

If the system, for example, monitors a position volume that can change frequently and smoothly (sometimes flipping through zero), then the very notion of a deal applies to it only in a purely technical sense (in terms of MQL5, for example).

 
Well, that's how all systems work.
Changing volume is self-defeating, it does not change the average trade if the conditions of the trades are the same. An adequate test should only be performed with fixed sizing.
 
secret #:
Well, that's how all systems work.
Changing volume is self-defeating, it does not change the average trade if the terms of the trades are the same. An adequate test should only be done with fixed sizing.

Also say that locks are evil and that all-way grids are a hidden martingale game...and give up on all signals and martingale :-)

 
secret #:
Let's be more specific: what is wrong with, for example, percentage of profit trades?

The percentage of profitable trades is too specific to the TS and reflects poorly on the quality of the TS. For example, 30% is OK for a trend trade, but 70% is not enough for a counter-trend trade.