A single quality indicator for the strategy - page 7

New comment
 
Youri Tarshecki:

To compare the two strategies, even a 60% market fit would be enough to make a choice in favour of one or the other. You will just have to increase the sample size.

For real code work the available 95% accuracy of simulation of the environment at 1m is very good . And a tick history will give all 99% .Many scientific and practical environment models cannot boast of such accuracy .

I do not see any reasons to believe that the automatic method of strategy creation is somehow miraculously different from the manual one in terms of history quality dependence.

If the history is bad, it is bad for all Expert Advisors.

In other words, the question of choosing the environment is not directly related to the criteria of system evaluation.

"I do not see any reasons to believe that the automatic method of strategy creation is somehow miraculously different from the manual one in terms of history quality dependence." - An automatic search can find a "strategy" that exploits the technical features(defects) of the tester itself combined with the quality of historical data. Apparently you haven't encountered such a problem)). Here is an illustrative example:

Suppose that historical data of OHLC format is used, ticks are not modelled, smaller periods are not used, market entry/exit conditions are calculated at each arrival of O, H, L or C value.

In the figure:

As (O-L) + (H - C) < (H - O) + (C-L) the tester simulates market movements using values in the sequence shown in the figure, which is quite logical.

As a result, it turns out that at the point in time indicated by the red line, the two values Open and Low (for the strategy, it will be Close) will contain information about the "future".

Here is the rule of "taking profit":

if (Open[0] - Close[0])> X and (High[0] == Open[0]), then buy, where X is some value overlapping the spread.

Note that at this point in time the OHLC values for the strategy will match the OOLL values of the historical data!!!

The tester defect and the poor quality of the history is shown.

 
Aliaksandr Hryshyn:


Shows the defect of the tester and the poor quality of the story.

That's what you mean by the future. This phenomenon has been known for a long time. I remember about three years ago there was even an article about studying the consequences of taking profit on the basis of an EA with this feature. The profit was generated by the Expert Advisor without any pauses.

But what is the probability that the strategy will learn to catch this chip by itself and even taking into account the spread if you don't teach it to do so? It seems to me negligible.

Of course, in reality the price can go back and forth as it pleases inside a candle. That's what creates those 5-10% errors of the test from the real. I mean it is not critical for comparing strategies on a test because they will all be in the same conditions anyway.

And I mean if you think it is critical, then take the real tick history, sacrificing the speed.

I.e. it's all unimportant compared to the testing method to be used.

 
You are definitely missing the point)). In the strategy tester, in my example with the defect, the EA will only give positive trades, a grail)). When autodetecting strategies, which is designed to search for "grails", it may stumble on these defects.
Believe me, this is a serious problem for the strategy generator, it is already there, certainly better than in the example. Good Equity for non-optimized plot can give out in a minute, while I'm working on increasing "intelligence". Similar programs that I found on the Internet do not and cannot have many problems, this is due to certain rule generation pattern, example of a generated rule: (A<B) & (A>C) & if(D>C;true;A>B).
 
There were times when the system found a "strategy" that in about 150 trades had about 15 losing trades out of them, and this on a non-optimised plot. Somehow there is little confidence in these results.
 
Aliaksandr Hryshyn:
There were times when the system found a "strategy" that in about 150 trades had about 15 losing trades out of them, and this on a non-optimised plot. Somehow there is little confidence in these results.
Will you shoot yourself?
 
There are other defects as well... By the way, they can also give the best results in a non-optimised area, as they use non-price behaviour patterns.
 
Алексей Тарабанов:
Shoot yourselves?
))
 
Aliaksandr Hryshyn:
You are definitely missing the point)). In the strategy tester, in my example with the defect, the EA will only give positive trades, a grail)). When autodetecting strategies, which is designed to search for "grails", it may bump into these defects.
Believe me, this is a serious problem for the strategy generator, it is already there, certainly better than in the example. Good Equity for non-optimized plot can give out in a minute, while I'm working on increasing "intelligence". I have found many problems with analogous programs on the Internet, it has to do with specific rule generation pattern, e.g. generated rule: (A<B) & (A>C) & if(D>C;true;A>B).

We are talking about two different things.

1.I am aware of your example long ago. My opinion - probability of such an accidental creation during "evolution" is insignificant. If you're still afraid of such and similar "non-price" tricks - just do a tick history check with a plausible spread - and everything will reveal itself at once. Just change the type of testing in the ini-file sometimes.

Rule generation templates, in my opinion, so that there are no hiccups, should work at a sufficiently high level of generalisation of these rules and block compatibility. And then the user himself can set these dependencies and the elements that the generator has to work with.

2. If the generator contains test-based selection of strategies, then volking forward is a MUST. Besides, it will dramatically simplify the selection criterion.

 
Youri Tarshecki:

We are talking about two different things.

1.I have been aware of your example for a long time. My opinion is that the probability of accidental creation of such a thing in the course of "evolution" is negligible. If you are still afraid of such and similar "non-price" tricks - just do a control test on a tick history with a plausible spread - and everything will reveal itself immediately. Just change the type of testing in the ini-file sometimes.

Rule generation templates, in my opinion, so that there are no hiccups, should work at a sufficiently high level of generalisation of these rules and block compatibility. And then the user himself can set these dependencies and the elements that the generator has to work with.

2. If the generator contains test-based selection of strategies, then volking forward is a MUST. Besides, it will dramatically simplify the selection criterion.

Probably it will).

What do you think the rules should be? Just wondering how you see the problem.

 

I suggest this option.

This assessment is suitable for strategies that have the following features/limitations:

  • the presence of a stop loss
  • the risk of each trade is fixed and measured in the currency. the lot is calculated based on the size of the stop loss, so when it triggers, we lose only the amount defined by the risk
  • several trades can be considered as one trade, if they are a part of some trading scheme and the total risk is fixed.

Profit and drawdown are represented as the ratio of their respective values to the fixed risk/stop loss.

The main performance indicators of the strategy are taken and transformed as shown in the figure below, the order of formulas corresponds to the order of indicators in the title.

On the chart there is also the complexity of the strategy, you can not use it, I needed it.

The volatility shows the sum of the maximum deviations of the equity values from the regression line in the corresponding values, graphically it is so:

Volatility is the sum of the absolute values of A B, i.e. A+||B|

There is also a slightly modified value of profitability which is calculated as follows

Profitability=(total profit + 1)/(total loss + 1)

It is transformed as follows:

Since the profitability is not very useful with a small number of trades, we do the following:

the chart above shows how the profitability value will be calculated depending on the number of trades.

Then we change the profitability once again (based on its importance depending on the number of trades) using the following formula

Profitability = 1 + Profitability * Significance - Significance

Then all obtained values of profit, drawdown, number of trades, volatility and profitability (from the last formula) are multiplied, as the result we get some single coefficient reflecting the general quality of the strategy based on the trade in a certain period of time.

Ratio values:

0 - mercilessly terrible)

0.3 - bad

~0.8 - good

>1 - very good

Here are the results of the assessment in this way:

Here this factor is represented as Z-coefficient, you can evaluate training (green line) and test (purple line) sets.

A good result for the green equity line (0.993) and the inferior purple line (0.5714)

For the purple line is slightly better than bad, the profitability is 1.447

Examples of bad results.

123456789
New comment