Question on genetic optimisation - page 9

 
Angela >> :

Can you tell me, who knows, how much do mismatch errors in graphs distort the test result, can this test be trusted?

>> distort. By how much? Different every time. But it's better to have no mistakes at all, don't you agree? Here's a link where Igor Kim explains popularly how to make a quality story
 

The tester is killing me! Again the question is how to treat this and what to believe?

Here are two TC runs in a row, with the same settings and on the same interval:

Strategy Tester Report
Alpari-Demo (Build 225)


Symbol GBPUSD (Great Britain Pound vs US Dollar)
Period 5 Minutes (M5) 2009.08.03 00:00 - 2009.09.08 23:55 (2009.08.03 - 2009.09.09)
Model All ticks (most accurate method based on all smallest available timeframes)
Parameters Lot=0.1; StopLoss=0; TrailingStop=0;
Bars in history 8716 Modelled ticks 1462505 Modeling quality n/a
Chart mismatch errors 797
Initial deposit 1000.00
Net profit 660.08 Total profit 763.88 Total loss -103.80
Profitability 7.36 Expected payoff 36.67
Absolute drawdown 3.80 Maximum drawdown 89.00 (7.07%) Relative drawdown 7.07% (89.00)
Total trades 18 Short positions (% win) 5 (60.00%) Long positions (% win) 13 (92.31%)
Profitable trades (% of all) 15 (83.33%) Loss trades (% of all) 3 (16.67%)
Largest profitable trade 124.44 losing deal -52.90
Average profitable deal 50.93 losing trade -34.60
Maximum number continuous wins (profit) 10 (566.00) Continuous losses (loss) 1 (-52.90)
Maximum Continuous Profit (number of wins) 566.00 (10) Continuous loss (number of losses) -52.90 (1)
Average continuous winnings 5 Continuous loss 1

Strategy Tester Report
Alpari-Demo (Build 225)


Symbol GBPUSD (Great Britain Pound vs US Dollar)
Period 5 Minutes (M5) 2009.08.03 00:00 - 2009.09.08 23:55 (2009.08.03 - 2009.09.09)
Model All ticks (most accurate method based on all smallest available timeframes)
Parameters Lot=0.1; StopLoss=0; TrailingStop=0;
Bars in history 8716 Modelled ticks 1462505 Modeling quality n/a
Chart mismatch errors 797
Initial deposit 1000.00
Net profit 851.04 Total profit 928.34 Total loss -77.30
Profitability 12.01 Expected payoff 40.53
Absolute drawdown 3.30 Maximum drawdown 83.40 (4.44%) Relative drawdown 5.60% (64.50)
Total trades 21 Short positions (% win) 8 (75.00%) Long positions (% win) 13 (92.31%)
Profitable trades (% of all) 18 (85.71%) Loss trades (% of all) 3 (14.29%)
Largest profitable trade 124.84 losing deal -52.90
Average profitable deal 51.57 losing trade -25.77
Maximum number continuous wins (profit) 12 (637.50) Continuous losses (loss) 1 (-52.90)
Maximum Continuous Profit (number of wins) 637.50 (12) Continuous loss (number of losses) -52.90 (1)
Average continuous winnings 5 Continuous loss 1

The results are completely different, I can only explain this by the fact that the ticks are randomly generated in each variant and because of this the TS works differently, and what to do about it, how can anything be adjusted if the result of the run depends on random tick formation and will be different each time?

 

The quality should be in percentages. And you have n/a

 
OrlandoMagic писал(а) >>

The quality should be in percentages. And you have n/a.

It's not just the quality of the data that is the problem. I have TC running on ticks analysis and the fact that they are formed randomly leads to unstable operation of the system. I reloaded the history and run it with the same settings but the results are not stable.

Strategy Tester Report
Alpari-Demo (Build 225)


Symbol GBPUSD (Great Britain Pound vs US Dollar)
Period 5 Minutes (M5) 2009.08.03 00:00 - 2009.09.08 23:55 (2009.08.03 - 2009.09.09)
Model All ticks (most accurate method based on all smallest available timeframes)
Parameters Lot=0.1; StopLoss=0; TrailingStop=0;
Bars in history 8716 Modelled ticks 1466929 Modeling quality 90.00%
Plot mismatch errors 7
Initial deposit 1000.00
Net profit 792.14 Total profit 871.24 Total loss -79.10
Profitability 11.01 Expected payoff 37.72
Absolute drawdown 3.20 Maximum drawdown 77.70 (4.33%) Relative drawdown 5.59% (64.40)
Total trades 21 Short positions (% win) 8 (75.00%) Long positions (% win) 13 (84.62%)
Profitable trades (% of all) 17 (80.95%) Loss trades (% of all) 4 (19.05%)
Largest profitable trade 124.94 losing deal -52.80
Average profitable deal 51.25 losing trade -19.77
Maximum continuous wins (profit) 12 (620.00) Continuous losses (loss) 1 (-52.80)
Maximum Continuous Profit (number of wins) 620.00 (12) Continuous loss (number of losses) -52.80 (1)
Average continuous winnings 3 Continuous loss 1

 
But the results of the last two stats are very similar. There are EAs that produce different results every time, but this is very rare... Maybe it makes sense to move on? I think the profits are very good.
 
OrlandoMagic писал(а) >>
But the results of the last two stats are very similar. There are Expert Advisors that show different results every time, but they are very rare. Maybe it makes sense to move further. The profits seem to be very good.

Of course I'm not going to stop, but I don't want to fight with windmills, you try to achieve stability by debugging the system, but the problem is that the quotes are not stable (I mean not the stability of quotes formation in the tester), but not the TS, and it often takes a long time to understand what is wrong and who is to blame.

 

All the numbers we play with in the tester reports are very relative, just one losing trade, which may follow immediately after the test stops, is enough to completely distort the picture of profits, drawdowns, etc. And it is impossible to guarantee that the next trade will not be loss-making. That is why I ask experienced real traders what to focus on when setting up a system for working in a real account, what parameters to use when performing system debugging.

For example, here is another test to the above mentioned one with slightly changed parameters. Which variant is more preferable for real trading, with bigger profit and bigger drawdown or vice versa?

Strategy Tester Report
Alpari-Demo (Build 225)


Symbol GBPUSD (Great Britain Pound vs US Dollar)
Period 5 Minutes (M5) 2009.08.03 00:00 - 2009.09.08 23:55 (2009.08.03 - 2009.09.09)
Model All ticks (most accurate method based on all smallest available timeframes)
Parameters Lot=0.1; StopLoss=0; TrailingStop=0;
Bars in history 8716 Modelled ticks 1466929 Modeling quality 90.00%
Plot mismatch errors 7
Initial deposit 1000.00
Net profit 915.80 Total profit 994.90 Total loss -79.10
Profitability 12.58 Expected payoff 48.20
Absolute drawdown 3.50 Maximum drawdown 80.20 (6.30%) Relative drawdown 6.30% (80.20)
Total trades 19 Short positions (% win) 7 (57.14%) Long positions (% win) 12 (100.00%)
Profitable trades (% of all) 16 (84.21%) Loss trades (% of all) 3 (15.79%)
Largest profitable trade 139.31 losing deal -46.10
Average profitable deal 62.18 losing trade -26.37
Maximum continuous wins (profit) 11 (742.66) continuous losses (loss) 2 (-53.80)
Maximum Continuous Profit (number of wins) 742.66 (11) Continuous loss (number of losses) -53.80 (2)
Average continuous winnings 8 Continuous loss 2

 
Angela писал(а) >>

All the numbers we play with in the tester reports are very relative, just one losing trade, which may follow immediately after the test stops, is enough to completely distort the picture of profits, drawdowns, etc. And it is impossible to guarantee that the next trade will not be loss-making. That is why I ask experienced real traders what to focus on when setting up a system for working in real trading, what parameters to use when performing system debugging.

For example, here is another test to the above mentioned one with slightly changed parameters. Which variant is more preferable for real trading, with bigger profit and bigger drawdown or vice versa?

Strategy Tester Report

Total trades

19

let me reiterate - this number of trades is not suitable for analysis, it's like pointing your fingers at luck!

 
Move on, I mean optimise on single (large) periods, and do forwards. But optimise on all ticks. By the way, about divergences, it's good to find out what causes them, maybe it's a bug in the code after all?
 
xeon писал(а) >>

Once again, this number of trades is not suitable for analysis, it's like pointing a finger in the sky at luck!

Can you draw any conclusions based on this number? I cannot use a larger interval, I have no one-minute history. My TS is not optimized, I've given up optimizing it, it does not do me any good, but it spends too much time. I have adjusted my entries/exits using August's history (trades from 82 to 93). However I do not know what to do further, even 10% drawdown bothers me, and in this case it is more than 14%, if MM is used, it may increase many times.

Strategy Tester Report
Alpari-Demo (Build 225)


Symbol GBPUSD (Great Britain Pound vs US Dollar)
Period 5 Minutes (M5) 2009.05.18 00:00 - 2009.09.11 22:55 (2009.05.18 - 2009.09.12)
Model All ticks (most accurate method based on all smallest available timeframes)
Parameters Lot=0.1; StopLoss=0;TrailingStop=0;
Bars in history 25270 Modelled ticks 5461730 Modeling quality 89.91%
Chart mismatch errors 93
Initial deposit 1000.00
Net profit 1558.44 Total profit 2837.09 Total loss -1278.65
Profitability 2.22 Expected payoff 16.07
Absolute drawdown 97.63 Maximum drawdown 316.90 (14.21%) Relative drawdown 16.97% (184.40)
Total trades 97 Short positions (% win) 43 (60.47%) Long positions (% win) 54 (75.93%)
Profitable trades (% of all) 67 (69.07%) Loss trades (% of all) 30 (30.93%)
Largest profitable trade 252.40 losing transaction -56.61
Average profitable deal 42.34 losing trade -42.62
Maximum number continuous wins (profit) 7 (594.53) Continuous losses (loss) 3 (-121.00)
Maximum Continuous Profit (number of wins) 594.53 (7) Continuous loss (number of losses) -121.00 (3)
Average continuous winnings 3 Continuous loss 1