Strategy optimization result do not match single run test result - page 3

 
Alain Verleyen #:

Up to now, I have read a lot of affirmations and assumptions, but I have never seen anyone providing evidence of an MT5 bug on this issue.

I am using MT5 Strategy Tester for optimizations of complex EAs and I don't have any issue or discrepancy between optimization/single test. (Using local agents only).


If someone want to work seriously on this, I am ready to help. But it needs at least an ex5 to reproduce the issue, the tester settings and all detailed informations needed.

And secondly, if the issue is happening, it can only be investigated from the EA source code, because there is NO general issue (it doesn't happen with all EAs).

Hey Alain,

I went ahead and disabled the local network farm and only using local cores and it appears that all the forwards tests are now accurate again.

I saw someone posted their EX5 code and was able to replicate the issue. I will reach out to you and provide all the details you require. 

 
Audi #:

Hey Alain,

I went ahead and disabled the local network farm and only using local cores and it appears that all the forwards tests are now accurate again.

I saw someone posted their EX5 code and was able to replicate the issue. I will reach out to you and provide all the details you require. 

Ok.
 
Alain Verleyen #:

Up to now, I have read a lot of affirmations and assumptions, but I have never seen anyone providing evidence of an MT5 bug on this issue.

I am using MT5 Strategy Tester for optimizations of complex EAs and I don't have any issue or discrepancy between optimization/single test. (Using local agents only).


If someone want to work seriously on this, I am ready to help. But it needs at least an ex5 to reproduce the issue, the tester settings and all detailed informations needed.

And secondly, if the issue is happening, it can only be investigated from the EA source code, because there is NO general issue (it doesn't happen with all EAs).

Hi, I've prepared a package containing an EA that triggers the problem, I've included the setfile and a snapshot of the settings, optimized params are in the setfile.

I'm using tickstory data but it should not make any difference, I had the problem with broker data as well.

It's not necessary to run the whole optimization (it's quite lengthy), after a few results it can be stopped and a single test run.

In particular I've tested this result:

OpenBuyHour: 1

CloseBuyHour: 18

which reports a profit in the optimization and a loss in the single test run.

Hope this helps.


Files:
 
Francesco Baldi #:

Hi, I've prepared a package containing an EA that triggers the problem, I've included the setfile and a snapshot of the settings, optimized params are in the setfile.

I'm using tickstory data but it should not make any difference, I had the problem with broker data as well.

It's not necessary to run the whole optimization (it's quite lengthy), after a few results it can be stopped and a single test run.

In particular I've tested this result:

OpenBuyHour: 1

CloseBuyHour: 18

which reports a profit in the optimization and a loss in the single test run.

Hope this helps.


Thanks I will check.
 
Francesco Baldi #:

Hi, I've prepared a package containing an EA that triggers the problem, I've included the setfile and a snapshot of the settings, optimized params are in the setfile.

I'm using tickstory data but it should not make any difference, I had the problem with broker data as well.

It's not necessary to run the whole optimization (it's quite lengthy), after a few results it can be stopped and a single test run.

In particular I've tested this result:

OpenBuyHour: 1

CloseBuyHour: 18

which reports a profit in the optimization and a loss in the single test run.

Hope this helps.


Hey Franceso,


I'm curious, are you also using a Local Network Farm, or are you only strictly using your Local CPU cores? 

 
Francesco Baldi #:

Hi, I've prepared a package containing an EA that triggers the problem, I've included the setfile and a snapshot of the settings, optimized params are in the setfile.

I'm using tickstory data but it should not make any difference, I had the problem with broker data as well.

It's not necessary to run the whole optimization (it's quite lengthy), after a few results it can be stopped and a single test run.

In particular I've tested this result:

OpenBuyHour: 1

CloseBuyHour: 18

which reports a profit in the optimization and a loss in the single test run.

Hope this helps.


It's running but the below message in the log could be an issue.

2024.06.14 22:11:46.548    SmartDayTraderMT5    tester_file 'MQL5\Files\News_history.json' is not exist

 

It's running but the below message in the log could be an issue.

2024.06.14 22:11:46.548    SmartDayTraderMT5    tester_file 'MQL5\Files\News_history.json' is not exist

 
Alain Verleyen #:

It's running but the below message in the log could be an issue.

2024.06.14 22:11:46.548    SmartDayTraderMT5    tester_file 'MQL5\Files\News_history.json' is not exist

Hi, I've done further research.

The latest working build is 4232.

The discrepancy was introduced with build 4233.


There's another strange thing: after I run the optimization on one of the 'working' builds, it happens sometimes that launching a single test pass fails with the following errors:

HM 0 10:06:34.097 Core 1 2024.01.02 00:00:18   Placing Buy USDJPY Market Order at price 140.896 (140.896), stop loss: 0, take profit: 0

JG 2 10:06:34.097 Core 1 2024.01.02 00:00:19   failed market buy 0.01 USDJPY [Market closed]

CM 2 10:06:34.097 Core 1 2024.01.02 00:00:19   runtime error

GH 2 10:06:34.097 Core 1 OnTester critical error


If just I run it again, it works and completes correctly.

If it happens, it happens only on the first single run test after the optimization.

Although this might be an uncorrelated bug, I thought that this critical error might be happening during optimization and preventing the faulty passes to complete.


I think there's now enough information to hand over this issue to the development team, along with the sample ea that I sent before.

I hope this issue will be taken into account and addressed.

 

The forum lost my posts

You are right with older build, all passes are fine, I checked.

I had found that the passes which provide wrong optimizations results, when run with "Single Test" have these errors :

All of these passes, and only these passes. The "good" passes doesn't have these errors.

Details of the log when this error occurs :


 In older build, these errors happens too. But it doesn't affect the optimization.

That means that something changed in MT5 which makes these passes to fail when running as "optimization". I will report it to MetaQuotes attention.

Though by fixing these errors in your code you can probably workaround this issue.