Time jumps in strategy tester, need help please!

New comment

juxeii 2011.03.19 03:45

Hello,

I'm currently writing an EA which works with trendlines.

The algorithm for these work now and I was trying to backtest the EA with the strategy tester.

However, it seems that there is either a huge bug in the tester or I have a wrong configuration of it.

I have two log files from the tester attached, where you can see what is going on.

The file 20110319.log.OK contains the log in which trading was disabled in the EA.

The file 20110319.log.NOK contains the log in which trading was enabled in the EA.

Here are the snippets of the first 23 log entries:

20110319.log.OK (no trades):

03:19:23 FireFX: loaded successfully
03:19:36 FireFX inputs: OverNightBeginTime=16; ObeyMarketCloseinMin=30; MARKET_CLOSE_TIME=23; NewsDistanceInMinutes=30; FSMLookbackBars=10; LotSize=0.1; FractionalRisk=0.01; AdjustFastSLRewardFactor=4; MaxDrawDownPercent=50; MaxOpenPositions=1;
03:19:36 2010.03.01 00:00 FireFX EURUSD,M15: Logging is disabled for current session.
03:19:36 2010.03.01 00:00 FireFX EURUSD,M15: itme 2010.02.26 22:45 distance 8
03:19:36 2010.03.01 00:00 FireFX EURUSD,M15: itme 2010.02.26 22:45 distance 2
03:19:36 2010.03.01 00:00 FireFX EURUSD,M15: oldestUp 8 oldestDown 2
03:19:36 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 205
03:19:36 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 201
03:19:36 2010.03.01 00:15 FireFX EURUSD,M15: oldestUp 0 oldestDown 0
03:19:36 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 205
03:19:36 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 201
03:19:36 2010.03.01 00:30 FireFX EURUSD,M15: itme 2010.03.01 00:15 distance 206
03:19:36 2010.03.01 00:30 FireFX EURUSD,M15: itme 2010.03.01 00:15 distance 202
03:19:36 2010.03.01 00:30 FireFX EURUSD,M15: itme 2010.03.01 00:15 distance 201
03:19:36 2010.03.01 00:30 FireFX EURUSD,M15: oldestUp 0 oldestDown 0
03:19:36 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 207
03:19:36 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 203
03:19:36 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 202
03:19:36 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 2
03:19:36 2010.03.01 00:45 FireFX EURUSD,M15: oldestUp 0 oldestDown 2
03:19:36 2010.03.01 01:00 FireFX EURUSD,M15: itme 2010.03.01 00:45 distance 208
03:19:36 2010.03.01 01:00 FireFX EURUSD,M15: itme 2010.03.01 00:45 distance 204
03:19:36 2010.03.01 01:00 FireFX EURUSD,M15: itme 2010.03.01 00:45 distance 203

And now the 20110319.log.NOK (with trades):

03:17:17 FireFX: loaded successfully
03:17:19 FireFX inputs: OverNightBeginTime=16; ObeyMarketCloseinMin=30; MARKET_CLOSE_TIME=23; NewsDistanceInMinutes=30; FSMLookbackBars=10; LotSize=0.1; FractionalRisk=0.01; AdjustFastSLRewardFactor=4; MaxDrawDownPercent=50; MaxOpenPositions=1;
03:17:19 2010.03.01 00:00 FireFX EURUSD,M15: Logging is disabled for current session.
03:17:19 2010.03.01 00:00 FireFX EURUSD,M15: itme 2010.02.26 22:45 distance 8
03:17:19 2010.03.01 00:00 FireFX EURUSD,M15: itme 2010.02.26 22:45 distance 2
03:17:19 2010.03.01 00:00 FireFX EURUSD,M15: oldestUp 8 oldestDown 2
03:17:19 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 205
03:17:19 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 201
03:17:19 2010.03.01 00:15 FireFX EURUSD,M15: oldestUp 0 oldestDown 0
03:17:19 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 205
03:17:19 2010.03.01 00:15 FireFX EURUSD,M15: itme 2010.03.01 00:00 distance 201
03:17:19 2010.03.01 00:15 FireFX EURUSD,M15: open #1 buy 0.90 EURUSD at 1.3632 sl: 1.3612 tp: 1.3772 ok
03:17:19 2010.03.01 00:30 FireFX EURUSD,M15: itme 2010.03.01 00:15 distance 206
03:17:19 2010.03.01 00:30 FireFX EURUSD,M15: itme 2010.03.01 00:15 distance 202
03:17:19 2010.03.01 00:30 FireFX EURUSD,M15: itme 2010.03.01 00:15 distance 201
03:17:19 2010.03.01 00:30 FireFX EURUSD,M15: oldestUp 0 oldestDown 0
03:17:19 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 207
03:17:19 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 203
03:17:19 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 202
03:17:19 2010.03.01 00:45 FireFX EURUSD,M15: itme 2010.03.01 00:30 distance 2
03:17:19 2010.03.01 00:45 FireFX EURUSD,M15: oldestUp 0 oldestDown 2
03:17:19 2010.03.01 03:03 Tester: stop loss #1 at 1.3612 (1.3612 / 1.3614)
03:17:19 2010.03.01 03:15 FireFX EURUSD,M15: itme 2010.03.01 03:00 distance 12

As you can see, there is huge time jump before the stop loss is triggered.

The tester jumps from 00:45 to 03:03 in time, whereas in the first log it does not.

You will much more examples of this beavior in the attached files.

What can I do about this?

The real problem is, that there are apperently data missing, so my trendline calculations are wrong, as you can see with my debug messages(distance values are different).

Can you help me, please?

Thanks in advance and good trades to you all :-)

William Roeder 2011.03.19 13:59 #1

Slow the tester down or look in the actual log file, not the tester window. When you generate lots of messages, the window sometimes skips them.
You say the first log is with the EA disabled, how can you test an EA that's disabled? There is no disable button in the tester. In addition, where are these messages coming from if the EA is disabled?

Questions from Beginners MQL5 Any rookie question, so problems testing strategy in

juxeii 2011.03.19 14:42 #2

WHRoeder:

Slow the tester down or look in the actual log file, not the tester window. When you generate lots of messages, the window sometimes skips them.
You say the first log is with the EA disabled, how can you test an EA that's disabled? There is no disable button in the tester. In addition, where are these messages coming from if the EA is disabled?

Hi WHRoeder,

thx for your answer :-)

1) These snippets above are the real logfiles. I could not figure out how to upload them here, sorry. I renamed the first log file to 20110319.log.OK and you can see the results.

2)The term "disabled" refers to the code of the EA. I made a comment before all lines beginning with OrderSend(...), so -> //OrderSend(...

Meanwhile I opened another demo account on Alpari UK.

Here there is exactly the same problem.

Here is a snippet of this log file:

14:27:40 FireFX: loaded successfully
14:27:48 FireFX inputs: OverNightBeginTime=16; ObeyMarketCloseinMin=30; MARKET_CLOSE_TIME=23; NewsDistanceInMinutes=30; FSMLookbackBars=10; LotSize=0.1; FractionalRisk=0.01; AdjustFastSLRewardFactor=6; MaxDrawDownPercent=50; MaxOpenPositions=1;
14:27:48 2010.01.04 00:00 FireFX EURUSD,M15: Logging is disabled for current session.
14:27:48 2010.01.04 00:00 FireFX EURUSD,M15: ask 1.4332
14:27:48 2010.01.04 00:15 FireFX EURUSD,M15: ask 1.4335
14:27:48 2010.01.04 00:15 FireFX EURUSD,M15: open #1 buy 0.70 EURUSD at 1.43354 sl: 1.43154 tp: 1.44754 ok
14:27:48 2010.01.04 00:30 FireFX EURUSD,M15: ask 1.433
14:27:48 2010.01.04 00:45 FireFX EURUSD,M15: ask 1.4334
14:27:48 2010.01.04 00:59 Tester: stop loss #1 at 1.43154 (1.43153 / 1.43214)
14:27:48 2010.01.04 01:00 FireFX EURUSD,M15: ask 1.4322
14:27:48 2010.01.04 01:15 FireFX EURUSD,M15: ask 1.4296
14:27:48 2010.01.04 01:30 FireFX EURUSD,M15: ask 1.4319
14:27:48 2010.01.04 01:45 FireFX EURUSD,M15: ask 1.4307
14:27:48 2010.01.04 01:45 FireFX EURUSD,M15: open #2 buy 0.70 EURUSD at 1.43073 sl: 1.42873 tp: 1.44473 ok
14:27:48 2010.01.04 02:00 FireFX EURUSD,M15: ask 1.4295
14:27:48 2010.01.04 02:00 Tester: stop loss #2 at 1.42873 (1.42868 / 1.42929)
14:27:48 2010.01.04 02:15 FireFX EURUSD,M15: ask 1.4289
14:27:48 2010.01.04 02:30 FireFX EURUSD,M15: ask 1.4286
14:27:48 2010.01.04 02:30 FireFX EURUSD,M15: open #3 buy 0.70 EURUSD at 1.42856 sl: 1.42656 tp: 1.44256 ok
14:27:48 2010.01.04 02:39 Tester: stop loss #3 at 1.42656 (1.42654 / 1.42715)
14:27:48 2010.01.04 02:45 FireFX EURUSD,M15: ask 1.4267
14:27:48 2010.01.04 03:00 FireFX EURUSD,M15: ask 1.4276
14:27:48 2010.01.04 03:00 FireFX EURUSD,M15: open #4 buy 0.70 EURUSD at 1.42763 sl: 1.42563 tp: 1.44163 ok
14:27:48 2010.01.04 03:15 FireFX EURUSD,M15: ask 1.4293
14:27:48 2010.01.04 03:30 FireFX EURUSD,M15: ask 1.4285
14:27:56 2010.01.04 10:59 FireFX EURUSD,M15: modify #4 buy 0.70 EURUSD at 1.42763 sl: 1.43768 tp: 1.44163 ok
14:27:56 2010.01.04 11:30 Tester: stop loss #4 at 1.43768 (1.43763 / 1.43824)
14:27:56 2010.01.04 11:45 FireFX EURUSD,M15: ask 1.4389
14:27:56 2010.01.04 11:45 FireFX EURUSD,M15: open #5 buy 0.70 EURUSD at 1.43889 sl: 1.43689 tp: 1.45289 ok
14:27:56 2010.01.04 12:00 FireFX EURUSD,M15: ask 1.4381
14:27:56 2010.01.04 12:15 FireFX EURUSD,M15: ask 1.4403
14:27:57 2010.01.05 19:32 Tester: stop loss #5 at 1.43689 (1.43685 / 1.43746)
14:27:57 2010.01.05 19:45 FireFX EURUSD,M15: ask 1.4361
14:27:57 2010.01.05 19:45 FireFX EURUSD,M15: open #6 buy 0.70 EURUSD at 1.43612 sl: 1.43412 tp: 1.45012 ok
14:27:57 2010.01.05 20:00 FireFX EURUSD,M15: ask 1.4369
14:27:57 2010.01.05 20:15 FireFX EURUSD,M15: ask 1.436
14:27:57 2010.01.06 01:39 Tester: stop loss #6 at 1.43412 (1.43408 / 1.43469)
14:27:57 2010.01.06 01:45 FireFX EURUSD,M15: ask 1.4344
14:27:57 2010.01.06 01:45 FireFX EURUSD,M15: open #7 buy 0.70 EURUSD at 1.43444 sl: 1.43244 tp: 1.44844 ok
14:27:57 2010.01.06 02:00 FireFX EURUSD,M15: ask 1.4353
14:27:57 2010.01.06 02:15 FireFX EURUSD,M15: ask 1.4355
14:27:57 2010.01.06 07:46 Tester: stop loss #7 at 1.43244 (1.43241 / 1.43302)
14:27:57 2010.01.06 08:00 FireFX EURUSD,M15: ask 1.4317
14:27:57 2010.01.06 08:15 FireFX EURUSD,M15: ask 1.4296
14:27:57 2010.01.06 08:30 FireFX EURUSD,M15: ask 1.4319
14:27:57 2010.01.06 08:45 FireFX EURUSD,M15: ask 1.4329

14:27:57 2010.01.06 08:45 FireFX EURUSD,M15: open #8 buy 0.70 EURUSD at 1.43289 sl: 1.43089 tp: 1.44689 ok

Again, you can see that the tester is jumping around in time!

I would expect that the tester would model all 15min bars, but thats not the case.

See for example:

14:27:57 2010.01.06 02:15 FireFX EURUSD,M15: ask 1.4355

14:27:57 2010.01.06 07:46 Tester: stop loss #7 at 1.43244 (1.43241 / 1.43302)

Here are 5 hours of data missing.

Again, the only difference is, if you open orders or let the tester run without orders.

Thats very bad behavior :-(

Have you any ideas?

Thx

BrainSystem: Trading System Development What is order_id ? Looking for an EA

Ickyrus 2011.03.19 14:57 #3

Your problem may lie in the genetic search. To find this open the stratergy tester and click on the expert properties button you should see

try unticking it and see if that helps.

juxeii 2011.03.19 15:44 #4

Sorry guys for the confusion, but I think I know now at least where the problem is located.

In my EA I check if a new Bar has formed, for the PERIOD specified:

Snip:

bool isNewBar()
{
  static int bars_total = 0;
  Print("bars_total ", bars_total, " iBars ", iBars(strSymbol, timeFrame));
  if(iBars(strSymbol, timeFrame) > bars_total)
  {
    bars_total = iBars(strSymbol, timeFrame);
    return(true);
  }
  return(false);
}

In the start function there is the check for it and a little debug message, to see if there is actualy a new bar:

if(isNewBar() == false)
{
   return(0);
}
Print(Ask);

In the log file you can see this:

15:20:21 2010.01.05 15:00 FireFX EURUSD,M15: bars_total 1157 iBars 1158 <-----
15:20:21 2010.01.05 15:00 FireFX EURUSD,M15: bars_total 1158 iBars 1158 !!!ERORR!!!!

15:20:21 2010.01.05 15:00 FireFX EURUSD,M15: bars_total 1158 iBars 1158

As you can see, the Print()-Message is not shown although there is a new bar!

On the other hand, sometimes the Print()-Message is shown correctly:

15:20:20 2010.01.04 02:15 FireFX EURUSD,M15: bars_total 1010 iBars 1011 <------
15:20:20 2010.01.04 02:15 FireFX EURUSD,M15: 1.4289 !!!CORRECT!!!!!

15:20:20 2010.01.04 02:15 FireFX EURUSD,M15: bars_total 1011 iBars 1011

I dont get this, why it is sometimes shown and why sometimes its not.

Definitely, this is the root cause of the time jumps, but I still don't know what to do about it.

Thx

Indicator won't fire alerts How to get an MQL4 first step

juxeii 2011.03.19 22:45 #5

Hi guys,

I could figure it out somehow.

If you copy the code of isNewBar() directly into the start() function, then there are no jumps anymore.

Very strange I know, but it does work.

I still believe this is a framework bug of MT4.

Anyway, thank you all and best regards :-)

Strategy I need help isNewBar across different timeframes IFxOverEasy1 system

New comment