Is the Strategy Tester limited how long time you can test in one go?

New comment
 

Hi,

I am currently running back tests in the Strategy Tester in MetaTrader 4 Build 225 (I have had too many crashes with build 226).

It seems like there is a limit for how long period I can test in one go. It seems to be somewhere between 1.5 and 2 years. if I set the date interval to test from Jan 2007 to Jan 2010, it might test from Jan 2007 to Jan 2009. There are no errors and it is not because i run out of balance, I am in profit when it stops. And there is nothing wrong with my more recent test data. Cause if I set the dates to Jul 2008 to Jan 2010 it will test all the way to Jan 2010.

I have downloaded M1 data from FXDD and then run period converted to get all other time frames. I have set Options/Charts/Max bars in history to max values.

Has anyone had this problem? Does anyone have an idea what could be wrong?

Thanks

Gosta

 

i made backtests from 2000 to 2010 so generally it works ;)

no clue what your problem could be...

one thing, if you have a old machine, like my old laptop with only one gb ram the tester can stop because he run's out of memory. check the journal, there should be a reason why the test was aborted

 
The Tester can handle FXT files of up to 2GB in size. If they are any larger, the testing would finish after 2GB of tick data... I don't know if that's your problem though. You can check the size of the *.FXT file in 'MetaTrader4\tester\history' folder during a test. If it's about 2GB in size, then you've hit this limit.
 
gordon:
The Tester can handle FXT files of up to 2GB in size. If they are any larger, the testing would finish after 2GB of tick data... I don't know if that's your problem though. You can check the size of the *.FXT file in 'MetaTrader4\tester\history' folder during a test. If it's about 2GB in size, then you've hit this limit.

That is correct, and it's because the MetaTrader developers used the int data type for use with the FXT file, whose maximum value is 2,147,483,647.

They should have used some other numerical data type (or used an unsigned int) that can store higher positive values, as it is obvious that this particular variable is needed to work with a large file.

e.g.

An unsigned int can hold a maximum value of 4,294,967,295 (because there is no need to accommodate for possibly negative values);

A long data type can hold a maximum value of 9,223,372,036,854,775,807;

An unsigned long can hold a maximum value of 18,446,744,073,709,551,615.


So it was a poor design decision by the developers to choose the int data type for use with the FXT file, when there are many other options.


This affects me because I backtest over real tick data with variable spread, instead of the usual fake tick data with fixed spread.

 

hyperdimension:

[...] because I backtest over real tick data with variable spread, instead of the usual fake tick data with fixed spread.

Variable spread...? AFAIK, using real tick data will not give u that option since the FXT file does not hold Bid/Ask prices. Ticks are modeled as bars, so spread is not involved. How exactly are u achieving this?
 
I know about the usual limitations, but it can be done in various ways if you know a bit of low-level computer stuff, e.g. patching the terminal or run-time memory.
 
hyperdimension:
I know about the usual limitations, but it can be done in various ways if you know a bit of low-level computer stuff, e.g. patching the terminal or run-time memory.
Patching the Terminal is required for disabling FXT file "recalculating" (which technically means bringing back the feature that was available in build 210 and prior, e.g. the "recalculate" option), but doing a patch to enable variable spread, a feature that was never there in the first place, that's a whole different story. Care to share?
 
It's possible to patch the terminal, but instead I fiddled with the memory. It took me quite a bit of time and effort to work it out.
 

Gordon, Hyperdimension,

Thanks for your input. This seems to be my problem. So how can I work around this? What data is stored in this file. How come Zzuegg managed to store 10 years of data in 2gb and I can't? Is there anything i can do to put less data into this file?

Right now I am testing GBPJPY on M30 for 5 years, and the FXT file size is 4.29 GB.

Thanks

 

for me the problem seems not to be the size of the FXT file but the trade history. clearly spoken, when working with market orders without sl/tp modifications the whole 10 year backtest goes nicly. but with pending orders, and tp/sl modifications it crashes after about 1000 trades.

there where 6 pending orders always on the chart, modifications where made every 5 minutes on all 6 orders. so i think its the tradehistory or the logfile which runs out of memory. btw, it's a notebook with 1gb ram.

10years backtest 600000 trading operations is no problem...

https://www.mql5.com/en/forum/126819

 
gostahulden:

[...] What data is stored in this file.

The interpolated Ticks that will be used in the simulation. Info on how these are calculated can be found in the article "Strategy Tester: Modes of Modeling during Testing". More detailed info can be found in the MQL5.com article "The Algorithm of Ticks’ Generation within the Strategy Tester of the MetaTrader 5 Terminal" (this might not be exactly the way it's calculated in MT4, but it still gives important insight...).

[...] How come Zzuegg managed to store 10 years of data in 2gb and I can't?

It has to do with the Volume information in your data, it must be much higher than in Zzueng's data. See this post -> https://www.mql5.com/en/forum/124870.

[...] Is there anything i can do to put less data into this file?

You can change Volume info, although this should be done with 'care'. See here -> https://www.mql5.com/en/forum/124870.

12
New comment