Large M1 data disappears from MT4 between re-launches, but .hst files all present
I wonder if anyone else has come across this. Is it a bug?
I'm able to import _SP500.csv which contains 30 years' worth of minute data successfully through History Center (for the method followed see https://forum.mql4.com/51224), and create all my .hst files for the market. I'm then able to run back tests using M1 and all other time frames created.
However, when I close and re-launch MT4, the M1 time frame data is gone, and has to be re-imported from _SP500.csv. BUT the _SP5001.hst file (the M1 history data file) is still present.
Through trial and error, I found that this does not happen if the M1 data is no more than around 750,000 lines long, which History Center says is 297899 records.
The work around is that I have to re-import each time I re-launch MT4, which is quick work. But its very annoying! :)
What do you have Tools>Options>Charts Max Bars in History set to ?
I followed the method in https://forum.mql4.com/51224, so 99999999999999999999 (2147483647)! Should be plenty! :)
I followed the method in https://forum.mql4.com/51224, so 99999999999999999999 (2147483647)! Should be plenty! :)
I've tried various installs:
- Windows XP Pro - C:\tester1
- Windows XP Pro - C:\program files\tester1
- Wine on Linux (CentOS 6)
I've been checking through the data, and have found something strange. I wonder if what I've found is at the root of this problem.
With the help of BREW_MissingData_sep.mq4 found on this site, I narrowed down where there seemed to be gaps in the data, the most recent being October 2012. I then examined the file on a Linux box with Joe text editor - its too big for anything I have under Windows. The gap in the raw data isn't there. So for some reason MT4 didn't import this part.
So I isolated the missing bit, and saved it to "oct15_31-2012.txt". I also reverse sorted it using Sed and created "oct15_31-2012-r.txt". I've attached the first hereto, and the second in the next comment.
I'm completely unable to see why this data won't import into MT4. It all looks fine to me. But I must be missing something, so I need all the help I can get! Sorry! :(
10/15/2012,0931,1430.81,1431.54,1430.81,1431.3,0
10/15/2012,0932,1431.49,1432.16,1431.49,1432.16,0
10/15/2012,0933,1432.68,1432.73,1432.37,1432.59,0
AUDJPY,20010102,230100,64.30,64.30,64.30,64.30,4
AUDJPY,20010102,230300,64.29,64.29,64.29,64.29,4
AUDJPY,20010102,230400,64.29,64.29,64.29,64.29,4
No volume.
You didn't check the volume button did you?
The data I got through Problem importing data for backtesting - MQL4 forum looks like
AUDJPY,20010102,230100,64.30,64.30,64.30,64.30,4
AUDJPY,20010102,230300,64.29,64.29,64.29,64.29,4
AUDJPY,20010102,230400,64.29,64.29,64.29,64.29,4
No volume.
You didn't check the volume button did you?
Was just about to type the same thing ;-)
Volume doesn't usually matter much, it doesn't actually mean Volume, it means tick count. It is needed for the Strategy Tester though, the Strategy Tester uses the "Volume" to determine how many ticks to synthesise when it creates the .fxt tick data file..
I'm extremely happy (not least relieved!) to be able to report that I've resolved this problem.
The raw minute data I have is formatted thus:
"Date","Time","Open","High","Low","Close","Volume"
06/28/2004,0931,37.49,37.50,37.45,37.46,1049200
Over almost all of the 30 years' worth, MT4 is happy to import this straight and without editing into its history for back testing. Almost! There must be something about certain dates it just can't cope with.
The answer is to reformat the whole 30 years' worth of data, or 2,824,276 records(!) thus:
"Date","Time","Open","High","Low","Close","Volume"
So, Sed to the rescue:2004/06/28,0931,37.49,37.50,37.45,37.46,1049200
$ sed -e 's_\(..\)\/\(..\)\/\(....\)_\3/\1/\2_' in_file.txt > out_file.txt
Goodness! What a steeeep learning curve I'm on..! Hope this info proves useful to someone else too!
Still, keeps me off the streets...
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I wonder if anyone else has come across this. Is it a bug?
I'm able to import _SP500.csv which contains 30 years' worth of minute data successfully through History Center (for the method followed see https://forum.mql4.com/51224), and create all my .hst files for the market. I'm then able to run back tests using M1 and all other time frames created.
However, when I close and re-launch MT4, the M1 time frame data is gone, and has to be re-imported from _SP500.csv. BUT the _SP5001.hst file (the M1 history data file) is still present.
Through trial and error, I found that this does not happen if the M1 data is no more than around 750,000 lines long, which History Center says is 297899 records.
The work around is that I have to re-import each time I re-launch MT4, which is quick work. But its very annoying! :)