Import of "minute bars" from forexite.com - script per MetaTrader 4

Sometimes the mql4.com quote archive lacks "small" data (for example, "minute bars"). For example, for gold: the minute bar history covers the period from 2006 to date. This is clearly not enough for a decent test of a "rough" strategy that works on medium and higher timeframes and which is not too sensitive to the differences in data from different dealers on the smaller timeframes. However, it is possible to use the quotes available on forexite.com, up to 2001 - the more so that this data has already earned quite a decent reputation.

This site has the "minute bar" quotes in the format more or less friendly to the MetaTrader 4: the MetaStock format. Unfortunately, the quotes for QuoteRoom have a more complicated format, even though they are collected into single files for each year. This script is designed for processing the data in the MetaStock format.

The problem is that the "minute bars" are collected in separate files by trading days. Besides, every file contains data not only on the required instrument (e.g. gold), but also on all other instrument traded by the dealer. And third: there are quote files for the weekends, which are usually unnecessary. Therefore, in order to import the required data on gold to the MetaTrader 4 history center, it is necessary to processed all these files somehow and to gather all "minute bars" in a single file.

The script performs this particular task. But first, the data needs to be prepared for processing:

1. Find a site download manager. The Teleport Pro program was used here. Even the unregistered demo version is sufficient for this purpose. The demo version limits the number of downloaded files per single project to 500. As one trading year, even with the holidays (forexite.com has trading days that are holidays), can not be greater than 365 days, this is enough for the purpose of script.

2. Go to the site section corresponding to the "minute bars" for the required years and download all daily archives. Separate project will need to be created per each year, but the files of different years can be stored in a single directory. Only the *.zip extension should be specified in the setting for the file download, as those are the only required files. This way it is possible to download approximately 1500 files for years 2002-2007.

3. Batch unpack all files to the "\experts\files\Forexite\" folder. Batch unpacking is built into the Explorer. The result is a list of files with the names like 250705.txt (quotes for July 25, 2005) in that folder.

4. Then open MetaTrader 4 and attach the script. There are thee external parameters:

a. name of the instrument as presented in the file from forexite.com. For gold it is «XAUUSD»,
b. the import start date as a string in the format «YYYY.MM.DD»,
c. the import end date as a string in the same format.

Run the script. The script does not process files corresponding to the holidays. By looking into the script execution log or in the «Experts» tab, it is possible to manually add the missing ones and run the script again. It was empirically found the earlier years usually do not contain the files for January 1 (New Year) and 25 December (Christmas). An updating line with the number of processed days is displayed at the top left corner, so as not to lose the sense of reality during the execution of the script (the processing of data from 2002 to 2007 to extract the quotes for gold lasted about 8 minutes).

5. There is large file in the "\experts\files\" folder with a name like XAUUSD_ALL.csv. This is the preliminary file ready for import (it was pleasant to see that MetaTrader 4 is able to distinguish dates in "different" format). It is enough to set the omission of the first column (as the first column in the created file is "XAUUSD", which is unnecessary), to specify the column separator (comma), to tick the Volume (if it is not) and also to specify the offset in hours. I did not have to specify the offset separately, as the quotes correspond to the server time of Alpari, which I work with.

After importing all the files in the directory "\experts\files\Forexite\", and the file received as a result of the script operation (XAUUSD_ALL.csv) can be deleted.

Finally: the minute bar quotes from forexite.com have no data on volumes. When importing to MetaTrader 4 the volumes are calculated if they have not been specified in the source data or if the volume data is contradictory, since the tester, as it turns out, does not test the data without volumes. If your strategy does not consider volumes and it is "rough" enough so that the different quotes from different dealers do not affect it, this is the right tool.

Note for the moderator (and others): for convenience of testing the operability of the script, attached is a small archive with the quotes of all symbols in the format of forexite.com for MetaStock from 01.02.2006 to 12.02.2006 (including holidays). Extract the archive to the "\experts\files\Forexite\" folder, set the external parameters of the script (instrument and both dates) and run the script. It is also possible to try the other instruments by simply changing the _sSymbol external parameter.