MetaTrader 5 / Examples
Русский 中文 Español Deutsch 日本語 Português
MQL5 Programming Basics: Files

MQL5 Programming Basics: Files

MetaTrader 5 — Examples | 4 November 2016, 15:19
54 040 3
Dmitry Fedoseev
Dmitry Fedoseev

Contents

Introduction

Like many other programming languages, MQL5 features the functions for working with files. Although working with files is not a very common task when developing MQL5 Expert Advisors and indicators, each developer faces it sooner or later. The range of issues that require working with files is wide enough. It includes generating a custom trading report, creating special files with complex parameters for an EA or indicator, reading market data (for example, a news calendar), etc. This article covers all functions for working with files in MQL5. Each of them is accompanied by a simple practical task aimed at honing your skills. Apart from tasks, the article considers a number of useful functions that can be applied in practice.

MQL5 documentation contains description of file functions here.  

Reading a text file

The simplest and most frequently used function is reading text files. Let's jump straight to practice. Open MetaEditor. Select File — Open Data Folder. Open MQL5 folder in the new window. After that, open Files folder. This folder is the one that contains files available for processing by the MQL5 file functions. That restriction ensures data security. MetaTrader users actively share MQL5 applications. Without the restriction, it would have been too easy for intruders to cause harm to your PC by deleting or corrupting important files or stealing personal data.

Create a text file in the newly opened MQL5/Files folder. To do this, click somewhere within the folder and select New — Text Document. Name the file "test". Its full name should be "test.txt". I recommend that you enable the display of file extensions on your PC.

After re-naming the file, open it. It opens in Notepad editor. Write 2-3 text lines to the file and save it. Make sure that ANSI encoding is selected in the drop-down list at the bottom of the Save As window (Fig. 1).


Fig. 1. Saving a text file in Windows Notepad. The red arrow shows the selected file encoding 

Now, we are going to read the file by means of MQL5. Create a script in MetaEditor and name it sTestFileRead.

The file should be opened before reading it or writing to it and closed afterwards. The file is opened by the FileOpen() function having two mandatory parameters. The first one is the file name. We should specify "test.txt" here. Please note that we specify the path from the MQL5/Files folder and not the full path. The second parameter is the combination of flags defining the mode of working with files. We are going to read the file, therefore we should specify the FILE_READ flag. "test.txt" is a text file applying ANSI encoding, which means we should use two more flags: FILE_TXT and FILE_ANSI. The flags are combined via the "or" logical operation indicated by "|" symbol.

The FileOpen() function returns the file handle. We will not go into details about the handle's function. Let's just say that it is a numeric value (int) to be used instead of the file's string name. The file's string name is specified when opening a file, while the handle is used afterwards to perform actions with that file.

Let's open the file (write the code in the OnStart() function of the sTestFileRead script):

int h=FileOpen("test.txt",FILE_READ|FILE_ANSI|FILE_TXT);

After that, make sure the file is actually opened. This is done by checking the value of the received handle:

if(h==INVALID_HANDLE){
   Alert("Error opening file");
   return; 
}

The file opening error is quite common. If the file is already opened, it cannot be opened the second time. The file may be opened in some third-party application. For example, the file may be opened in Windows Notepad and MQL5 simultaneously. But if it is opened in Microsoft Excel, then it can be opened anywhere else.  

Reading data from a text file (opened with the FILE_TXT flag) is done by the FileReadString() function. Reading is performed line by line. One function call reads a single file line. Let's read one line and display it in the message box:

string str=FileReadString(h);
Alert(str);

Close the file:

FileClose(h);

Please note that calling the FileReadString() and FileClose() functions is performed by specifying the handle (h variable) received when opening the file by the FileOpen() function.

Now, you can execute the sTestFileRead script. If something goes wrong, compare your code with the sTestFileRead file attached below. The window with the first line from the "test.txt" file should appear as a result of the script operation (Fig. 2).

 
Fig. 2. The sTestFileRead script operation result

We have read only one line from the "test.txt" file so far. In order to read the remaining two ones, we may call the FileReadString() function two more times but in actual practice the number of file lines is usually not known in advance. To solve this issue, we should apply the FileIsEnding() function and the while operator. If we reach the end of a file while reading it, the FileIsEnding() function returns 'true'. Let's write a custom function for reading all file lines and displaying them in the message box using the FileIsEnding() function. It can be useful for a variety of educational experiments concerning working with files. We obtain the following function:

void ReadFileToAlert(string FileName){
   int h=FileOpen(FileName,FILE_READ|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   Alert("=== Start ===");   
   while(!FileIsEnding(h)){
      string str=FileReadString(h);
      Alert(str);   
   }
   FileClose(h);

 Let's create the sTestFileReadToAlert script, copy the function to it and call it from the script's OnStart() function:

void OnStart(){
   ReadFileToAlert("test.txt");
}

The message box containing the "=== Start ===" line and all three lines of the "test.txt" file appears. The file is now read completely (Fig. 3). 


Fig. 3. We have applied the FileIsEnding() function and the 'do while' loop to read the entire file   

Creating a text file

In order to create a file, open it using the FileOpen() function. Open the "test.txt" file using the FILE_READ flag instead of the FILE_WRITE one:

int h=FileOpen("test.txt",FILE_WRITE|FILE_ANSI|FILE_TXT);

After opening the file, make sure to check the handle just like when reading the file. If the function is executed successfully, the new "test.txt" file is created. If the file already exists, it is completely cleared. Be careful when opening files for writing, do not lose valuable data.  

Writing to the text file is performed by the FileWrite() function. The first parameter sets the file handle, while the second one sets the line written to the file. The new line is written at each call of the FileWrite() function.

Let's write to the file ten lines in a loop. The final script code (sTestFileCreate) looks as follows:

void OnStart(){
   int h=FileOpen("test.txt",FILE_WRITE|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }
   for(int i=1;i<=10;i++){
      FileWrite(h,"Line-"+IntegerToString(i));
   }
   FileClose(h);
   Alert("File created");
}

After executing the code, the "test.txt" file should contain ten lines. To check the file contents, open it in Notepad or execute the sTestFileReadToAlert script.

Note the FileWrite() function. It may have more than two arguments. You may pass multiple string variables to the function, and they are combined into one line when writing. In the specified code, the call of the FileWrite() function can be written as follows:

FileWrite(h,"Line-",IntegerToString(i));

The function will automatically combine the lines when writing.

Writing to the end of a text file

Sometimes, it is necessary to add one or several new text lines to the existing file leaving the rest of the contents intact. To perform such actions, the file should be opened for reading and writing simultaneously. This means the both flags (FILE_READ and FILE_WRITE) should be specified when calling the FileOpen() function.

int h=FileOpen("test.txt",FILE_READ|FILE_WRITE|FILE_ANSI|FILE_TXT);

If the file with the specified name does not exist, it is created. If it already exists, it is opened while its contents is kept intact. However, if we start writing to the file at once, its previous contents is deleted since the writing is performed from the beginning of the file.

When working with files, there is such a thing as a "pointer" — numerical value indicating the position, from which the next entry or reading from the file is performed. When opening the file, the pointer is automatically set at the beginning of the file. During the reading or data writing, it is automatically relocated by the size of read or written data. You can relocate the pointer yourself if necessary. To do this, use the FileSeek() function.  

In order to save the previous contents and add the new one to the end of the file, relocate the pointer to the end of the file before writing:

FileSeek(h,0,SEEK_END);

The three parameters are sent to the FileSeek() function: handle, pointer relocation value and position the shift is calculated from. In this example, the SEEK_END constant means the end of the file. Thus, the pointer is shifted by 0 bytes from the end of the file (meaning shifting to its very end).

The final script code for adding to the file is as follows:

void OnStart(){
   int h=FileOpen("test.txt",FILE_READ|FILE_WRITE|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }
   FileSeek(h,0,SEEK_END);
   FileWrite(h,"Additional line");
   FileClose(h);
   Alert("Added to file");
}

This script is also attached below (sTestFileAddToFile). Launch the script and check the contents of the test.txt file. Each call of the sTestFileAddToFile script adds one line to test.txt.

Changing the specified line of a text file

In case of text files, the ability to freely move the pointer all over the file can be used only to make additions to the file and is not applicable for making changes. It is impossible to make changes in a certain file line, since a file line is just a concept as the file actually contains a continuous series of data. Sometimes, such series contain special characters invisible in a text editor. They point out that the following information should be displayed in a new line. If we set the pointer at the beginning of the line and start writing, previous data in the line will remain if the size of written data is less than the existing line. Otherwise, new line symbols will be deleted together with a part of the next line data.

Let's try to replace the second line in test.txt. Open the file for reading and writing, read one line to relocate the pointer to the start of the second line and write a new line consisting of two letters "AB" (sTestFileChangeLine2-1 script attached below):

void OnStart(){
   int h=FileOpen("test.txt",FILE_READ|FILE_WRITE|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }
   string str=FileReadString(h);
   FileWrite(h,"AB");
   FileClose(h);
   Alert("Done");
} 

The obtained test.txt file now looks as follows (Fig. 4):

 
Fig. 4. The contents of the text file after an attempt to change one line was made 

Now, we have two lines instead of one: "AB" and "-2". "-2" is what remains from the second line where four characters have been deleted. The reason is that when writing a line using the FileWrite() function, it adds new line symbols to the end of the written text. In Windows operating systems, the new line symbol consists of two characters. If we add to them two characters in "AB" line, we can understand why four characters were deleted in the resulting file.  

Execute the sTestFileCreate script to restore test.txt and try replacing the second line with a longer one. Let's write "Line-12345" (sTestFileChangeLine2-2 script attached below):

void OnStart(){
   int h=FileOpen("test.txt",FILE_READ|FILE_WRITE|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }
   string str=FileReadString(h);
   FileWrite(h,"Line-12345");
   FileClose(h);
   Alert("Done");
}

Let's see the resulting file (Fig. 5):

 
Fig. 5. Results of the second attempt to change the single line of the text file 

Since the new line is longer than the previous one, the third line has been affected as well.  

The only way to make changes to text files is reading and re-writing them in full. We should read the file to the array, make changes to the necessary array elements, save it line by line to another file, delete the old file and rename the new one. Sometimes, the array is not needed: while reading the lines from one file, they can be written to another. At a certain point in time, changes are made and saved in the necessary line. After that, the old file is deleted and the new one is renamed.

Let's apply the latter option (changing without the array). First, we should create a temporary file. Let's write the function for receiving the unique name of the temporary file. The file name and extension are passed to the function. The check whether the file exists is performed (by the standard FileIsExists() function) within the function itself. If the file exists, a number is added to it till no file with such name is detected. The function looks as follows:

string TmpFileName(string Name,string Ext){
   string fn=Name+"."+Ext; // forming name
   int n=0;
   while(FileIsExist(fn)){ // if the file exists
      n++;
      fn=Name+IntegerToString(n)+"."+Ext; // add a number to the name
   }
   return(fn);
}

Let's create the sTestFileChangeLine2-3 script, copy the function to it and place the following code in the OnStart() function.

Open test.txt for reading:

int h=FileOpen("test.txt",FILE_READ|FILE_ANSI|FILE_TXT);

Receive the name of the temporary file and open it:

string tmpName=TmpFileName("test","txt");

int tmph=FileOpen(tmpName,FILE_WRITE|FILE_ANSI|FILE_TXT);

Read the file line by line counting them. All read lines are sent to the temporary file, and the second line is replaced:

   int cnt=0;
   while(!FileIsEnding(h)){
      cnt++;
      string str=FileReadString(h);
      if(cnt==2){
         // replace the line
         FileWrite(tmph,"New line-2");
      }
      else{
         // rewrite the line with no changes
         FileWrite(tmph,str);
      }
   }

Close both files:

FileClose(tmph);
FileClose(h);

Now, all we have to do is delete the original file and rename the temporary one. The standard FileDelete() function is used for deletion.

FileDelete("test.txt");

To rename the file, we should use the standard FileMove() function designed to move or rename files. The function receives the four mandatory parameters: name of the relocated file (source file), file location flag, new file name (destination flag), overwrite flag. It should be all clear with file names, so now it is time to have a closer look at the second and fourth parameters — flags. The second parameter defines the location of the source file. Files available for processing in MQL5 can be located not only in MQL5/Files folder of the terminal but also in the common folder of all terminals. We will consider this in more detail later. For now, let's just set 0. The last parameter defines the destination file location. It also may have an additional flag defining actions if the target file exists. Since we have deleted the source file (destination file), the fourth parameter is set to 0:

FileMove(tmpName,0,"test.txt",0);

Before executing the sTestFileChangeLine2-3 script, restore test.txt using the sTestFileCreate script. After the sTestFileChangeLine2-3 script operation, text.txt should have the following contents (Fig. 6):

 
Fig. 6. The file contents after replacing the line

Let's return to the FileMove() function. If we set the FILE_REWRITE flag (allowing us to rewrite the destination file) as the fourth parameter:

FileMove(tmpName,0,"test.txt",FILE_REWRITE);

it is not necessary to delete the source file from the script. This option is used in the sTestFileChangeLine2-3 script attached below. 

Instead of the FileMove() function, we may use another standard function FileCopy(), but in this case, we need to delete the temporary file:

FileCopy(tmpName,0,"test.txt",FILE_REWRITE);
FileDelete(tmpName);

Reading a text file to an array

One useful function has already been described in this article (receiving an unoccupied file name). Now, let's develop yet another function that is often used when working with files — reading a file to an array. The file name and the line array are passed to the function. The array is passed via the link and filled with the file contents in the function. The function returns true/false depending on its operation results. 

bool ReadFileToArray(string FileName,string & Lines[]){
   ResetLastError();
   int h=FileOpen(FileName,FILE_READ|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      int ErrNum=GetLastError();
      printf("Error opening file %s # %i",FileName,ErrNum);
      return(false);
   }
   int cnt=0; // use the variable to count the number of file lines
   while(!FileIsEnding(h)){
      string str=FileReadString(h); // read the next line from the file
      // remove spaces to the left and to the right to detect and avoid using empty lines
      StringTrimLeft(str); 
      StringTrimRight(str);
      if(str!=""){ 
         if(cnt>=ArraySize(Lines)){ // array filled completely
            ArrayResize(Lines,ArraySize(Lines)+1024); // increase the array size by 1024 elements
         }
         Lines[cnt]=str; // send the read line to the array
         cnt++; // increase the counter of read lines
      }
   }
   ArrayResize(Lines,cnt);
   FileClose(h);
   return(true);
}

We will not consider this function in details, since all should already be clear from the data provided here so far. Besides, it has already been commented in details. We should only mention some nuances. After reading the line from the file to the str variable, spaces at the line edges are deleted by the StringTrimLeft() and StringTrimRight() functions. Then, the check is performed on whether the str string is not empty. This is done in order to skip unnecessary empty lines. While the array is filled, it is increased in blocks by 1024 elements rather than by a single one. The function works much faster this way. Finally, the array is scaled according to the actual number of read lines.

The function can be found in the sTestFileReadFileToArray script attached below.

Creating a text file with separators

So far, we have considered only simple text files. However, there is another kind of text files — the ones with separators. Usually, they have .csv extension (short for "comma separated values"). In fact, these are plain text files that can be opened in text editors, as well as read and edited manually. A certain character (not necessarily a comma) is used as a field separator in lines. Thus, you can perform some different actions with them as compared to simple text files. The main difference is that in a simple text file, an entire line is read when the FileRedaString() function is called, while in files with separators, reading is performed up to a separator or end of line. The FileWrite() function also works differently: all written variables enumerated in the function are not simply connected to one line. Instead, a separator is added between them. 

Let's try to create a csv file. Open the text file like we have already done for writing specifying the FILE_CSV flag instead of the FILE_TXT one. The third parameter is a symbol used as a separator:

int h=FileOpen("test.csv",FILE_WRITE|FILE_ANSI|FILE_CSV,";");

Let's write to the file ten lines with three fields per line:

   for(int i=1;i<=10;i++){
      string str="Line-"+IntegerToString(i)+"-";
      FileWrite(h,str+"1",str+"2",str+"3");
   }

Make sure to close the file in the end. The code can be found in the sTestFileCreateCSV script attached below. "test.csv" file is created as a result. The file contents is shown in Fig. 7. As we can see, the FileWrite() function parameters now form a single line with a separator between them.

 
Fig. 7. Contents of a file with separators

Reading a text file with separators

Now, let's try to read the csv file the same way as the text file at the very beginning of this article. Let's make a copy of the sTestFileReadToAlert script named sTestFileReadToAlertCSV. Change the first string in the ReadFileToAlert() function: 

int h=FileOpen(FileName,FILE_READ|FILE_ANSI|FILE_CSV,";");

Rename the ReadFileToAlert() function into ReadFileToAlertCSV() and change the name of the file passed to the function:

void OnStart(){
   ReadFileToAlertCSV("test.csv");
}

The script operation result shows that the file has been read by one field. It would be good to determine when the fields of one line are read and the new line starts. The FileIsLineEnding() function is applied for that.

Let's make a copy of the sTestFileReadToAlertCSV script named sTestFileReadToAlertCSV2, rename the ReadFileToAlertCSV function into ReadFileToAlertCSV2 and change it. Add the FileIsLineEnding() function: if it returns 'true', display the dividing line "---". 

void ReadFileToAlertCSV2(string FileName){
   int h=FileOpen(FileName,FILE_READ|FILE_ANSI|FILE_CSV,";");
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   Alert("=== Start ===");   
   while(!FileIsEnding(h)){
      string str=FileReadString(h);
      Alert(str);
      if(FileIsLineEnding(h)){
         Alert("---");
      }
   }
   FileClose(h);
}

 Now, the fields sent to the message window by the script are divided into groups (Fig. 8).


Fig. 8. "---" separators between field groups of a single file line 

Reading a file with separators to an array

Now that we familiarized ourselves with working with csv files, let's develop yet another useful function for reading a csv file to an array. Reading is performed to a structure array where each element corresponds to one file line. The structure will contain a line array with every of its elements corresponding to a single line field. 

Structure:

struct SLine{
   string line[];
};  

Function:

bool ReadFileToArrayCSV(string FileName,SLine & Lines[]){
   ResetLastError();
   int h=FileOpen(FileName,FILE_READ|FILE_ANSI|FILE_CSV,";");
   if(h==INVALID_HANDLE){
      int ErrNum=GetLastError();
      printf("Error opening file %s # %i",FileName,ErrNum);
      return(false);
   }   
   int lcnt=0; // variable for calculating lines 
   int fcnt=0; // variable for calculating line fields    
   while(!FileIsEnding(h)){
      string str=FileReadString(h);
      // new line (new element of the structure array)
      if(lcnt>=ArraySize(Lines)){ // structure array completely filled
         ArrayResize(Lines,ArraySize(Lines)+1024); // increase the array size by 1024 elements
      }
      ArrayResize(Lines[lcnt].field,64);// change the array size in the structure
      Lines[lcnt].field[0]=str; // assign the first field value
      // start reading other fields in the line
      fcnt=1; // till one element in the line array is occupied
         while(!FileIsLineEnding(h)){ // read the rest of fields in the line
            str=FileReadString(h);
            if(fcnt>=ArraySize(Lines[lcnt].field)){ // field array is completely filled
               ArrayResize(Lines[lcnt].field,ArraySize(Lines[lcnt].field)+64); // increase the array size by 64 elements
            }     
            Lines[lcnt].field[fcnt]=str; // assign the value of the next field
            fcnt++; // increase the line counter
         }
      ArrayResize(Lines[lcnt].field,fcnt); // change the size of the field array according to the actual number of fields
      lcnt++; // increase the line counter
   }
   ArrayResize(Lines,lcnt); // change the array of structures (lines) according to the actual number of lines
   FileClose(h);
   return(true);
}

We will not consider this function in details dwelling only on the most critical points. One field is read at the beginning of the while(!FileIsEnding(h)) loop. Here we find out about an element to be added to the structure array. Check the array size and increase it by 1024 elements if necessary. Change the field array size at once. A size of 64 elements is set for it immediately, and the value of the first line field read from the file is assigned to the element with 0 index. After that, read the remaining fields in the while(!FileIsLineEnding(h)) loop. After reading another field, check the array size and increase it if necessary and send the line read from the file to the array. After reading the line to the end (exiting the while(!FileIsLineEnding(h)) loop), change the field array size according to their actual number. At the very end, resize the line array according to the actual number of read lines. 

The function can be found in the sTestFileReadFileToArrayCSV script attached below. The script reads the test.csv file to the array and displays the array in the message window. The result is the same as the one shown in Fig. 8. 

Writing an array to a text file with separators

The task is quite simple if the number of fields in the line is known in advance. A similar task has already been solved in the "Creating a text file with separators" section. If the number of fields is unknown, all fields can be gathered into a single line with separators in a loop, while the line is then written to the file opened with the FILE_TXT flag.

Open the file: 

int h=FileOpen("test.csv",FILE_WRITE|FILE_ANSI|FILE_TXT);

Gather all fields (array elements) into a single line using a separator. There should be no separator at the end of the line, otherwise there will be a redundant empty field in the line:

   string str="";
   int size=ArraySize(a);
   if(size>0){
      str=a[0];
      for(int i=1;i<size;i++){
         str=str+";"+a[i]; // merge fields using a separator 
      }
   }

Write the line to the file and close it:  

FileWriteString(h,str);
FileClose(h);

This example can be found in the sTestFileWriteArrayToFileCSV script attached below.

UNICODE files

Until now, the FILE_ANSI flag was always specified when opening files to define their encoding. In this encoding, one character corresponds to one byte, therefore, the entire set is limited to 256 symbols. However, the UNICODE encoding is widely used nowadays. In this encoding, one symbol is defined by several bytes and a text file may contain a very large number of characters, including letters from different alphabets, hieroglyphics and other graphic symbols.

Let's perform some experiments. Open the sTestFileReadToAlert script in the editor, save it under the sTestFileReadToAlertUTF name and replace the FILE_ANSI flag with the FILE_UNICODE one: 

int h=FileOpen(FileName,FILE_READ|FILE_UNICODE|FILE_TXT);

Since test.txt is saved in ANSI, the new window contains garbled text (Fig. 9).

  
Fig. 9. Garbled text that can be seen when a file's original encoding does not match the one specified when opening a file

Apparently, this happens because the file's original encoding does not match the one specified when opening a file.

Open the sTestFileCreate script in the editor, save it under the sTestFileCreateUTF name and replace the FILE_ANSI flag with the FILE_UNICODE one:

int h=FileOpen("test.txt",FILE_WRITE|FILE_UNICODE|FILE_TXT);

Launch the sTestFileCreateUTF script to create a new test.txt file. Now, the sTestFileReadToAlertUTF display the readable text (Fig. 10).

 
Fig. 10. Using the sTestFileReadToAlertUTF script to read the file generated by the sTestFileCreateUTF script

Open test.txt in Notepad and execute "Save As..." command in the main menu. Note that Unicode is selected in the Encoding list at the bottom of the "Save As" window. Notepad has somehow defined the file encoding. Unicode files start with the standard set of symbols, the so-called BOM (byte order mark). Later, we will get back to this and write the function for defining a text file type (ANSI or UNCODE). 

Additional functions for working with text files having separators

From the variety of file functions for working with the contents of text files (both simple ones and the ones with separators), we actually need only two ones: FileWrite() and FileReadString(). Apart from other things, the FileReadString() function is also used for working with binary files (more on them below). Apart from the FileWrite() function, the FileWriteString() function can be used, though that is not critical. 

When working with text files having separators, a few other functions can be used making work more convenient: FileReadBool(), FileReadNumber() and FileReadDatetime(). The FileReadNumber() function is used to read the numbers. If we know in advance that the field read from a file contains only a number, we may apply this function. Its effect is identical to reading a line with the FileReadString() function and converting it into a number with the StringToDouble() function. Similarly, the FileReadBool() function is used to read bool-type values. The string may contain true/false or 0/1. The FileReadDatetime() function is used to read the data in line format and convert it to a numeric datetime-type value. Its effect is similar to reading a line and converting it using the StringToTime() function.  

Binary files

The text files discussed earlier are rather convenient since their contents read by program means is consistent with what you see in the file when you open it in a text editor. You can easily manage the program operation results by examining a file in the editor. If necessary, the file can be changed manually. The drawbacks of text files include the limited options when working with them (this is evident if we think of the difficulties we faced when replacing a single file line).

If a text file is small, it is quite comfortable in use. But the larger its size, the more time it takes to work with it. If you need to quickly process large volumes of data, use binary files.

When opening a file in binary mode, the FILE_BIN flag is specified instead of FILE_TXT or FILE_CSV. There is no point in specifying FILE_ANSI or FILE_UNCODE encoding files, since the binary file is a file with numbers.

Of course, we can have a look at the binary file contents in Notepad text editor. Sometimes, we even can see letters and a readable text there but this has more to do with Notepad itself rather than the file contents.

Anyway, you definitely should not edit the file in a text editor because it becomes damaged in the process. We will not go into details about the causes, let's just accept that fact. Of course, there are special binary file editors but the editing process is still not intuitive.

Binary files, variables

Most functions for working with files in MQL5 are designed for the binary mode. There are functions for reading/writing the variables of different types:  

FileReadDouble() FileWriteDouble()
FileReadFloat() FileWriteFloat()
FileReadInteger() FileWriteInteger()
FileReadLong() FileWriteLong()
FileReadString() FileWriteString()
FileReadStruct() FileWriteStruct()

We will not describe all variable writing/reading functions here. We need only one of them, while all the rest are used the same way. Let's experiment with the FileWriteDouble() and FileReadDouble() functions.

First, create a file, write three variables to it and read them in random order. 

Open the file:

int h=FileOpen("test.bin",FILE_WRITE|FILE_BIN);

Write three double variables with the values 1.2, 3.45, 6.789 to file:

FileWriteDouble(h,1.2);
FileWriteDouble(h,3.45);
FileWriteDouble(h,6.789);

Do not forget to close the file.

The code can be found in the attached sTestFileCreateBin script. As a result, test.bin file appears in the MQL5/Files folder. Have a look at its contents in Notepad (Fig. 11). Open Notepad and drag the file to it:

 
Fig. 11. Binary file in Notepad

As we can see, there is no point in viewing such files in Notepad.

Now, let's read the file. Obviously, the FileReadDouble() function should be used for reading. Open the file:

int h=FileOpen("test.bin",FILE_READ|FILE_BIN);

Declare three variables, read their values from the file and show them in the message box:

double v1,v2,v3;
   
v1=FileReadDouble(h);
v2=FileReadDouble(h);
v3=FileReadDouble(h);
   
Alert(DoubleToString(v1)," ",DoubleToString(v2)," ",DoubleToString(v3));
  

Do not forget to close the file. The code can be found in the attached sTestFileReadBin script. As a result, we receive the following message: 1.20000000 3.45000000 6.78900000.

Knowing the structure of binary files, it is possible to make some limited changes in them. Let's try to change the second variable without re-writing the entire file.

Open the file:

int h=FileOpen("test.bin",FILE_READ|FILE_WRITE|FILE_BIN);

After opening, move the pointer to the specified position. The sizeof() function is recommended for the position calculation. It returns the size of the specified data type. It would also be good to be well acquainted with the data types and their sizes. Move the pointer to the beginning of the second variable:

FileSeek(h,sizeof(double)*1,0);

For more clarity, we implement the sizeof(double)*1 multiplication, so that it is clear this is the end of the first variable. If it was necessary to change the third variable, we would need to multiply by 2.

Write the new value: 

FileWriteDouble(h,12345.6789);

The code can be found in the attached sTestFileChangeBin script. After executing the script, launch the sTestFileReadBin script and receive: 1.20000000 12345.67890000 6.78900000.

You can read a certain variable (instead of the entire file) the same way. Let's read the code for reading the third double variable from test.bin.

Open the file:

int h=FileOpen("test.bin",FILE_READ|FILE_BIN);

Move the pointer, read the value and show it in the message box:

FileSeek(h,sizeof(double)*2,SEEK_SET);
double v=FileReadDouble(h);
Alert(DoubleToString(v));

This example can be found in the sTestFileReadBin2 script attached below. As a result, we receive the following message: 6.78900000 — the third variable. Change the code to read the second variable.

You can save and read the variables of other types and their combinations the same way. It is important to know the file structure to calculate the pointer setting position correctly. 

Binary files, structures

If you need to write several variables of different types to a file, it is much more convenient to describe the structure and read/write the entire structure rather than reading/writing the variables one by one. The file usually starts with the structure describing the location of data in the file (file format) followed by the data. However, there is a limitation: the structure should not have dynamic arrays and lines since their size is unknown.

Let's experiment with writing and reading the structure to the file. Describe the structure with several variables of different types:

struct STest{
   long ValLong;
   double VarDouble;
   int ArrInt[3];
   bool VarBool;
};

The code can be found in the attached sTestFileWriteStructBin script. Declare two variables and fill them with different values in the OnStart() function:

STest s1;
STest s2;
   
s1.ArrInt[0]=1;
s1.ArrInt[1]=2; 
s1.ArrInt[2]=3;
s1.ValLong=12345;
s1.VarDouble=12.34;
s1.VarBool=true;
         
s2.ArrInt[0]=11;
s2.ArrInt[1]=22; 
s2.ArrInt[2]=33;
s2.ValLong=6789;
s2.VarDouble=56.78;
s2.VarBool=false;  

Now, open the file:

int h=FileOpen("test.bin",FILE_WRITE|FILE_BIN);

Write the both structures to it:

FileWriteStruct(h,s1);
FileWriteStruct(h,s2); 

Do not forget to close the file. Execute the script to create a file.

Now, let's read the file. Read the second structure.

Open the file:

int h=FileOpen("test.bin",FILE_READ|FILE_BIN);

Move the pointer to the start of the second structure:

FileSeek(h,sizeof(STest)*1,SEEK_SET);

Declare the variable (add the STest structure description to the beginning of the file) and read the data from the file to it:

STest s;
FileReadStruct(h,s);

Describe the values of the structure fields in the window:

Alert(s.ArrInt[0]," ",s.ArrInt[1]," ",s.ArrInt[2]," ",s.ValLong," ",s.VarBool," ",s.VarDouble);  

As a result, we will see the following line in the message box: 11 22 33 6789 false 56.78. The line corresponds to the second structure data.

The code of the example can be found in the sTestFileReadStructBin script attached below.

Writing structures by variables

In MQL5, the structure fields follow one another without a shift (alignment), therefore it is possible to read certain structure fields without any difficulties.

Read the value of the double variable from the second structure in the test.bin file. It is important to calculate a position for setting a pointer: 

FileSeek(h,sizeof(STest)+sizeof(long),SEEK_SET);

The rest is similar to what we have already done many times in this article: open the file, read, close. The code of the example can be found in the sTestFileReadStructBin2 script attached below.

Defining UNICODE file, FileReadInteger function

After familiarizing ourselves with binary files a bit, we can create a useful function for defining a UNICODE file. These files can be distinguished by the value of the initial byte equal to 255. Code 255 corresponds to a non-printable symbol, therefore it cannot be present in an ordinary ANSI file.

This means we should read one byte from the file and check its value. The FileReadInteger() function is used to read various integer variables, except long, since it receives the parameter specifying the size of a read variable. Read one byte to the v variable from the file:

uchar v=FileReadInteger(h,CHAR_VALUE);

Now, we only have to check the variable value. The full function code is shown below:

bool CheckUnicode(string FileName,bool & Unicode){
   ResetLastError();
   int h=FileOpen(FileName,FILE_READ|FILE_BIN);
   if(h==INVALID_HANDLE){
      int ErrNum=GetLastError();
      printf("Error opening file %s # %i",FileName,ErrNum);
      return(false);
   }
   uchar v=FileReadInteger(h,CHAR_VALUE);
   Unicode=(v==255);
   FileClose(h);
   return(true);
}

The function returns true/false depending on whether the check was successful. The file name is passed to the function as the first parameter, while the second one (passed via a link) contains the variable equal to true for UNICODE files and false for ANSI files after the function is executed. 

The function code and the example of its call can be found in the sTestFileCheckUnicode script attached below. Launch the sTestFileCreate script and check its type using the sTestFileCheckUnicode script. After that, launch the sTestFileCreateUTF script and run the sTestFileCheckUnicode script once again. You will obtain a different result.  

Binary files, arrays, structure arrays

The main advantage of the binary files becomes noticeable when working with large amount of data. The data is usually located in arrays (since it is difficult to receive large amounts using separate variables) and in strings. Arrays may consist of both standard variables and structures that should meet the requirements mentioned above. They should not contain dynamic arrays and strings.

Arrays are written to the file using the FileWriteArray() function. The file handle is passed to the function as the first parameter followed by the array name. The two following parameters are optional. If you do not need to save the entire array, specify the array's initial element index and the number of saved elements. 

Arrays are read using the FileReadArray() function, the function parameters are identical to the FileWriteArray() function parameters.

Let's write the int variable consisting of three elements to the file: 

void OnStart(){
   int h=FileOpen("test.bin",FILE_WRITE|FILE_BIN);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   int a[]={1,2,3};   
   FileWriteArray(h,a);   
   FileClose(h);
   Alert("File written");
}

The code can be found in the sTestFileWriteArray file attached below.

Now, read (sTestFileReadArray script) and display it in the window:

void OnStart(){
   int h=FileOpen("test.bin",FILE_READ|FILE_BIN);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   int a[];   
   FileReadArray(h,a);   
   FileClose(h);
   Alert(a[0]," ",a[1]," ",a[2]);   
}

As a result, we obtain the "1 2 3" line corresponding to the previously specified array. Note that the array size is not defined, and it was not specified when calling the FileReadArray() function. Instead, the entire file was read. But the file may have multiple arrays of different types. Therefore, it would be reasonable to save the file size as well. Let's write int and double arrays to the file both starting with an int variable containing their size:

void OnStart(){
   int h=FileOpen("test.bin",FILE_WRITE|FILE_BIN);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   
   // two arrays
   int a1[]={1,2,3}; 
   double a2[]={1.2,3.4};
   
   // define the size of the arrays
   int s1=ArraySize(a1);
   int s2=ArraySize(a2);
   
   // write the array 1
   FileWriteInteger(h,s1,INT_VALUE); // write the array size
   FileWriteArray(h,a1); // write the array
   
   // write the array 2
   FileWriteInteger(h,s2,INT_VALUE); // write the array size
   FileWriteArray(h,a2); // write the array   
      
   FileClose(h);
   Alert("File written");
}

The code can be found in the sTestFileWriteArray2 script attached below. 

Now, while reading the file, we read the array size first and read to the array the specified number of elements afterwards:

void OnStart(){
   int h=FileOpen("test.bin",FILE_READ|FILE_BIN);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   int a1[];
   double a2[];
   int s1,s2;
   
   s1=FileReadInteger(h,INT_VALUE); // read the size of the array 1
   FileReadArray(h,a1,0,s1); // read the number of elements set in s1 to the array 
   
   s2=FileReadInteger(h,INT_VALUE); // read the size of the array 2
   FileReadArray(h,a2,0,s2); // read the number of elements set in s2 to the array    

   FileClose(h);
   Alert(ArraySize(a1),": ",a1[0]," ",a1[1]," ",a1[2]," :: ",ArraySize(a2),": ",a2[0]," ",a2[1]);   
}

The code can be found in the sTestFileReadArray2 script attached below.

As a result, the script shows the message: 3 : 1 2 3 - 2 : 1.2 3.4 corresponding to the size and contents of the previous arrays written to the file.

When reading the arrays using the FileReadArray() function, the array is automatically scaled. However, scaling is performed only if the current size is less than the number of read elements. If the array size exceeds the number, it remains unchanged,. Only a part of the array is filled instead.

Working with the structure arrays is completely identical to working with arrays of standard types since the structure size is defined correctly (there are no dynamic arrays and strings). We will not provide an example containing the structure arrays here. You can experiment with them on your own.

Also, note that since we are able to move the pointer all over the file, it is possible to read only one of the array elements or a part of the array. It is important to calculate the future pointer position correctly. An example of reading separate elements is also not displayed here to shorten the article length. You can try it on your own as well.

Binary files, strings, line arrays

The FileWriteString() function is used to write a string to a binary file. The two mandatory parameters are passed to the function: file handle and a line written to the file. The first parameter is optional: you can set the number of written symbols if only a part of the line should be written. 

The line is read by the FileReadString() function. In this function, the first parameter is a handle, while the second (optional) one is used to set the number of read characters.

In general, writing/reading the lines is very similar to working with an array: a line is similar to the entire array, while one line character has much in common with a single array element, therefore we will not display an example of a single line writing/reading. Instead, we will consider the more complex example: writing and reading a string array. First, let's write to the file the int variable with the array size, then write separate elements in a loop adding the int variable with its size at the beginning of each of them: 

void OnStart(){
   int h=FileOpen("test.bin",FILE_WRITE|FILE_BIN);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   
   string a[]={"Line-1","Line-2","Line-3"}; // written array 

   FileWriteInteger(h,ArraySize(a),INT_VALUE); // write the array size
   
   for(int i=0;i<ArraySize(a);i++){
      FileWriteInteger(h,StringLen(a[i]),INT_VALUE); // write the line size (a single array element)
      FileWriteString(h,a[i]);
   }

   FileClose(h);
   Alert("File written");
}

The code can be found in the attached sTestFileWriteStringArray script.

When reading, read the array size first, then change its size and read separate elements reading their size:

void OnStart(){
   int h=FileOpen("test.bin",FILE_READ|FILE_BIN);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }   
   
   string a[]; // read the file to this array
   
   int s=FileReadInteger(h,INT_VALUE); // read the array size
   ArrayResize(a,s); // change the array size
   
   for(int i=0;i<s;i++){ // by all array elements
      int ss=FileReadInteger(h,INT_VALUE); // read the line size
      a[i]=FileReadString(h,ss); // read the line
   }

   FileClose(h);

   // display the read array
   Alert("=== Start ===");
   for(int i=0;i<ArraySize(a);i++){
      Alert(a[i]);
   }

}

The code can be found in the attached sTestFileReadStringArray script. 

Shared folder for files

Until now, we have dealt with the files located in the MQL5/Files directory. However, this is not the only place where the files can be located. In the MetaEditor's main menu, execute File - Open Common Data Folder. The folder with the Files directory will open. It may also contain files available from applications developed in MQL5. Note the path to it (Fig. 12):


Fig. 12. Path to the common data folder 

The path to the common data folder has nothing to do with the path to the terminal and to the Files directory we dealt with throughout the article. No matter how many terminals you have launched (including the ones running with the "/portable" key), the same shared folder is opened to all of them.

The path to the folders can be defined programmatically. The path to the data folder (containing the MQL5/Files directory we worked with throughout the entire article):

TerminalInfoString(TERMINAL_DATA_PATH);

Path to the shared data folder (containing the Files directory):

TerminalInfoString(TERMINAL_COMMONDATA_PATH);

Similarly, you can define the path to the terminal (root directory where the terminal is installed):

TerminalInfoString(TERMINAL_PATH);

Similar to the MQL5/Files directory, there is no need to specify the full path when working with files from the shared folder. Instead, you only need to add the FILE_COMMON flag to the combination of flags passed to the FileOpen() function. Some file functions have a certain parameter for specifying the shared folder flag. These are FileDelete(), FileMove(), FileCopy() and some others.

Copy test.txt from the MQL5/Files folder to the common data folder:

   if(FileCopy("test.txt",0,"test.txt",FILE_COMMON)){
      Alert("File copied");
   }
   else{
      Alert("Error copying file");
   }

The code can be found in the attached sTestFileCopy script. After executing the script, test.txt file appears in the shared Files folder. If we launch the script the second time, we will receive the error message. In order to avoid it, allow the file overwrite by adding the FILE_REWRITE flag:

FileCopy("test.txt",0,"test.txt",FILE_COMMON|FILE_REWRITE)

Now, copy the file from the shared folder to the same folder with a different name (sTestFileCopy2 script):

FileCopy("test.txt",FILE_COMMON,"test_copy.txt",FILE_COMMON)

Finally, copy the file from the common folder to MQL5/Files (sTestFileCopy3 script):

FileCopy("test.txt",FILE_COMMON,"test_copy.txt",0)

The FileMove() function is called the same way, though the copy is not created. Instead, the file is moved (or renamed).

Files in the tester

Up to this point, our work with files was only related to MQL5 programs (scripts, EAs, indicators) running on an account (launched on a chart). However, everything is different when launching an EA in the tester. The MetaTrader 5 tester has the ability to perform distributed (cloud) testing using remote agents. Roughly speaking, the optimization runs 1-10 (numbers are conditional) are performed on one PC, runs 11-20 are performed on another, etc. This may cause difficulties and affect working with files. Let's consider these features and form the principles you should follow when working with files in the tester.

When working with files, the FileOpen() function accesses the files located in the MQL5/Files directory inside the terminal data folder. When testing, the function accesses the MQL5/Files directory files inside the testing agent folder. If the files are needed during a single optimization run (or single testing), for example, to store data on a position or pending orders, then all you have to do is clear the files before the next run (when initializing the EA). If the file is generated manually and is also to be used to determine any EA operation parameters, then it will be located in the MQL5/Files directory of the terminal data folder. This means that the tester will not be able to see it. In order to let the EA access the file, it should be passed to the agent. This is done in the EA via setting the "#property tester_file" property. Thus, it is possible to send any amount of files:

#property tester_file "file1.txt"
#property tester_file "file2.txt"
#property tester_file "file3.txt"

However, even if the file is specified using "#property tester_file", the EA still writes to the file copy located in the testing agent directory. The file in the terminal data folder remains unchanged. Further reading of the file by the EA is performed from the agent folder. In other words, a changed file is read. Therefore, if you need to save some data for further analysis during EA test and optimization, saving data to file is not applicable. You should use frames instead.

If you do not use remote agents, work with files from the shared folder (set the FILE_COMMON flag when opening a file). In this case, there is no need to specify the file name in the EA properties and the EA is able to write to the file. In short, when using the common data folder, working with files from the tester is much simpler, apart from the fact that you should not use remote agents. Also, keep an eye on file names, so that the tested EA does not corrupt the file used by the actually working EA. Working in the tester can be defined programmatically:

MQLInfoInteger(MQL5_TESTER)

When testing, use other file names.

Sharing access to files

As already mentioned, if the file is already opened, it cannot be opened the second time. If a file is already processed by one application, another program has no access to it till the file is closed. However, MQL5 provides the ability to share files. When opening a file, set the additional FILE_SHARE_READ (shared reading) or FILE_SHARE_WRITE (shared writing) flag. Use the flags with care. Present-day operating systems feature multi-tasking. Therefore, there is no guarantee that the writing — reading sequence will be executed correctly. If you allow shared writing and reading, it may happen that one program writes data, while another one reads the same (unfinished) data at the same time. Therefore, we should take additional measures to synchronize access to the file from different programs. This is a complicated task considerably extending beyond the scope of this article. Moreover, it is most likely possible to do without synchronization and file sharing (this will be shown below when using files to exchange data between the terminals).

The only time you can safely open the file with shared reading (FILE_SHARE_READ) and when such sharing is justified is when the file is used to define EA or indicator operation parameters, like configuration file for instance. The file is created manually or by an additional script and then read by several instances of an EA or indicator during initialization. During initialization, several EAs may try to open the file almost simultaneously, so you should allow them to do that. At the same time, there is a guarantee that reading and writing are not occurring simultaneously.  

Using files for data exchange between terminals

You can arrange a data exchange between the terminals saving files to the shared folder. Of course, using the files for such purposes may not the best solution, but it is useful in some cases. The solution is quite simple: no shared access to the file is used. Instead, the file is opened the usual way. No one else can open the file till writing is in progress. After writing is complete, the file is closed and other program instances cannot read it. Below is the code of the data writing function code of the sTestFileTransmitter script:

bool WriteData(string str){
   for(int i=0;i<30 && !IsStopped();i++){ // several attempts
      int h=FileOpen("data.txt",FILE_WRITE|FILE_ANSI|FILE_TXT);
      if(h!=INVALID_HANDLE){ // file opened successfully
         FileWriteString(h,str); // write the data  
         FileClose(h); // write the file
         Sleep(100); // increase the pause to let other programs 
		     // read the data
         return(true); // return 'true' if successful
      }
      Sleep(1); // minimum pause to let other programs 
                // finish reading the file and catch 
                // the moment when the file is available
   }
   return(false); // if writing data failed
}

Several attempts to open the file are made. Opening the file is followed by writing, closing and a relatively long pause (Sleep(100) function) to let other programs open the file. In case of a file opening error, a short pause is made (Sleep(1) function) to catch the moment when the file is available.

The accepting (reading) function follows the same principle. The sTestFileReceiver script attached below features such a function. Obtained data are displayed by the Comment() function. Launch the transmitter script on one chart and the receiver script on another (or in another terminal instance). 

Some extra functions

We have already considered almost all functions for working with files except for some rarely used ones: FileSize(), FileTell() and FileFlush(). The FileSize() function returns the size of an opened file in bytes:

void OnStart(){
   int h=FileOpen("test.txt",FILE_READ|FILE_ANSI|FILE_TXT);
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }
   ulong size=FileSize(h);
   FileClose(h);
   Alert("File size "+IntegerToString(size)+" (bytes)");
}

The code can be found in the attached sTestFileSize script. When executing the script, the message window with a file size is opened. 

The FileTell() function returns the position of the file pointer of an opened file. The function is used so rarely that it is difficult to think of any appropriate example. Simply note its existence and remember it in case the need arises.

The FileFlush() function is more useful. As stated in the documentation, the function sends all data remaining in the file input/output buffer to the disk. The effect of the function call is similar to closing and re-opening the file (though it is more resource efficient and the file pointer remains at its initial location). As we know, files are stored as entries on a disk. However, writing is performed to the buffer instead of the disk till the file is opened. Writing to the disk is performed when the file is closed. Therefore, the data is not saved in case of an emergency program termination. If we call FileFlush() after each writing to the file, the data is saved on the disk and program crashes do not cause any problems.

Working with folders

In addition to working with files, MQL5 features a number of functions for working with folders: FolderCreate(), FolderDelete() and FolderClean(). The FolderCreate function is used to create a folder. All functions have two parameters. The first one is mandatory for a folder name. The second one is additional to the FILE_COMMON flag (for working with folders in the common data folder). 

FolderDelete() deletes a specified folder. Only an empty folder can be deleted. However, clearing the contents of the folder is not a problem since the FolderClean() function is used for that. The entire contents including subfolders and files is deleted. 

Receiving the list of files

Sometimes, you do not remember exactly the name of a file you need. You may remember a beginning but not a numerical ending, for example file1.txt, file2.txt, etc. In this case, file names can be obtained using a mask and FileFindFirst(), FileFindNext(), FileFindClose() functions. These functions search both files and folders. A folder name can be distinguished from a file name by a back slash at the end.

Let's write a useful function for obtaining a list of files and folders. Let's gather file names in one array, while folder names in another:

void GetFiles(string folder, string & files[],string & folders[],int common_flag=0){

   int files_cnt=0; // files counter
   int folders_cnt=0; // folders counter   
   
   string name; // variable for receiving a file or folder name 

   long h=FileFindFirst(folder,name,common_flag); // receive a search handle and a name 
                                      // of the first file/folder (if present)
   if(h!=INVALID_HANDLE){ // at least a single file or folder is present
      do{
         if(StringSubstr(name,StringLen(name)-1,1)=="\\"){ // folder
            if(folders_cnt>=ArraySize(folders)){ // check the array size, 
                                                 // increase it if necessary
               ArrayResize(folders,ArraySize(folders)+64);
            }
            folders[folders_cnt]=name; // send the folder name to the array
            folders_cnt++; // count the folders        
         }
         else{ // file
            if(files_cnt>=ArraySize(files)){ // check the array size, 
                                             // increase it if necessary
               ArrayResize(files,ArraySize(files)+64);
            }
            files[files_cnt]=name; // send the file name to the array
            files_cnt++; // count the files
         }
      }
      while(FileFindNext(h,name)); // receive the name of the next file or folder
      FileFindClose(h); // end search
   }
   ArrayResize(files,files_cnt); // change the array size according to 
                                 // the actual number of files
   ArrayResize(folders,folders_cnt); // change the array size according to 
                                        // the actual number of folders
}

Experiment with this feature. Let's call it from the script the following way: 

void OnStart(){

   string files[],folders[];

   GetFiles("*",files,folders);
   
   Alert("=== Start ===");
   
   for(int i=0;i<ArraySize(folders);i++){
      Alert("Folder: "+folders[i]);
   }      
   
   for(int i=0;i<ArraySize(files);i++){
      Alert("File: "+files[i]);
   }

} 

 The sTestFileGetFiles script is attached below. Note the "*" search mask:

GetFiles("*",files,folders);

The mask allows searching for all files and folders in the MQL5/Files directory.

In order to find all files and folders starting with "test", you can use the "test*" mask. If you need txt files only, you will need the "*.txt" mask, etc. Create a folder (for example, "folder1") with a number of files. You can use the "folder1\\*" mask to receive the list of files it contains.  

Сode page

In the present article, the FileOpen() function is often applied in sample codes. Let's consider one of its parameters that we have not described yet — codepage. A code page is a conversion table of text symbols and their numerical values. Let's have a look at ANSI encoding for more clarity. The encoding character table contains only 256 characters meaning that a separate code page defined in the operating system settings is used for each language. The CP_ACP constant, from which the FileOpen() function is called by default, corresponds to the code page. It is very unlikely that someone will ever need to use a different code page, therefore it makes no sense to go into details on this subject. General knowledge would be quite sufficient.  

Working with files without limitations

Sometimes, you may want to work with files outside the terminal's file "sandbox" (outside MQL5/Files or shared folder). This may significantly expand the functionality of MQL5 applications allowing you to process source code files, automatically change them, generate image files for the graphical interface on the fly, generate the code, etc. If you do it for yourself or hire a trusty programmer, then you can do that. You can read more on that in the article "File Operations via WinAPI". There is also an easier way. MQL5 has all necessary means of working with files, therefore you may move the file to the terminal's "sandbox", perform all necessary actions and move it back. A single WinAPI function (CopyFile) is sufficient for that.

The application of the WinAPI functions should be allowed so that an MQL5 application can use them. The permission is enabled in the terminal settings (Main menu - Tools - Options - Expert Advisors - Allow DLL imports). In this case, the permission is enabled for all programs launched afterwards. Instead of the general permission, you can enable the permission only for a program you are going to launch. If the application is to access WinAPI functions or other DLLs, Dependencies tab with the "Allow DLL imports" option appears in its settings window.

There are two versions of the CopyFile function: CopyFileA() and more modern CopyFileW(). You can use any of them. However, when using the CopyFileA() function, you need to convert string arguments first. Read the "Calling API Functions" section of the article "MQL5 Programming Basics: Strings" to know the details. I recommend using the more up-to-date CopyFileW() function. In this case, the string arguments are specified as is, no conversions are required. 

In order to use the CopyFileW() function, you should import it first. You can find it in the kernel32.dll library:

#import "kernel32.dll"
   int CopyFileW(string,string,int);
#import

The code can be found in the attached sTestFileWinAPICopyFileW script.

The script copies the file containing its source code to MQL5/Files:

void OnStart(){
   
   string src=TerminalInfoString(TERMINAL_DATA_PATH)+"\\MQL5\\Scripts\\"+MQLInfoString(MQL_PROGRAM_NAME)+".mq5";
   string dst=TerminalInfoString(TERMINAL_DATA_PATH)+"\\MQL5\\Files\\"+MQLInfoString(MQL_PROGRAM_NAME)+".mq5";
   
   if(CopyFileW(src,dst,0)==1){
      Alert("File copied");
   }
   else{
      Alert("Failed to copy the file");   
   }
}

If successful, CopyFileW() returns 1, otherwise 0. The third function parameter indicates whether a file can be overwritten if there is a target file: 0 — enabled, 1 — disabled. Launch the script. If it works successfully, check the MQL5/Files folder. 

Please note that the operating system imposes limitations on file copying. There are the so-called "user account control parameters". If they are enabled, files cannot be copied to/from some locations. For example, it is impossible to copy a file to a system drive's root.

Some useful scripts

In addition to creating useful functions for working with files, let's create a couple of useful scripts for more practice. We will develop the scripts for exporting the quotes to a csv file and exporting trading results.

The quote export script will have parameters for defining data start and end dates, as well as parameters defining if the dates are to be used or all data should be exported. Set the necessary property to open the script's properties window:

#property script_show_inputs

The external parameters are declared afterwards:

input bool     UseDateFrom = false; // Set the start date
input datetime DateFrom=0; // Start date
input bool     UseDateTo=false; // Set the end date
input datetime DateTo=0; // End date

Write the code in the OnStrat() script function. Define the dates according to the script parameters:

   datetime from,to;
   
   if(UseDateFrom){
      from=DateFrom;
   }
   else{
      int bars=Bars(Symbol(),Period());
      if(bars>0){
         datetime tm[];
         if(CopyTime(Symbol(),Period(),bars-1,1,tm)==-1){
            Alert("Error defining data start, please try again later");
            return;
         }
         else{
            from=tm[0];
         }
         
      }
      else{
         Alert("Timeframe is under construction, please try again later");
         return;
      }
   }
   
   if(UseDateTo){
      to=DateTo;
   }
   else{
      to=TimeCurrent();
   }  

If the date defining variables are used, their values are used. Otherwise, TimeCurrent() for end date is used, while the start date is defined by the first bar time. 

Now that we have the dates, copy the quotes to the MqlRates type array:

   MqlRates rates[];
   
   if(CopyRates(Symbol(),Period(),from,to,rates)==-1){
      Alert("Error copying quotes, please try again later");
   }

Save the array data to the file:

   string FileName=Symbol()+" "+IntegerToString(PeriodSeconds()/60)+".csv";
   
   int h=FileOpen(FileName,FILE_WRITE|FILE_ANSI|FILE_CSV,";");
   
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }
   
   // write data to the file in the format: Time, Open, High, Low, Close, Volume, Ticks
   
   // the first line to know the location
   FileWrite(h,"Time","Open","High","Low","Close","Volume","Ticks");  
   
   for(int i=0;i<ArraySize(rates);i++){
      FileWrite(h,rates[i].time,rates[i].open,rates[i].high,rates[i].low,rates[i].close,rates[i].real_volume,rates[i].tick_volume);
   }
   
   FileClose(h);

   Alert("Save complete, see the file "+FileName);  

If successful, the script opens the appropriate message informing that the file has been successfully saved. Otherwise, the error message appears. The ready-made sQuotesExport script is attached below.

Now, let's develop the trading history saving script. The beginning is approximately the same: external variables come first, although defining time is implemented much simpler, as start time 0 is enough when requesting history:

   datetime from,to;
   
   if(UseDateFrom){
      from=DateFrom;
   }
   else{
      from=0;
   }
   
   if(UseDateTo){
      to=DateTo;
   }
   else{
      to=TimeCurrent();
   }  

Allocate history: 

   if(!HistorySelect(from,to)){
      Alert("Error allocating history");
      return;
   }

Open the file:

   string FileName="history.csv";
   
   int h=FileOpen(FileName,FILE_WRITE|FILE_ANSI|FILE_CSV,";");
   
   if(h==INVALID_HANDLE){
      Alert("Error opening file");
      return;
   }

Write the first line with field names:

   FileWrite(h,"Time","Deal","Order","Symbol","Type","Direction","Volume","Price","Comission","Swap","Profit","Comment");    

While walking through all trades, write buy and sell trades to the file:

   for(int i=0;i<HistoryDealsTotal();i++){
      ulong ticket=HistoryDealGetTicket(i);
      if(ticket!=0){         
         long type=HistoryDealGetInteger(ticket,DEAL_TYPE);         
         if(type==DEAL_TYPE_BUY || type==DEAL_TYPE_SELL){      
            long entry=HistoryDealGetInteger(ticket,DEAL_ENTRY);      
            FileWrite(h,(datetime)HistoryDealGetInteger(ticket,DEAL_TIME),
                        ticket,
                        HistoryDealGetInteger(ticket,DEAL_ORDER),
                        HistoryDealGetString(ticket,DEAL_SYMBOL),
                        (type==DEAL_TYPE_BUY?"buy":"sell"),
                        (entry==DEAL_ENTRY_IN?"in":(entry==DEAL_ENTRY_OUT?"out":"in/out")),
                        DoubleToString(HistoryDealGetDouble(ticket,DEAL_VOLUME),2),
                        HistoryDealGetDouble(ticket,DEAL_PRICE),
                        DoubleToString(HistoryDealGetDouble(ticket,DEAL_COMMISSION),2),
                        DoubleToString(HistoryDealGetDouble(ticket,DEAL_SWAP),2),
                        DoubleToString(HistoryDealGetDouble(ticket,DEAL_PROFIT),2),
                        HistoryDealGetString(ticket,DEAL_COMMENT)                     
            );
         }
      }
      else{
         Alert("Error allocating a trade, please try again");
         FileClose(h);
         return;
      }
   }

Note: for a trade type (buy/sell) and direction (in/out), the values are converted to strings, while some double-type values are transformed to strings with two decimal points. 

In the end, close the file and display the message: 

   FileClose(h);
   Alert("Save complete, see the file "+FileName);

 The sHistoryExport script is attached below.

More on the topic

The Articles section contains a huge amount of notable materials related in some way to working with files:

Conclusion

In this article, we have considered all functions for working with files in MQL5. Despite the seemingly narrow topic, the article has turned out to be quite large. However, some topic-related issues have been examined rather cursory and without enough practical examples. Anyway, the most common tasks are discussed in details, including working with files in the tester. In addition, we have developed a number of useful functions and all examples are practical and logically complete. All codes are attached below as scripts.

Attachments

  1. sTestFileRead — reading a single line from the ANSI text file and displaying it in the message box.
  2. sTestFileReadToAlert — reading all lines from the ANSI text file and displaying them in the message box.
  3. sTestFileCreate — creating the ANSI text file.
  4. sTestFileAddToFile — adding a line to the ANSI text file.
  5. sTestFileChangeLine2-1 — invalid attempt to change a single line in the ANSI text file.
  6. sTestFileChangeLine2-2 — yet another invalid attempt to change a single line in the ANSI text file.
  7. sTestFileChangeLine2-3 — replacing a single line in the ANSI text file by rewriting the entire file.
  8. sTestFileReadFileToArray — the useful function for reading the ANSI text file to the array.
  9. sTestFileCreateCSV — creating the ANSI CSV file.
  10. sTestFileReadToAlertCSV — reading the ANSI CSV file to the message box by fields.
  11. sTestFileReadToAlertCSV2 — reading the ANSI CSV file to the message box by fields with lines separation. 
  12. sTestFileReadFileToArrayCSV — reading the ANSI CSV file to the structure array.
  13. sTestFileWriteArrayToFileCSV — writing the array as a single line to the CSV ANSI file.
  14. sTestFileReadToAlertUTF — reading the UNICODE text file and displaying it in the message box.
  15. sTestFileCreateUTF — creating the UNICODE text file.
  16. sTestFileCreateBin — creating the binary file and writing three double variables to it.
  17. sTestFileReadBin — reading three double variables from the binary file.
  18. sTestFileChangeBin — rewriting the second double variable in the binary file.
  19. sTestFileReadBin2 — reading the third double variable from the binary file. 
  20. sTestFileWriteStructBin — writing the structure to the binary file.
  21. sTestFileReadStructBin — reading the structure from the binary file.
  22. sTestFileReadStructBin2 — reading a single variable from the binary file with structures.
  23. sTestFileCheckUnicode — checking the file type (ANSI or UNCODE).
  24. sTestFileWriteArray — writing the array to the binary file.
  25. sTestFileReadArray — reading the array from the binary file.
  26. sTestFileWriteArray2 — writing two arrays to the binary file.
  27. sTestFileReadArray2 — reading two arrays from the binary file.
  28. sTestFileWriteStringArray — writing the string array to the binary file.
  29. sTestFileReadStringArray — reading the string array from the binary file.
  30. sTestFileCopy — copying the file from MQL5/Files to the shared folder.
  31. sTestFileCopy2 — copying the file to the shared folder.
  32. sTestFileCopy3 — copying the file from the shared folder to MQL5/Files. 
  33. sTestFileTransmitter — script for transmitting data via the file in the shared folder.
  34. sTestFileReceiver — script for receiving data via the file in the shared data folder.
  35. sTestFileSize — defining the file size.
  36. sTestFileGetFiles — receiving the list of files by the mask.
  37. sTestFileWinAPICopyFileW — example of using the WinAPI CopyFileW() function.
  38. sQuotesExport — script for exporting quotes.
  39. sHistoryExport — script for saving the trading history.

Translated from Russian by MetaQuotes Ltd.
Original article: https://www.mql5.com/ru/articles/2720

Attached files |
Download ZIP
files.zip (27.47 KB)
Warning: All rights to these materials are reserved by MetaQuotes Ltd. Copying or reprinting of these materials in whole or in part is prohibited.

Other articles by this author

Last comments | Go to discussion (3)
patheer
patheer | 4 Dec 2017 at 20:51

this article is great, thank you very much for the information.

Rosimir Mateev
Rosimir Mateev | 25 Jul 2018 at 18:56

А как можно уменшить размер файла? Примерно имею какой то BIN файл в которой поддерживаю какие то данни, потом делаю дефрагмент файла и наконец хочу умешнит его длина, потому что сзади уже есть лишное пространство. Как етого сделать? Нужна какая та функция типа FileResize(int newSize).

KjLNi
KjLNi | 6 Dec 2021 at 15:29

Thanks a lot, exactly what I was looking for.

There is just a little detail in some of the code examples.

On my computer / my version of mql5, the example for "file open" does not work.

I have changed it slightly, so that it works in my case.

See here below.

However, many thanks again, this is absolutely appreciated!!!


int h = FileOpen("test1.txt", FILE_WRITE|FILE_ANSI|FILE_COMMON,"|",CP_ACP);

Graphical Interfaces X: The Standard Chart Control (build 4) Graphical Interfaces X: The Standard Chart Control (build 4)
This time we will consider the Standard chart control. It will allow to create arrays of subcharts with the ability to synchronize horizontal scrolling. In addition, we will continue to optimize the library code to reduce the CPU load.
Graphical Interfaces X: Updates for Easy And Fast Library (Build 3) Graphical Interfaces X: Updates for Easy And Fast Library (Build 3)
The next version of the Easy And Fast library (version 3) is presented in this article. Fixed certain flaws and added new features. More details further in the article.
MQL5 Programming Basics: Global Variables of the MetaTrader 5 Terminal MQL5 Programming Basics: Global Variables of the MetaTrader 5 Terminal
Global variables of the terminal provide an indispensable tool for developing sophisticated and reliable Expert Advisors. If you master the global variables, you will no more be able to imagine developing EAs on MQL5 without them.
Neural network: Self-optimizing Expert Advisor Neural network: Self-optimizing Expert Advisor
Is it possible to develop an Expert Advisor able to optimize position open and close conditions at regular intervals according to the code commands? What happens if we implement a neural network (multilayer perceptron) in the form of a module to analyze history and provide strategy? We can make the EA optimize a neural network monthly (weekly, daily or hourly) and continue its work afterwards. Thus, we can develop a self-optimizing EA.