MT4TerminalSync - System for the Synchronization of MetaTrader 4 Terminals

Introduction

Many traders and trading system developers have several copies of MetaTrader 4 terminals for technical purposes. Different charts and accounts are opened on them, different strategies are tested. Together with this from time to time new versions of the terminal are released, new indicators and Expert Advisors appear, another MQL code is written.

Sooner or later for a normal operation we face a question of synchronizing used terminal copies. For this purpose we need a technical tool able to solve this task. In this article we will view the process of creating it and finally get a ready tool.

For Whom Is The Article Intended?

The article analyzes the creation of a program system example for the synchronization of several terminal copies.

Accordingly, the article may be interesting to two categories of readers:
1. Users-traders can use the ready program system from this article.
2. Professional developers of program systems beyond MQL code may analyze this system for personal educational purposes, and perhaps give wholesome comments.

Necessity of the System of Terminals Synchronization

Suppose, you have eight terminal copies. You download a newly released version. How? Do you click in sequence eight times the downloaded file https://www.metaquotes.net/files/mt4setup.exe and indicate your folders with the copies? Well, it is not difficult.

Suppose you have developed three new indicators, two Expert Advisors and one library. Now you need to test them in different situation on your eight terminal copies. It means you need to update source MQL files and binary ex4 files. You have developed all the above enumerated in one source copy of the terminal. Now they need to be replicated into other copies. You open one by one the subfolders of the source terminal: experts, then experts\indicators and experts\libraries. Then in each subfolder in Windows Explorer you copy necessary files, then open in each updated terminal copy the same folder and copy the files into it. And repeat this double cycle for each source subfolder and each analogous subfolder for the updated terminal.

Is such a situation familiar to you? If you do it quite often, you have probably thought about the simplification of this work. For this purpose we need a tool that allows to automate this task. I.e. it is better to start the update of terminals by clicking one button. And it would be very useful to have the possibility of setting parameters for this process, like the way to the source control terminal, many ways of updating terminals, setting filters for files that should be updated. For example, the synchronization system should show us if there is an error in the path to the terminal folder.

It means our tool must be flexible and efficient. Let us try to create such a tool.

Technical Provisions

This article is devoted to the topic "Widening possibilities of MQL4 programs by using functions of operating systems and other means of program development". That is why the used means are beyond MetaTrader 4 terminal and MQL language.

The created and described here synchronization system for terminal copies requires for its operation the following program features:
1. Operating system - Microsoft Windows 2000 SP4 or higher: Microsoft Windows 2000, Windows XP, Windows Server 2003, Windows Server 2003 R2, Windows Vista.
2. Platform Microsoft .NET Framework 2.0. It can be downloaded from Microsoft website at:
2.1. http://www.microsoft.com/downloads/details.aspx?displaylang=ru&FamilyID=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5 - a distributive in Russian (RUS) for x86 processors, fiel dotnetfx.exe size 22 Mb.
2.2. http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5 - a distributive in English (EN) for x86 processors, file dotnetfx.exe size 22 Mb.
2.3. http://www.microsoft.com/downloads/details.aspx?familyid=B44A0000-ACF8-4FA1-AFFB-40E78D788B00&displaylang=en - a distributive in English (EN) for x64 processors, file NetFx64.exe size 45 Mb.
3. Desktop single-user database server Microsoft SQL Server 2005 Compact Edition. This redaction of the SQL server consists of eight dll files. It can be downloaded at: http://www.microsoft.com/downloads/details.aspx?FamilyId=%2085E0C3CE-3FA1-453A-8CE9-AF6CA20946C3&displaylang=en - SQLServerCE31-EN.msi file size 1.7 Mb. More information is in the MSDN article "Installing on a Desktop and a Tablet PC (SQL Server Compact Edition)" at http://technet.microsoft.com/en-us/library/bb190958.aspx (EN).

Below is the additional information for developers.

If you want to create a similar system using the indicated technologies, you will need Microsoft Visual Studio 2005, for example, one of free redactions of Visual Studio 2005 Express Editions for the language C# or VB.NET. You can download Microsoft® Visual Studio® 2005 Express Editions Service Pack 1 at: /go?link=http://www.microsoft.com/en-us/download/details.aspx?id=804. You can also order trial versions of Visual Studio 2005 at: http://www.microsoft.com/emea/msdn/visualstudio/ruru/getthetrials/. Or you can use a newer version of Visual Studio 'Orcas', which on the moment of writing the article existed in Beta 1 version. Its preliminary name Orcas may be conveyed to Visual Studio 2008. You can find the new information about Visual Studio and its new versions at: http://msdn2.microsoft.com/en-us/vstudio/aa700831.aspx.

It takes no more than two-three minutes to install Microsoft .NET 2.0 and SQL Compact Edition if you already have two necessary files (dotnetfx.exe and SQLServerCE31-EN. msi for x86 Windows versions). Simply download the file, click Next and Finish.

System Components

Except the enumerated outside software, the synchronization system itself consists of many files, described below.

Here .exe and .dll files are a binary executable code, the file config.xml is a setup file of start shell (more information is further), the file MT4TerminalSync. sdf is the system database, and the file Log.txt contains records of all errors in the system operation.

Note: due to the ideology of networking program development, especially those working on terminal servers, systems may have no right to make records to a current folder, from which they are executed. Therefore all setup files, error logging files and others that refer to each user separately must be saved in Windows folders of user profiles. However in this system, which is actually an example and a framework of a full-value network system, all files referring to it are stored in one folder for the simplification of its usage by individual users.

You do not need to install the synchronization system proper. You simply need to install the enumerated outside programs and copy the system files into a folder on a local hard disk.

The Ideology of the Developed System.

There are different ways of creating such a system. Each developer can find his own solution of this task. The advantage of the presented solution is at least that it exists and operates.

All needed for the solution is also available on script languages, for example VBS files, commands like xcopy, etc. Perhaps, this will look easier, than the given system. However, in my opinion such an approach has several disadvantages:

1. You will have to include system settings (paths to the source/destination terminal, file extensions) directly into source script texts. Or you will have to write your own parser (analyzer) of .ini file.

2. You will have almost no control of file copying errors. It is not so easy to find whether there were errors using commands like xcopy, as well as to gather them into a single form available for viewing.

So, I decided to develop a compiled system with storing all settings in a database for the maximal flexibility of extension. For the system creation I preferred to use development technologies from Microsoft that are familiar to me. Within these boundaries the following tools selection was conducted.

Description of the System Database

We can use different variants for saving the system parameters (paths, file masks, etc.): write everything into a program code, use text ini-file, XML-file or DBMS. I chose the last variant to use possibilities of DBMS and not to invent my own methods.

What DBMS to choose? There are two relevant criteria:
1. Probably the system will be used as a corporate tool, i.e. it will be a network multiuser system. It means we can take as a database Microsoft Access file or SQL server. However Access can be installed not for any user, because it is included into the version Microsoft Office Professional Edition.
2. The system will also be used by individual users. It means the size of the system should be diminished and unrolling process should be simplified. From this point of view SQL server has free redactions: Express Edition and Compact Edition. For the size minimization I have chosen the redaction SQL Server 2005 Compact Edition. Its installation takes 1.7 Mb in one file.

The redaction of the server SQL 2005 Compact Edition allows to combine these criteria - it is a desktop, compact-for-downloading version DBMS, which though shows all required options when moving to the corporate network version of the synchronization system. You simply need to recreate the database on a more powerful redaction of SQL server 2005 - any, starting from Express Edition.

We have chosen a database. Now what are we going to store?

Note: for a proper understanding of the following section you will need knowledge about SQL - Structured Query Language, a language of structured requests to databases. More information about this language is in Wikipedia at: https://en.wikipedia.org/wiki/SQL.

Here is a screenshot with tables of the used SQL database.

These tables store the information about the following system parameters:
1. Table CopyExtentions - masks of copied files, probably with relative paths. Examples: *.ini, experts\*.mq?, experts\indicators\*.ex4.
2. Table DestiantionTerminal - absolute paths to terminals for updates. These terminal copies will be updated from a template - source terminal copy. For each user profile (see the table UserProfile) there is a certain number of terminals.
3. Table SourceTerminal - absolute path to a source terminal copy. This source terminal is a check template for updating terminal copies from the table DestiantionTerminal. Each user has his own source terminal.
4. Table UserProfile - list of system users (users' profiles). This is an element of designing a multiuser system, where each user stores his setting of terminal paths with a link to his own profile. An individual user can store in different profiles logically grouped terminal sets. For example, you can divide them into two groups - one is often updated,the second one is rarely updated. Switching between the groups you may chose what group should be updated. Unnecessary profiles may be blocked. If a system contains only one unblocked profile, it will be automatically chosen as an active one in a client system section.

The above picture does not show a detailed scheme of the database table. So, let us view some more pictures of SQL tables with field types. Unfortunately Compact Editor does not have the option of showing the database diagram, so each table is presented separately.

Below is the scheme of the table CopyExtentions:

Here is the scheme of the table DestinationTerminal:

Here is the scheme of the table SourceTerminal:

Here is the scheme of the table UserProfile:

So, the above pictures showed the detailed schemes of all database tables.

Development Ideology of the Client Part of the System

This section with technical details will probably be more interesting for professional system developers, than for common users.

The client part is actually a .NET-application of Windows Forms type, i.e. the so called fat exe-client. You do not need to install the client, simply copy all its files into one folder on a local hard disk.

Here is a short description of my ideology of developing such systems.
I believe such systems should be created on the module principle. There are at least two advantages:
1. You get one final application with all necessary functions assembled from the modules like from bricks. A user does not have to start dozens of different programs, he always starts one program that contains all necessary functions. A list of modules can be adjusted for each system user separately according to the position.
2. The module principle of development allows to scale the system development process, i.e. attract more developers at the increase of development volumes.

So, what exactly do I propose? I have created a frame for starting .NET modules - application-shell of Windows Forms class for downloading .NET assemblies and activation of .NET components (User Control), i.e. technically of any class developed from the standard .NET 1.1 or 2.0 class System.Windows.Forms.UserControl.

The shell is run by the contents of XML file config.xml that is in the same folder. Below this shell is shown.

I think, its advantage is in a wide choice of programming languages and development environments (IDE). Actually they are any .NET compatible IDE for the versions .NET Framework 1.1 and 2.0, such as Borland Developer Studio 2006 (BDS) under . NET 1.1, Microsoft Visual Studio .NET 2003 under .NET 1.1, Microsoft Visual Studio 2005 under .NET 2.0, Visual Studio Orcas under .NET 2.0.

From standard programming languages it has C# and VB.NET, though you can use any CLR-compatible language that can create a project of Windows Control Library type and User Control in it.

Below is an example of the component operation, which is developed in Visual Studio .NET 2003 and loaded into a shell.

And one more example of a component created in Visual Studio Orcas March 2007 CTP.

So, we have analyzed the method of developing the client part. Now let us move on to its creation.

Client Part of the System

Use Microsoft Visual Studio 2005 for the development of the synchronization system components.
Create a project of the type Windows Control Library.

All the system components (user controls) will be in one project. The physical result will be the file MT4TSyncUCLib.dll in the project folder, which contains a binary code of components. Let us fill in the project with appropriate program components.

As a result we will get several files shown in the picture.

These are the project resources, as well as components for each system entity that is stored in SQL database: source terminal, many destination terminals, many masks of copied files and the component of the synchronization execution. Also the project has classes for each enumerated entity, for the process of data passing and their editing in a standard component PropertyGrid (a table with properties).

Note: The article does not aim at a detailed teaching to develop program systems.That is why for the client part you will see the description of check points - ready modules included into the system.

Below is the configuration of XML-file config.xml, describing all visible components of the system.

As you see from the contents of the file config.xml, I divided a user's menu into two parts: the synchronization start proper and the administrative part of the system setup. It is a relative division and can be changed by a user by editing XML text file, but not disturbing the key information. Enter the tag <menu>, section <items> and change the positions of tags <item> with the attribute type="folder" and type="control" (final custom component).
The system includes two user profiles of the menu - a full one with all components called "Administrator (all)", and a short one named "Synchronization start" only with the start component.

Now it is time to show the system itself.
The below pictures show all visible system components as in the menu bottom-upwards.

At the system start, choose the menu profile: full or simplified.

Here is the component of managing users.

A user profile blocking is used for disabling unnecessary profiles without deleting them. Leaving one active profile you save time on choosing them, because the system itself will choose the remaining profile. Of course, you do not have to block all user profiles. The system will not allow execute needed operations or will show an error in calling components, requiring selecting an active profile.

Below is the component of editing masks of copied files.

In my opinion a list of copied files should consist of the following masks:

Here please pay attention to the logical field Including Subfolders, created by me for simplifying the process of system setting. Actually, for example the files *.mq4 and *.ex4 are located in the terminal in different subfolders - experts, experts\indicators etc. In order not to add each subfolder, you can indicate one record from the root of the terminal folder, i.e. without a relative path and enable the search feature in subfolders. The system then will check all the subfolders of the source terminal and copy files with the same mask into a proper destination subfolder.

And here is one more secret. Look at the masks. The terminal includes files with the extension mq4, mqh, mqt. For not indicating all of them, we will indicate one record with the mask *.mq?, which includes three previous file types - '?' replaces one symbol in the mask, * replaces a free number of symbols. The same is with the mask *.ex? - this is a mask for the files *.exe (executable files of the terminal) and *.ex4 - binary MQL files. It is convenient, isn't it?

The following is the management component of synchronized destination terminals.

When selecting Path to Terminal on the right field border you will see a button with three dots, by clicking it you will open a standard Windows window for selecting files from a disk (Open Dialog). You may indicate the file terminal.exe in a necessary folder.

The component of the source terminal is mostly analogous to selecting destination terminals.

You may wonder, why I separated in the database the tables of source and destination terminals. Note, that for destination terminals, as distinct from the source ones, you may indicate additional features. For example, for them there is a field in the base called Start after Update. You can see it in the destination terminal components. It is not processed in the system, though for creating it you need to write two code lines - create a process for a full path to the file terminal.exe, and start it by the second line.

As a result of such an extension the system may acquire additional features and functions.

And finally the last and the most important component of synchronization start, for which the system was created.

All previous components were used for the system parameters setup, making it maximally flexible. In this component a click on the button Start starts the synchronization process, i.e. copying necessary files. In the groupbox Log Operations you will see check points of the conducted synchronization, namely paths to folders and the number of copied files.

If there are errors on the process of copying, they will be shown in the group Error Log marked by the red color. They will also be recorded into the file Log.txt in the system folder. Such a method is used for separating informing messages from critical error messages. The reason for the error may be, for example, a wrong path to a source or destination terminal (it is entered into the base, but the folder deleted from the hard disk), or an error in a monopolistic file blockage without access even for reading.

There was a choice for this block - either to single out copying into a separate flow, or leave it within the boundaries of the main flow of the system process. The first variant would give a better reaction of the system on a user's actions in the process of copying, i.e. without system "fades". However the flow would operate with lower priority, and probably the copying speed would be lower. So I decided to leave copying inside the current main flow, making it handler on a timer in the component. Because of this, the system may respond to mouse clicks only in periods of processing of a timer event. However you can stop the synchronization process by clicking Stop - the system will process the click when moving upon the list of destination terminals.

Conclusion

We have managed to solve the task and got a technical tool - synchronization system with its parameters setup, logging and displaying of possible errors, etc. I have taken the mask for copied files from the file INSTALL.LOG of a root folder of the terminal, section [CopyFiles]. Probably during operation you will find more files with other extensions that should be synchronized. It would be very useful and interesting for other users, if you share your point of view in comments to the article.

The ready binary project and the example code of the system module are attached to the article.