Tips from a professional programmer (Part I): Code storing, debugging and compiling. Working with projects and logs

Contents

• Introduction
• Store your code in separate subdirectories
• One code for multiple terminals
• Use a version control system
• Use a separate terminal with a demo account to debug the code
• Compile all code files at once
• Use project and task management systems
• Log selection
• Log highlighting
• Contextual search
• Conclusion

Introduction

Everyone has their own programming habits, styles and preferences. I will share some methods which make my job (my favorite activity) easier. I hope this information will be useful for beginners. More experienced programmers might also find something useful.

Store your code in separate subdirectories

Terminal program files are located under the MQL5 directory. This catalog is a so-called "sandbox". Data access from the outside is closed. This is a good decision. However, the use of a DLL probably enables access to anywhere.

For example, here is the structure of the Cayman project:

• /Experts/Cayman/ - Expert Advisor
• /Files/Cayman/ - data files (settings, parameters)
• /Include/Cayman/ - library of classes (functions)
• /Scripts/Cayman/ - main operational scripts
• /Scripts/CaymanDev/ - developer scripts (used for debugging)

The main advantages of this placement are:

• Contextual search only in project files via TotalCommander
• Version control via Git (control is enabled only for the project files, while all other files are ignored)
• Easy copying to another terminal (demo -> real – release)

One code for multiple terminals

One of the best practices in programming is avoiding duplicate code. If the same code lines appear in several different places, then you should better "wrap" these lines in a function. The same applies to MQL5 files: there should be only one text of a program file. This can be implemented by using a symbolic link to the MQL5 directory.

Suppose the project category is located at D:\Project, while the terminal data directory is located at C:\Users\Pro\AppData\Roaming\MetaQuotes\Terminal\9EB2973C469D24060397BB5158EA73A5

1. Close the terminal
2. Go to the data directory
3. Move the MQL5 directory to the projects directory
4. While in the data directory, run cmd and enter the following command
   
   mklink /D MQL5 D:\Project\MQL5
   
   
5. Launch the terminal

The terminal will not even notice that the "sandbox" (program files) has moved to D:\Project\MQL5.

The main advantage of this placement is that all personal projects are gathered under the same directory (D:\Project).

Use a version control system

Professional programmers do not doubt the need to use such system. It is an indispensable tool, especially when a team of programmers is working on the same project. The question is which system you are going to use. This is de facto Git.

The main advantages of Git are:

• Local repository. Ability to experiment with branches. Switching to any branch (version) with one click.
• Convenient graphical interface (TortoiseGit). Control with the mouse.
• Free cloud repository for personal projects (Bitbucket). No fear that your PC hard disk will fail
• File change history with the ability to restore or view older versions (conveniently viewed in Bitbucket)

I will not provide here detailed installation and configuration instructions, but I will mention some specific features. Install Git (for command line operation) and TortoiseGit (to use the mouse). Under the D:\Project\MQL5 directory, create the .gitignore file with the following contents

# exclude files
*.ex4
*.ex5
*.dat
log.txt

# exclude directories completely
Images
Indicators
Libraries
Logs
Presets
Profiles
Services
"Shared Projects"
Levels
Params

# exclude directory contents
Experts/*
Files/*
Include/*
Scripts/*

# except for directories
!Experts/Cayman
!Files/Cayman
!Include/Cayman
!Scripts/Cayman
!Scripts/CaymanDev

This file will enable the version tracking of (*.mq?) program files of the project. Create a local repository in the MQL5 directory. Add files and make the first commit (commit the version). The local repository is ready. Work with the code and do not forget to make frequent commits with a brief description of changes. The comments make it easier to browse and search the history later on.

To connect to a cloud repository, you first need to create a Bitbucket account and repository. Logically, the repository name should match the project name. In my case it is CaymanMQL5, while there is also CaymanMQL4. Import the local repository to the cloud one. Now, the cloud repository is ready. Basic actions via TortoiseGit(TG):

• Work with the code (one task - one commit)
• Check modifications (TG/Check for modifications…)
• Add new files (Add)
• Delete unnecessary (missing) files (Delete)
• Commit to the local repository (Commit)
• Push to the cloud repository (Push)

Use a separate terminal with a demo account to debug the code

You should have the latest working version of code on a real account (only *.ex? files, without *.mq?). The code in the debugging process should be on a demo account. To copy from demo to real, you can use the following batch file:

@echo off
setlocal
set PROJECT=Cayman
set SOURCE=d:\Project\MQL5
set TARGET=c:\Users\Pro\AppData\Roaming\MetaQuotes\Terminal\2E8DC23981084565FA3E19C061F586B2\MQL5
set PARAMS=/MIR /NJH /NJS
rem MIR - MIRror a directory tree and delete dest files/folders that no longer exist in source
rem NJH - No Job Header
rem NJS - No Job Summary
echo Copy *.ex? // Source to Production
echo Source = %SOURCE%
echo Production = %TARGET%
robocopy %SOURCE%\Experts\%PROJECT% %TARGET%\Experts\%PROJECT% *.ex? %PARAMS%
robocopy %SOURCE%\Scripts\%PROJECT% %TARGET%\Scripts\%PROJECT% *.ex? %PARAMS%
rem Copy all files except AppSettings.txt, [Levels], [Params]
robocopy %SOURCE%\Files\%PROJECT% %TARGET%\Files\%PROJECT% *.* %PARAMS% /XF AppSettings.txt /XD Levels Params
robocopy %SOURCE%\Scripts\Cayman %TARGET%\Scripts\Cayman *.ex? /NJH /NJS
robocopy %SOURCE%\Scripts\CaymanDev %TARGET%\Scripts\CaymanDev *.ex? /NJH /NJS
echo.
endlocal
pause

Compile all code files at once

The goal is to maintain code consistency. So, you should avoid, for example, changes in function parameters. The Expert Advisor can be compiled well, but there can also be a script which uses the old version of the function. The script (which was compiled earlier) can run well, but it will not perform the required functionality. For batch compilation, you can use the following file:

@echo off
setlocal
set METAEDITOR="C:\Program Files\RoboForex - MetaTrader 5\metaeditor64.exe"
set CAYMAN=d:\Project\MQL5\Scripts\Cayman
set CAYMAN_DEV=d:\Project\MQL5\Scripts\CaymanDev

echo METAEDITOR=%METAEDITOR%
echo CAYMAN=%CAYMAN%
echo CAYMAN_DEV=%CAYMAN_DEV%
echo.
echo Wait compile...

D:

cd %CAYMAN%
echo %CAYMAN%
for %%F in (*.mq?) do (
        %METAEDITOR% /compile:%%F /log
        type %%~dpnF.log
)       
del *.log

cd %CAYMAN_DEV%
echo %CAYMAN_DEV%
for %%F in (*.mq?) do (
        %METAEDITOR% /compile:%%F /log
        type %%~dpnF.log
)
del *.log

endlocal
echo.
pause

Use project and task management systems

Such systems are a must for development teams. They are also useful for personal projects. There are a plethora of different project management systems and methodologies. I personally use ZenKit. It is free for small teams. It offers a very convenient Kanban board.

I used to have multiple Kanban boards (one for each project). The disadvantage of using multiple boards is that you do not see the overall picture. Then I decided to add a project as a development stage. This way it is much easier to manage tasks. For example, my Kanban board has 5 development stages:

• Legend - project description, links and instructions. Entries are not moved to other stages. You can always view the general purpose of the board and can easily access links to useful resources.
• Cayman – MQL5 and MQL4 project
• Website – my personal website project
• Accepted - task in progress
• Done - task completed

The board using process is very simple and intuitive. I find an interesting idea, a solution or a program error. Then I briefly formulate it and add it to the board, to the corresponding project. You can also add here images and files. Next, I select a task and move it to the "Accepted" stage. I work with the task, test it and move to the "Done" stage.

By analyzing the number of tasks in projects you can easily determine problems and bottlenecks. I try to take tasks from "overloaded" projects. My goal is to align projects so that they have similar number of tasks. The team development has another stage entitled "Testing" stage, which is not relevant to me as I work alone.

Log selection

As you know, all outputs of Print functions are written to the Terminal log MQL5\Logs\yyyyMMdd.log. All symbols (trading instruments are mixed in this file. I use the following batch file to select logs for a required symbol:

echo off
if "%2"=="" goto Help
find /I "%2" < %1 > "%~n1-%2.log"
goto Exit
:Help
echo.
echo Filter log-file
echo findLog.cmd logFile text
echo Example
echo findLog.cmd 20200515.log gbpjpy
echo Result
echo 20200515-gbpjpy.log
echo.
pause
:Exit

Log highlighting

A lot of information is written to log when I perform debugging. Analyzing it in a running Terminal can be quite tricky. Here is what you can do to simplify this process:

• Clear the log in the Toolbox/Experts tab
• Stimulate printing to log
• Copy all lines to the log.txt file
• Analyze this file in Notepad++ (NPP) using highlighting

NPP supports different text highlighting variants

• Double click on a word highlights this word throughout the entire text
• Search/Mark/Using style - this applies the selected style to the required text throughout the document
• Custom syntax with the highlighting of specific tokens

Contextual search

To find a specific text, I use the TotalCommander file manager. The main advantages of this method are as follows:

• Search is only performed in project files (all other files do not appear in search results)
• Ability to use regular expressions in search

Conclusion

This is my first article. I am planning to continue the series. Next time I will share some successful solutions, providing the relevant description and explanation. I invite everyone to share their programming experience and to discuss other useful tips and solutions.