The Principles of Economic Calculation of Indicators

Introduction

The idea of preservation of resources in one or another segment of people's practical activities - is perhaps one of the most significant and pressing topics on the path to human development and progress. Programming in MQL5, in this regard, is no exception. Of course, if the range of tasks is limited only to visual trading, then many flaws of programming can remain unnoticed.

But everything that is relates to automated trading, initially requires a maximally economical writing of the code, otherwise, the process of testing and optimization of trading robots can be stretched out over such a long period of time, that it would be virtually impossible to wait until their completion. The idea of creating something of value in this type of situation seems quiet doubtful.

So before embarking the implementation of trading strategies, it would make sense to get better familiarized with those details of programming, which have an effect on the time of optimization and testing of Expert Advisors. And since the majority of Expert Advisors contain calls to the user's indicators in their codes, then I suppose, we should begin with them.

In general, there aren't that many relevant points, which must be kept in mind when constructing indicators, so it is most logical to simply review each one of them in order.

Recalculation on each indicator tick of the ones, which have not yet been calculated, newly emerged bars in classic indicators

The very essence of classical indicators, such as RSI, ADX, ATR, CCI, etc, is that on the closed bars, the calculation of these indicators can be done only once, and after that calculations can only be done on the newly emerging bars. The only exception is the current open bar, on which the computation is done repeatedly at each tick, until this bar is closed.

The easiest way to find out whether it is reasonable to calculate the indicators on non-calculated bars - is to compare in the strategy tester, the results of running such (optimized) indicators with the indicators, calculated on all bars, all the time (not optimized).

This is done quiet easily. An Expert Advisor is created with the empty function of OnInit () and OnTick (). All you need to do is write the call in the Expert Advisor of the required version of optimized or non-optimized indicator, and admire the results of runs in the tester of such experts in both cases. As an example, I'll take the indicator SMA.mq5 from my article "User Indicators in MQL5 for Beginners", in which I will make a line replacement.  

   if (prev_calculated == 0) // if this is the first start, then make a recalculation of all existing bars
    first = MAPeriod - 1 + begin;
   else first = prev_calculated - 1; // on all subsequent starts, make calculations only on newly appeared bars

on 

   first = MAPeriod -  1  + Begin;  / / On all ticks the bars are being recalculated 

As a result, I get a non-optimized version of the programming code (SMA !!!!!!. mq5), which, unlike the original, will recalculate all of its values on each tick. Strictly speaking, versions of the expert code in both cases are practically identical, so I will provide only one of them (SMA_Test.mq5)

//+------------------------------------------------------------------+
//|                                                     SMA_Test.mq5 |
//|                        Copyright 2010, MetaQuotes Software Corp. |
//|                                              https://www.mql5.com |
//+------------------------------------------------------------------+
#property copyright "Copyright 2010, MetaQuotes Software Corp."
#property link      "https://www.mql5.com"
#property version   "1.00"
int Handle;
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//----+
   //----+ Obtaining the indicator handle
   Handle = iCustom(Symbol(), 0, "SMA");
   if (Handle == INVALID_HANDLE)
     Print(" Not able to obtain the indicator handle SMA");
//----+
   return(0);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
//----+
    //--- Release the indicator handle
    IndicatorRelease(Handle);
//----+   
  }
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
//----+
   double SMA[1];
   //----+ Using the indicator handle, copy the values of the indicator 
                   // buffer in a specially prepared for this statistical array
   CopyBuffer(Handle, 0, 0, 1, SMA);
//----+
  }
//+------------------------------------------------------------------+

We can begin the tests now. It should be noted that in all of the tests in this article, we will be using a simulation mode of bar changes, which is as close as possible to reality -"Every tick"!

 

Here's the result of running an optimized indicator in the tester:

 

The red color indicates the time over which the test was passed. Can't say that this is too much! But for the completion of testing the indicator SMA !!!!!!. mq5 we had to wait a very long time!

 

Basically, the time of test processing in this case differs from the previous test by more than 500 times. And this is despite the fact that a sufficiently short testing period was selected. But while during the testing of the Expert Advisor we can put up with such large computational costs, we better forget about the optimization of their parameters!

And therefore, this is the most eloquent evidence to the fact that an economically written code - is not just a pastime for professionals in the field of programming, but rather a quite topical approach to writing your own code.

On the Internet there is a whole website Overclockers.ru dedicated to the acceleration of personal computers in order to maximize its performance. The basic means of this practice is the use of much more expensive computer components to increase the clock speed of the CPU and the RAM memory.

After this is done, a more expensive water cooling systems, or even immersion in a liquid nitrogen processor, is used for this overclocked CPU. The result of such acts is a twofold or even threefold increase of the PC performance.

A competent, economical writing of the code can often help us to achieve a lot, while applying very little effort. Of course, this method is not able to transform a Celleron300A into a Core 2 Quad Q6600, however it does allow us to make a regular, standard -budget PC, function in a way, which would only be expected from a top model computer!

Repeated recalculation of the current closed bar in some, not quite classic, indicators

And everything would be just great if this method of program code optimization was suited for all indicators indiscriminately! But, alas, this is not true. There is a whole group of indicators, which under such approach begin to normally calculate only once, during the loading of the indicator on already existing historical data.

And on all of the bars that emerged after loading the indicator, its values turn out to be completely incorrect. The main reason this happens is that some variables from the indicator code are dependent on those indicators, which had the same variables after the calculation of the indicator on the previous bar. Officially, it will look as follows:

                                                                                                                                                                                                                           

SomeVariable(bar) = Function(SomeVariable(bar - 1))

 

where:

• SomeVariable() — the value of some variable for some bar;
• bar - bar number, on which the calculation is made.

For obvious reasons, in the actual code, such dependences have a less clear functional form. But the essence of this does not change, for example, for moving T3 (non-optimized indicator - T3 !!!!!!. mq5) the part of the code, relevant to us, looks like this

   e1 = w1 * series + w2 * e1;
   e2 = w1 * e1 + w2 * e2;
   e3 = w1 * e2 + w2 * e3;
   e4 = w1 * e3 + w2 * e4;
   e5 = w1 * e4 + w2 * e5;
   e6 = w1 * e5 + w2 * e6;
   //----  
   T3 = c1 * e6 + c2 * e5 + c3 * e4 + c4 * e3;

The variables e1, e2, e3, e4, e5, e6 have precisely such functional dependence, which involves the use of this code for the calculation of each new bar just once! But the current bar, through a similar calculation, will be skipped repeatedly until it is closed.

And the values of these variables on the current bar will change all the time, although for the current bar, prior to changing it, they should remain as they were after the calculation on the previous bar!

And therefore, the values of these variables on the previous bar (with respect to the current bar) should be saved in the static variables, and transfer them to be reused until the next change of the bar, on which the penultimate values of the variables should be again saved e1, e2, e3, e4, e5 , e6.

The additional code, which performs similar manipulations with the values, is quite simple. First of all, you should declare the local static variables for storing values inside the function OnCalculate ()

   //---- declare the static variables for storing the valid values of the coefficients
   static double e1_, e2_, e3_, e4_, e5_, e6_;

After this, making the memorization of the variables values in the cycle on the current bar before any calculations, at the moment when the number of newly emerged bars is greater than zero:

     //---- memorize the values of variables before the run on the current bar
     if (rates_total != prev_calculated && bar == rates_total - 1)
      {
       e1_ = e1;
       e2_ = e2;
       e3_ = e3;
       e4_ = e4;
       e5_ = e5;
       e6_ = e6;
      }

And before the block of the cycle operator, restore the values of variables by inverse transformation:

   //---- restore the values of the variables
   e1 = e1_;
   e2 = e2_;
   e3 = e3_;
   e4 = e4_;
   e5 = e5_;
   e6 = e6_;

Quite naturally, the starting initialization of the calculated coefficients are now done only once, at the first start of the function OnCalculate (), and now initialization is done not of the coefficients themselves, but of the corresponding static variables.

//---- calculating the starting number first for the cycle of recalculation of bars
   if (prev_calculated == 0) // verification of the first start of the indicator calculation
    {
     first = begin; // the starting number for calculating all of the bars
     //---- the starting initialization of calculated coefficients
     e1_ = price[first];
     e2_ = price[first];
     e3_ = price[first];
     e4_ = price[first];
     e5_ = price[first];
     e6_ = price[first];
    }

As a result, the final indicator T3.mq5 began to make calculations in the most cost-effective way. All would be nothing, but it is not always that similar functional dependences can be so easily identified. In this case, the values of all the indicator variables can be memorized in static variables, and restored in the same way.

And only afterwards we can begin to figure out which variables really need to be recovered, and for which there is no such need. To do this we need to hang on the chart the unoptimized and optimized versions of the indicator, and check their work, gradually removing from the list of recovery one variable at a time. In the end, we are left with only those variables, which really need to be recovered.

Naturally, I provided this version of logic for working with the program code of normal indicators, in which there is a recount of the current bar and the newly emerging bars. For indicators, which repaint and look into the future, we will not be able to create an analogous, very standard and simple method of code optimization, due the unique traits of these indicators. And the majority of experienced Expert Advisor writers see no real need for this. Therefore, this is where we can consider the detailed analysis of these indicators complete.

Features of calls for indicators, which can make MQL5-code excessively slow 

It would seem that the task is complete, we have an optimized indicator, which counts bars in the most economical way, and now it is enough to write a few code lines, and call up this indicator into the Expert Advisor's or indicator's code to obtain the calculated values from the indicator buffer.

But this is not as easy as it might seem if approached in a formal matter, without bothering to figure out what type of operations are behind these few lines of code.

The specifics of obtaining the values from the user and technical indicators, just as from the time- series in MQL5, is that this is done through copying data into the users variable arrays. This can lead to a build up of completely unnecessary, for the current accounts, data.

The easiest way to verify all of this on a specific data receivership from some technical indicator. As an example, we can take moving iAMA and build, on the basis of this technical indicator, a custom indicator AMkA.

To copy the data we will be using the first version of the function call CopyBuffer (), with a request to the starting position and the number of required elements for copying. In the AMkA indicator, the Moving incrementation on the current bar is processed using a technical indicator Standard Deviation, and then, for obtaining the trading signals, this increment is compared with the total processed, standard deviation value.

Therefore, in the simplest case for the implementation of the AMkA indicator, you should first create an indicator, in which the indicator buffer contained the value of Moving incrementation (indicator dAMA). And then, in other indicator, using the indicator handle with AMA increments, we obtain the resulting value, processed by the Standard Deviation indicator .

The process of creation of analogous indicators has already been examined in detail in various articles on these topics, so I will not pause on it, and analyze only the details of accessing the indicator buffers of the called indicator, in the code of another indicator.

In the vast resourced of the Internet, we already see the emergence of MQL5 examples, in which their authors literally copy the entire contents of the indicator buffers into the intermediate, dynamical arrays. And after this, all of the values, one by one, are transferred into the final indicator buffers from these intermediate arrays, using the cycle operator.

To solve our problem, this approach looks maximally simple

   if (CopyBuffer(AMA_Handle, 0, 0, rates_total, Array) <= 0) return(0);
   ArrayCopy(AMA_Buffer, Array, 0, 0, WHOLE_ARRAY);

(Indicator dAMA !!!!!!. mq5) Or like this 

   if (CopyBuffer(AMA_Handle, 0, 0, rates_total, Array) <= 0) return(0);
   
   for(bar = 0; bar < rates_total; bar++)
    {
     AMA_Buffer[bar] = Array[bar];
     /*
      here is the code of indicator calculations
    */     
    }

But what is the price for such an unpretentious solution? First, it would be best to gain a bit of understanding of what would be the maximally rational course of action. First of all, there is no seriously justified need to use the intermediate array Array [], and the data should be copied directly to the indicator buffer AMA [].

Second of all, at each indicator tick it is necessary to copy the values only in three cases:

• from the newly emerged bars
• from closed bars,
• from the current open bars.
  

The remaining values in the indicator buffer already exist, and there is no sense in multiple rewriting of them. 

//--- calculation of the required number of copied data
   int to_copy;
   if(prev_calculated > rates_total || prev_calculated <= 0)// verification for the first start of indicator calculation
        to_copy = rates_total - begin; // calculated number of all bars
   else to_copy = rates_total - prev_calculated + 1; // calculated number of only new bars

//--- copy the reappearing data into the indicator buffer AMA_Buffer[]
   if (CopyBuffer(AMA_Handle, 0, 0, to_copy, AMA_Buffer) <= 0)
    return(0); 

 It is quite natural that the final code in this case will be somewhat more complex (Indicator dAMA.mq5), but now we can use the methodology, proposed by me in the beginning of this article, to conduct tests in both cases, and draw the appropriate conclusions. This time, let's increase the testing period to one year.

 

Finally, after passing the test in the tester strategy journal, we obtain the necessary time of testing the Expert Advisor dAMA_Test

The resulting time of passing the test in 43,937 ms is within a reasonable range. What, unfortunately, can not be said about the analogous resulting time of passing the test with the Expert Advisor dAMA !!!!!!_ Test

  

The time of passing the test is 960 625 ms, which is twenty times longer than in the previous case. The conclusion seems quiet obvious. The code should be written in the most economical way, so that it will not perform any unnecessary calculations!
The AMkA indicator, built on the principles described above, doesn't demonstrate anything new, so I will pause only on the details of copying data in this case.

//---- declaring local arrays
   double dAMA_Array[], StdDev_Array[];
//---- indexation of elements in arrays just like in time series
   ArraySetAsSeries(dAMA_Array, true);
   ArraySetAsSeries(StdDev_Array, true);

//--- calculation of the number of copied data
   int to_copy;
   if(prev_calculated > rates_total || prev_calculated <= 0)// verification of the first start of indicator calculation
        to_copy = rates_total - begin; // calculated number of all bars
   else to_copy = rates_total - prev_calculated + 1; // calculated number of only new bars
   
//--- copy the newly appeared data into the indicator buffer and local dynamic arrays
   if(CopyBuffer(dAMAHandle,   1, 0, to_copy, AMABuffer   ) <= 0) return(0);
   if(CopyBuffer(dAMAHandle,   0, 0, to_copy, dAMA_Array  ) <= 0) return(0);
   if(CopyBuffer(StdDevHandle, 0, 0, to_copy, StdDev_Array) <= 0) return(0);

Everything is done in a completely analogous way, except that now the data is copied into a single indicator buffer and two locally declared dynamic arrays for intermediate calculations. 

The implementation of all indicator calculations within the indicator, as one of the ways for optimization 

All this is very interesting, but such a complex structure of successive calls of user and technical indicators looks a bit too suspicious. And it should be measured somehow. But to do this, it would not hurt to have the code of the AMkA indicator, which would be located within the user's indicator, and would not use the calls for other indicators.

For a programmer who has thoroughly grasped the process of writing indicators on MQL5, this problem does not require much effort. First the code of the user's indicator AMA.mq5 is written, and then the necessary elements of the code are added to it for the implementation of the AMkA_.mq5 indicator. We receive another second, large cycle of indicator calculation, in which the user array gets loaded with the increment of the AMA indicator,

   //---- the main cycle of calculating the AMkA indicator
   for(bar = first; bar < rates_total; bar++)
    {
     //---- load the increments of the AMA indicator into the array for intermediate calculations
     for(iii = 0; iii < ama_period; iii++)
      dAMA[iii] = AMABuffer[bar - iii - 0] - AMABuffer[bar - iii - 1]; 

then this array is used to perform operations, analogous to calculation of technical indicator StDev based on the data of dAMA.mq5 indicator.

     //---- find the simple average of increments of AMA
     Sum = 0.0;
     for(iii = 0; iii < ama_period; iii++)
      Sum += dAMA[iii];
     SMAdif = Sum / ama_period;
     
     //---- find the sum of the square differences of increments and the average
     Sum = 0.0;
     for(iii = 0; iii < ama_period; iii++)
      Sum += MathPow(dAMA[iii] - SMAdif, 2);
     
     //---- determine the final value of the meansquare deviation of StDev from the increment of AMA
     StDev = MathSqrt(Sum / ama_period);

The rest of the code is absolutely analogous to the code of the AMkA.mq5 indicator, and doesn't pose any special interest to us. Now we can begin testing the indicators AMkA_.mq5 and AMkA.mq5 with the help of Expert Advisors AMkA__Test.mq5 and AMkA_Test.mq5.

There are no arising difficulties with testing the AMkA_.mq5 indicator, and the testing time is well within the acceptable framework

 

But the indicator AMkA.mq5 was incredibly slow

 

Its result turned out to be as much as seven times worse than its "brother's". What other comments can there be? The conclusion is quiet obvious: that building such complex constructions, consisting of several consecutive calls of indicators, one from another, is not very prudent and is suitable only for preliminary tests!

Obviously, these results were obtained on a test version of the client terminal, and it is hard to say right now what it will all look like in the future. But in regards to the upcoming championship of trading robots, it can be argued quite clearly that this is a relevant and efficient topic.

Some features of calling up indicators from the Expert Advisors

Everything that is related to optimization of access to user's data and technical indicators in the programming code of the indicators, is equally applicable to the optimization of access to user's data and technical indicators in the program code of Expert Advisors. In addition to the already presented situations, the Expert Advisor code has another factor, which may significantly influence the testing and optimization of trading systems.

A large number of Expert Advisors usually processes the indicator data only at the time of the bar change, and for this reason, in these Expert Advisors there is no need to pull up the function CopyBuffer () for every tick.

In this situation, there is a need for Expert Advisors to get the data from the indicator buffers only during a bar change. Thus, the calls of the indicators should be located in the block behind the brackets, the access to which is allowed only once for each change of the bar, if all of the necessary data from the indicator buffer are successfully copied into the arrays for intermediate calculations.

The best thing that serves as such filter is a user function, which returns a logical unit to the time when there was a change of the current bar. The file IsNewBar.mqh contains my, fairly universal, version of this function:

bool IsNewBar
            (
             int Number, // Number of call to the function IsNewBar in the program code of the expert Advisor
             string symbol, // The chart symbol on which the data calculation is done
             ENUM_TIMEFRAMES timeframe // Timeframe of the chart on which the data calculation is done
            )

When using this function in the Expert Advisor code, it may look like this:

    //---- declaration of a static variable - array for storing the values of the AMA indicator
    static double AMA_Array[3];

    //---- calling up the AMA indicator for copying its values into the AMA_Array array
    if (IsNewBar(0, Symbol(), 0))
     {
      CopyBuffer(AMA_Handle, 0, 1, 3, AMA_Array); 
     }

But it is far more rational to act differently in this case. The fact is that when you call up the CopyBuffer (), the data in the AMA_Array [] array may not be copied, and in such case, you will need to call up this function for each tick until there is a successful option for copying the data, which is implemented by a certain complication of the filter

   //---- declaration of a static variable - array for storing values of the AMA indicator
   static double AMA_Array[3];
    
   //---- declaration of the static variable for storing the results of copying the data from the AMA indicator
   static bool Recount;

   //---- calling up the AMA indicator for copying its values into the AMA_Array array
   if (IsNewBar(0, Symbol(), 0) || Recount)
     {
      if (CopyBuffer(AMA_Handle, 0, 1, 3, AMA_Array) < 0)
       {
        Recount = true; // attempt of data copying was unsuccessful 
        return; // exit the function OnTick()
       }
      
      //---- All operations of copying from the indicator buffers are successfully completed
           // there is no need for returning to this block until the next bar change
      Recount = false;
     }

Now that the details of the rational call of the copying function of the indicator values in the Expert Advisor's code are clear, you can test the benefits of applying the function IsNewBar () in the Expert Advisors.

So we have two options of Expert Advisors, available for being tested in the strategy tester, the first one - AMA_Test.ex5. It copies the data from the indicator buffer at each tick.

The second one - IsNewBar_AMA_Test.mq5 copies the data only during a bar change.

Yes! The test results are somewhat disappointing. It turns out that calling a function IsNewBar () on each tick is much more expensive than copying the data into three cells of the user's array! 

Here I would like to draw your attention to another important, but seemingly inconspicuous part of the indicator. The fact is that if we obtain the handle of the indicator in the OnInit () function, then regardless of whether or not we copy the data from this indicator within the function OnTick (), its calculations on a yet un-calculated and current bars, will still be done on each tick.

So if our Expert Advisor does not require the counted indicator values from the current open bar, then it is better, in terms of time saving, to disable the calculation of these values. This is done quite easily - reduce by one the right border of the main cycle of bar recounting in the indicator Prior to changes, this cycle in the AMA.mq5 indicator looked like this

   //---- main cycle of indicator calculation
   for(bar = first; bar < rates_total; bar++)

And after the change, it will look like this  

   //---- main cycle of indicator calculation
   for(bar = first; bar < rates_total - 1; bar++)

Indicator AMA_Ex.mq5. Now you can test this indicator (Expert Advisor AMA_Ex_Test.mq5)

 

Of course, this result is better than the test of AMA indicator by 21%, which is not too bad, but if we think about it, this result could be much better.

Conclusion

Ultimately, the efficiency of a program's code is a fairly objective parameter. Efficiency can be measured, logically analyzed and in certain situations, significantly increased. The methods by which this is done are not very complicated. All that is needed is some patience, and to do more than those practices which directly affect the automated trading system's profitability.