Find average value over number of bars? - page 2

New comment
 
R4tna C #: What do you mean by "deterministic execution time" ?

He means that the time it takes to execute is pre-determined or almost constant. Usually this is achieved by doing the calculations incrementally, on each bar for example, so that the calculation are spread out over many bars instead of having to do the whole calculation all at once.

This is one of the reasons why the exponential moving average can be calculated so much faster than the simple moving average for increasingly larger periods. An exponential moving average always takes the same amount of time to calculate irrespective of the period, while a simple moving average (which is the case here) gets slower, and slower the longer its period.

EDIT: That is the reason why in his calculations he is keeping track of a ring buffer, so that the sum can be calculated incrementally on each bar.

 
Fernando Carreiro #:

He means that the time it takes to execute is pre-determined or almost constant. Usually this is achieved by doing the calculations incrementally, on each bar for example, so that the calculation are spread out over many bars instead of having to do the whole calculation all at once.

This is one of the reasons why the exponential moving average can be calculated so much faster than the simple moving average for increasingly larger periods. An exponential moving average always takes the same amount of time to calculate irrespective of the period, while a simple moving average (which is the case here) gets slower, and slower the longer its period.

EDIT: That is the reason why in his calculations he is keeping track of a ring buffer, so that the sum can be calculated incrementally on each bar.

Thanks for the explanation.

From my previous roles/experiences, I am always conscious of performance when writing MQL code (which is still a relatively new area for me). So far I am yet to find any MQL computation which takes more than a couple of milli-secs and often it is far less.

Given it is predominantly in-memory and relatively simple in architecture, it has that advantage of not being heavily encumbered.

 
William Roeder #:

Mine is O(n), his is O(1)

Yes, I prefer this simpler approach also

 
R4tna C #:

Yes, I prefer this simpler approach also

I would say it depends very much on the use case.

If iterating in an EA on a high frequency symbol with hundreds of ticks, or while news, you do not have much time to spend on calculations.

Let's say you calculate the mean based on incoming bid prices and want to see the delta to most current price, calling CopyTicks, calculating the mean and then calculating the delta will take up a significant amount of time.

Another use case is calculating an MA ribbon, which is composed of maybe 15 seperaten MAs.

Another use case would be very long MAs.

And let's not forget, there is usually plenty more of work to do, besides calculating an MA on every OnTick() call.

I personally would say, an algorithm with O 1 compared to an algorithm with O n is always preferable.

Anyways, it's personal preference on how to do something.

Most algorithms don't need such solutions.



 
Dominik Christian Egert #:
Most algorithms don't need such solutions.



Yes exactly - one can start simple and test the performance - if it needs something better one can implement it.

It is just about getting the balance right between effort and benefit

 
Fernando Carreiro #:

He means that the time it takes to execute is pre-determined or almost constant. Usually this is achieved by doing the calculations incrementally, on each bar for example, so that the calculation are spread out over many bars instead of having to do the whole calculation all at once.

This is one of the reasons why the exponential moving average can be calculated so much faster than the simple moving average for increasingly larger periods. An exponential moving average always takes the same amount of time to calculate irrespective of the period, while a simple moving average (which is the case here) gets slower, and slower the longer its period.

EDIT: That is the reason why in his calculations he is keeping track of a ring buffer, so that the sum can be calculated incrementally on each bar.

Interesting. I didn't know EMA is calculated that way.

How would I need to adjust my EMA function to reflect that?

I understood the first value fed into the loop to calculate the EMA was an SMA to strat with.

Then as far as I understood, I need to iterate all values considered, because the next loop pass needs the result of the previous pass.

Here is my code to do that. 

#define ARRAY_INVALID_INDEX ((int)0xFFFFFFFE)


const int array_index(const int index, const int size)
{
    // Predefined cases
    switch(index)
    {
        // Return last element of array
        case -1:    return(size - 1);     // WHOLE_ARRAY

        // Return end of array
        case 0x00:  return((int)NULL);    // NULL
    }

    // Return validated index
    if( (index > NULL)
     && (index < size))
    { return(index); }

    // Move backwards on negative index value
    else if( ((size + index) < size)
          && ((size + index) > NULL))
    { return((size - 1) + index); }

    // Return failed index
    return(ARRAY_INVALID_INDEX);
}



template <typename T>
const T ema(const T &values[], const int _val_index = -1, const int periods = WHOLE_ARRAY, const T init_value = NULL)
{
    // Precheck input

        const int val_index = array_index(_val_index, ArraySize(values));
        if(val_index == ARRAY_INVALID_INDEX)
        { return(NULL); }

        if( (periods == NULL)
         || ((periods == 1) && (init_value == NULL)) )
        { return(values[val_index]); }


    // Local init

        const   bool    as_series   = ((ArrayIsDynamic(values)) && !(ArrayIsSeries(values)) && (ArrayGetAsSeries(values)) && (ArraySetAsSeries(values, false)));
        const   int     end         = ((periods > val_index) || (periods < 1)) ? val_index + 0x01 : periods;
        const   int     begin       = val_index - (end - 1);
        const   double  ema_f       = (2.0 / (end + 1));
                T       init_val    = (init_value != NULL) ? init_value : array_mean(values, begin - periods - 1, begin - 1);
                int     cnt         = NULL;

    // Calculate ema

        while( (cnt < end)
            && (!_StopFlag) )
        {
            // Next value
            init_val    = (T)((ema_f * values[begin + cnt]) + (1.0 - ema_f) * init_val);

            // Loop control
            cnt++;
        }


    // Restore array cfg
    (as_series && (ArraySetAsSeries(values, as_series)));

   // Return
    return(init_val);
}

// This is lend code....
template<typename T> T array_mean(const T& arr[], int iBeg, int iEnd){
   return array_sum(arr, iBeg, iEnd) / (iEnd - iBeg);
}
template<typename T> T array_sum(const T& arr[], int iBeg, int iEnd){
   T sum=0; for(; iBeg < iEnd; ++iBeg) sum += arr[iBeg]; 
   return sum;
}

I hope it compiles and I havent forgotten anything. I had to extract it from my library.


As far as I understand this, I would call the function once, and get a result, which is the EMA value. Then, on subsequent calls, I would feed in the result of the last call as init_val and use a period of 1 to calculate the next value. Is that right?



 
Dominik Christian Egert #:
I hope it compiles and I havent forgotten anything. I had to extract it from my library.

It does compile (in MQL5) - I suggest pasting it into a script and compiling it first before posting.

In fact better if you post a working script - makes it easier for others who wish to participate

 
R4tna C #:

It does compile (in MQL5)


Thank you for confirming.


 
Dominik Christian Egert #:
Here is what you need to calculate the mean of a series.

A variable to sum up the values.
An array with the length of the amount of values you want to consider.

I was thinking about this - if you have a variable to sum up the values, and another to count the values, you have the simplest/fastest (and deterministic) way to compute the mean.

Only 3 operations: 1. add the new value to the sum 2. increment the count 3. mean =  new sum divided by the new count.

Of course this does not retain a complete history of all values, but that is a simple matter of adding each new entry to an array, should one need it for deeper analysis.

Alternatively if one chooses to keep an incremental history array, just use ArraySize() instead of having a count variable, all of this would occur in microsecs anyway

 
R4tna C #:

I was thinking about this - if you have a variable to sum up the values, and another to count the values, you have the simplest/fastest (and deterministic) way to compute the mean.

Only 3 operations: 1. add the new value to the sum 2. increment the count 3. mean =  new sum divided by the new count.

Of course this does not retain a complete history of all values, but that is a simple matter of adding each new entry to an array, should one need it for deeper analysis.

Alternatively if one chooses to keep an incremental history array, just use ArraySize() instead of having a count variable, all of this would occur in microsecs anyway

I am not certain I understood that.

What I understood is that every new value, you add to the calculation, would also increase your length of the MA.

After all, if you have an array of length x and you want to calculate the mean on a portion of that array, you need to sum up all the values and divide by the amount of values you summed up. Now if you shift your scope by one, you need to do it all again.

If you keep the sum from the last iteration, all you need to do is subtract/remove the first value from your former scope and add the new value at the end from your current scope.

Et voila, your result.

But, this requires you to know the last scopes value being dropped. Hence, you need to track chi's value somehow. (Ie using a ring buffer)

There is a "dirty" way of doing a mean calculation. But it is mathematically wrong. You subtract, instead of the value dropped, the mean result and add the new value to your sum.

Some bad Trading view indicators do it like that, but it will always keep a shadow of former values in your equation.

The essential issue is in fact a limitation in MQLs OO model. Because you cannot bind an object to an external array. If that were possible, you could use the external array as a data feed, and wouldn't be required to use an internal ring buffer.

I did build such an object, but instead of binding to the external array, I use handles to store the relevant informations within the object, but it requires you to pass the correct array along with the handle to the xMA you want to get the result from.

Also not very perfect, but that's due to MQLs limitations on handling references and pointers. (Without using .dll imports)

123456
New comment