Bit mask for Pattern Usage

 

In the ExpertSignal Class of the Standard Library we can use the method PatternUsage (sets m_patterns_usage) if we want to use specific market models from the signal modules. The method has one parameter which is a bit mask.

I don't know how to set up the bit mask correctly. I found this thread which helped me a bit but not much.

In the LongCondition and ShortCondition methods of a module there is a check for every model if it should be used or not. The check is done by a macro called IS_PATTERN_USAGE which uses that bit mask. It looks like this:

//--- check if a market model is used
#define IS_PATTERN_USAGE(p)          ((m_patterns_usage&(((int)1)<<p))!=0)

I don't know much about bit masks but I think I know what this macro does: "Take the bit mask in m_patterns_usage and compare it with a sequence of 0s and one 1 at position p. If the bit mask has a 1 at position p, the macro returns true (pattern is used), otherwise false."

Now, if I want to use pattern 1 only, how should my bit mask look like?

int use_pattern=00000010; // ?

Or for pattern 1 and 3:

int use_pattern=00001010; // ?

This could be completly wrong and you might laugh now if you are an experienced developer, but as I said I don't know much about bit masks :)
So I need your help please.

 
yohmm:

In the ExpertSignal Class of the Standard Library we can use the method PatternUsage (sets m_patterns_usage) if we want to use specific market models from the signal modules. The method has one parameter which is a bit mask.

I don't know how to set up the bit mask correctly. I found this thread which helped me a bit but not much.

In the LongCondition and ShortCondition methods of a module there is a check for every model if it should be used or not. The check is done by a macro called IS_PATTERN_USAGE which uses that bit mask. It looks like this:

I don't know much about bit masks but I think I know what this macro does: "Take the bit mask in m_patterns_usage and compare it with a sequence of 0s and one 1 at position p. If the bit mask has a 1 at position p, the macro returns true (pattern is used), otherwise false."

Now, if I want to use pattern 1 only, how should my bit mask look like?

Or for pattern 1 and 3:

This could be completly wrong and you might laugh now if you are an experienced developer, but as I said I don't know much about bit masks :)
So I need your help please.

You are right, taking into consideration that first pattern is pattern 0.
 
angevoyageur:
You are right, taking into consideration that first pattern is pattern 0.

Yes, first is pattern 0.
Unfortunately, it doesn't work. When debugging, the parser enters not only if-clause for pattern 1, but for pattern 3 too (when using bit mask 00000010). As reference:

//+------------------------------------------------------------------+
//| "Voting" that price will fall.                                   |
//+------------------------------------------------------------------+
int CSignalRSI::ShortCondition(void)
  {
   int result=0;
   int idx   =StartIndex();
//---
   if(DiffRSI(idx)<0.0)
     {
      //--- the oscillator is directed downwards confirming the possibility of falling of price
      if(IS_PATTERN_USAGE(0))
         result=m_pattern_0;      // "confirming" signal number 0
      //--- if the model 1 is used, search for a reverse of the oscillator downwards behind the level of overbuying
      if(IS_PATTERN_USAGE(1) && DiffRSI(idx+1)>0.0 && RSI(idx+1)>70.0)
         result=m_pattern_1;      // signal number 1
      //--- if the model 2, 3, 4 or 5 is used, perform the extended analysis of the oscillator state
      if(IS_PATTERN_USAGE(2) || IS_PATTERN_USAGE(3) || IS_PATTERN_USAGE(4) || IS_PATTERN_USAGE(5))
        {
         ExtStateRSI(idx);
         //--- search for the "failed swing" signal
         if(IS_PATTERN_USAGE(2) && RSI(idx)<m_extr_osc[1])
            result=m_pattern_2;   // signal number 2
         //--- search for the "divergence" signal
         if(IS_PATTERN_USAGE(3) && CompareMaps(1,1)) // 0000 0001b
            result=m_pattern_3;   // signal number 3
         //--- search for the "double divergence" signal
         if(IS_PATTERN_USAGE(4) && CompareMaps(0x11,2)) // 0001 0001b
            return(m_pattern_4);  // signal number 4
         //--- search for the "head/shoulders" signal
         if(IS_PATTERN_USAGE(5) && CompareMaps(0x62662,5,true) && RSI(idx)<m_extr_osc[1]) // 01100010011001100010b
            result=m_pattern_5;   // signal number 5
        }
     }
//--- return the result
   return(result);
  }

Debugger shows that m_patterns_usage is 10 at this point => preceding 0s in the sequence are cut off. So I set m_patterns_usage to 10000010 to keep the mask "intact", but that doesn't make any difference.

What's wrong?

 
yohmm:

Yes, first is pattern 0.
Unfortunately, it doesn't work. When debugging, the parser enters not only if-clause for pattern 1, but for pattern 3 too (when using bit mask 00000010). As reference:

Debugger shows that m_patterns_usage is 10 at this point => preceding 0s in the sequence are cut off. So I set m_patterns_usage to 10000010 to keep the mask "intact", but that doesn't make any difference.

What's wrong?

How are you defining the pattern(s) to use ?
 
angevoyageur:
How are you defining the pattern(s) to use ?

 MyExpert.mq5 (only the relevant parts, without any error checks etc.):

#include <MyExpert/Expert.mqh>
#include <MyExpert/Filter/FilterRSI.mqh>

//--- Some init stuff
...
//--- Create signal
int use_pattern=00000010;
CExpertSignal *signal=new CExpertSignal(use_pattern);
//--- Initialize signal
...
//--- Attach filter
CFilterRSI *filter0=new CFilterRSI;
signal.AddFilter(filter0);
//--- Set filter parameters
...
//--- Add signal to the collection
Expert.AddSignal(signal);

 Constructor in ExpertSignal.mqh:

CExpertSignal::CExpertSignal(int patt=-1) : 
                                     m_patterns_usage(patt)
                                     // some more members init
  {
  }

AddFilter() in ExpertSignal.mqh:

//+------------------------------------------------------------------+
//| Setting an additional filter                                     |
//+------------------------------------------------------------------+
bool CExpertSignal::AddFilter(CExpertSignal *filter)
  {
//--- check pointer
...
//--- add the filter to the array of filters
   if(!m_filters.Add(filter))
      return(false);
   filter.PatternsUsage(m_patterns_usage);   // <- "copy" pattern usage from the "parent"(=the signal this filter is attached to)
//--- succeed                                      now  this filter "knows" what model to use
   return(true);
  }

Everything is set up.
In OnTick CheckSignals (see below) is called and it checks if any of the signals (sender/broadcaster) has a new signal. It is doing this by iterating over the collection of signals (m_signals) and calls Direction() on every signal. Direction method then iterates over all filters on that signal.

//+------------------------------------------------------------------+
//| Check for new signals                                            |
//+------------------------------------------------------------------+
bool CExpert::CheckSignals(void)
  {
   int total=m_signals.Total();
   for(int i=0;i<total;i++)
     {
      CExpertSignal *signal   =m_signals.At(i);      // m_signals <- collection of signals
      double         direction=signal.Direction();   // Direction in ExpertSignals.mqh.
      //--- the "prohibition" signal                 //    interates every filter that is appended to the signal
      if(direction==EMPTY_VALUE)
         continue;
      //--- check of exceeding the threshold value
      if(direction>=m_threshold)
        {
         Alert(direction," UP");
        }
      if(-direction>=m_threshold)
        {
         Alert(direction," DOWN");
        }
     }
//--- return without operations
   return(false);
  }

And here we are again in Long/ShortCondition. Debugger confirms correct m_pattern_usage at this point.

 
yohmm:

 MyExpert.mq5 (only the relevant parts, without any error checks etc.):

 Constructor in ExpertSignal.mqh:

AddFilter() in ExpertSignal.mqh:

Everything is set up.
In OnTick CheckSignals (see below) is called and it checks if any of the signals (sender/broadcaster) has a new signal. It is doing this by iterating over the collection of signals (m_signals) and calls Direction() on every signal. Direction method then iterates over all filters on that signal.

And here we are again in Long/ShortCondition. Debugger confirms correct m_pattern_usage at this point.

Wow...seems very complicated.

Never change directly a file from the standard library. Your changes will be lost on the next update of MT5 (it seems you make a copy which is even worst as if standard library is improved later, you don't get this improvements). Simply define your own classes derived from standard library.

class CMyExpertSignal : public CExpertSignal

By the way, it's not needed to define which pattern to use. We were talking about SignalRSI. Each module as its own patterns, so you only have to define it with existing method :

signal.AddFilter(filter0);
//--- Set filter parameters
filter0.PatternsUsage(2);         // Add this line
filter0.Symbol("GBPUSD");
filter0.Period(PERIOD_M1);
filter0.PeriodRSI(Signal_RSI_PeriodRSI);
filter0.Applied(Signal_RSI_Applied);
filter0.Weight(Signal_RSI_Weight);

Pattern value like "0000010" is a binary number, you can't write it as is, you have to write it  :

int use_pattern=00000010; 2;

What you write here is the decimal value 10, which is 1010b in binary notation. With mql5 you can write number in decimal, or hexadecimal (0x2128 for example), but not directly in binary notation.

 

Thanks Alain for your answer to yohmm question.

Can you help me find where I'm mistaking concerning the logic behind :

//--- check if a market model is used
#define IS_PATTERN_USAGE(p)          ((m_patterns_usage&(((int)1)<<p))!=0)

Case p=0: I believe that (((int)1)<<p) renders nothing else than decimal 1, or the 32 bit (int as four bytes ) : 0000 0000 0000 0000 0000 0000 0000 0001             ???

As without any custom setting of PatternsUsage(int)  m_patterns_usage=-1 = 1111 1111 1111 1111 1111 1111 1111 1111  all patterns in the signal class are used.

Let's assume that the user mistakes setting m_patterns_usage=0 thinking it'll restraint signal screening to pattern 0.

Then m_patterns_usage=0=0000 0000 0000 0000 0000 0000 0000 0000

(((int)1)<<0) goes as 0000 0000 0000 0000 0000 0000 0000 0001 

And bitwise OP ( m_patterns_usage & (((int)1)<<0)) gives  0000 0000 0000 0000 0000 0000 0000 0000. So the final test (0!=0) comes 'false' and no screening will be executed whatever p come to be in the test.

In fact we thus achieved the 'Hot ON/OFF Toggle ' mode of screening if some was looking after.

Hope there's no clue in my understanding until now.

Let's say that we set PatternsUsage() like :

filter0.PatternsUsage(2);         // Add this line

Then m_patterns_usage=0000 0000 0000 0000 0000 0000 0000 0010   ??? (Ok ?)

At execution IS_PATTERN_USAGE(2checks  ((m_patterns_usage&(((int)1)<<2))!=0)

Operation (((int)1)<<2) shifts 2 bits left in the binary mask of (int 1) decimal value :

0000 0000 0000 0000 0000 0000 0000 0001 << 2 = 0000 0000 0000 0000 0000 0000 0000 0100 (=4 in decimal, if this does not seem appealing for what we're seeking as of know this will be the trick as I think later, the '1' is in 3rd position ( 2 as of 0 BASE COUNT, but this trashes the decimal window logic when setting filter0.PatternsUsage(2) )

Then bitwise & on both parts brings:

0000 0000 0000 0000 0000 0000 0000 0010

&

0000 0000 0000 0000 0000 0000 0000 0100

=

0000 0000 0000 0000 0000 0000 0000 0000  = 0

As far as I do understand, test (0)!=0 brings 'false' and IS_PATTERN_USAGE(2) in fact voids pattern 2 screening in signal checking process !

Obviously, this is not what the user expected when customizing filter0.PatternsUsage(2), symmetrically his tweaking expected that only Pattern 2 checking shall be processed. Or am I wrong with that ?

May be I'm mistaking on some steps, but it comes to my opinion that shifting p bits left in the bit mask of 1 with ((int)1<<p), does not make the deal or involves a sideview to avoid conflicts !


Simply speaking as one "touches" m_patterns_usage from its defaults value (-1), the core logic is broken as the correspondence expected in mapping the '1' at position (p) in its bit mask with the '1' at  the same position in the bit mask of ((int)1)<<p) vanishes.

As of that tweaking m_patterns_usage can be achieved with a strict binary mapping target in mind, and not with a decimal setting stand view.
User must view pattern 2 bit toggling as bit number 3 in m_patterns_usage bit mask, not decimal position 2 in BASE 0 count as it is pretended in the code.
Setting m_patterns_usage=2 with filter0.PatternsUsage(2) sets (second=2?) bit '1' in its bitmask and (first=1?) bit at '1' by the same  way...does this strictly activate pattern 2 ? Of course not, Pattern 1 gets also activated for screening.
At execution time the test ((int)1)<<p) deals with bit p=2 being the third from right in the bit mask obtained, underlining the mismatch.

If one wants to exactly adress Pattern 2, then he should set
filter0.PatternsUsage(((int)1)<<2)) and disregards interpretation of decimal value of m_patterns_usage.
As of that m_patterns_usage=00000000000000000000000000000100 and Pattern 0 and Pattern 1 are not toggled in side effect of decimal value inputting like filter0.PatternsUsage(2)
The user could have set filter0.PatternsUsage(4), thus forcing the bit mask of m_patterns_usage to the correct processing flags 00000000000000000000000000000100.

But how would he knows that the mapping of decimal value logic in filter0.PatternsUsage(int value) and the internal testing of Pattern Usage implies such slights discrepancies ?

A bitwise AND on the expression (p & ( ((int) 1)<<p ) ) does not always "verify" that pattern(int p) is in use as the user may think when tweaking m_patterns_usage setting with filter0.PatternsUsage(int value p).

Simply because bit mask of integer p does not imply a '1' at position p (right/wrong ?), as this test asserts it to be (right/wrong ?)

Please, may be I'm completely wrong so I seek your help in understanding where all this is just mind garbage...and I apologize for wasting your time... feel free to HELP !!!

Otherwise in case this "sideview" underline meaningful insights, we then have a bunch of opportunities in signal screening, and I personally thanks Metaquotes for bringing it !

If (asserts) sizeof(int)=4 bytes, m_patterns_usage is then a 32 bit length bitmask, allowing 32^2 permutations of these 32 bits assuming their 2 states.

We are far from the 32 or 64 "filters" for a signal and this brings an impressive number of possibilities...some have been asking about the "power" of bitmasks and bitwise operators...hope this brings some light on it.

So how customizing m_patterns_usage from its default value -1 the right way ???

  1. start with
    int use_pattern=0;    // This way no clue, all patterns are dismissed
  2. Then add as much as required interceptions for the pattern you want to be screened :
    use_pattern|=((int)1)<<p);     // This Sets Only Pattern p, without side effects of mangling directly with use_pattern|=p

If we want to customize to something like "just play patterns 1 3 5" user can do something like :

int use_pattern=0;           //(0= 0000 0000 0000 0000 0000 0000 0000 0000)    => CAUTION : This CLEAR the mask, NO PATTERN SCREENED
use_pattern|=1;              //(1= 0000 0000 0000 0000 0000 0000 0000 0001)    => use_pattern=0000 0000 0000 0000 0000 0000 0000 0001 <= THIS SETS PATTERN 0 /1st PATTERN
Above line would activate Pattern 0 the 1st Pattern not the Pattern N°1, 2nd in the bitmask
The line bellow does what wanted exactly :
use_pattern|=((int)1)<<1);   //(-> 0000 0000 0000 0000 0000 0000 0000 0010)    => use_pattern=0000 0000 0000 0000 0000 0000 0000 0010 <= THIS SETS PATTERN 1 /2nd PATTERN

use_pattern|=((int)1)<<3);   //(3= 0000 0000 0000 0000 0000 0000 0000 1000)    => use_pattern=0000 0000 0000 0000 0000 0000 0000 1010 <= appends PATTERN 3 /4th PATTERN
use_pattern|=((int)1)<<5);   //(5= 0000 0000 0000 0000 0000 0000 0010 0000)    => use_pattern=0000 0000 0000 0000 0000 0000 0010 1010 <= appends PATTERN 5 /6th PATTERN
//The use_pattern bit mask in use is now 0000 0000 0000 0000 0000 0000 0010 1010
//Use it to set the filter for signal screening :
filter0.PatternsUsage(use_pattern);

That whay the USE_PATTERN(p) for p other than (1,3,5) always raise 'false', only patterns (1,3,5) will be fired by the test.


Thanks for reading, and let me know, cause if this works just think about what we can do to MT5 classes for REAL multi-timeframes/multi-pairs all-in-one EA, with a few tweaks by bitmasking and not so much lines of code.

I still prefer MT4 for hedging-like strategies and its wonderfull CloseBy(), but I need C++ mind power of MT5 classes...see what I'm doing ?

If someone is in that process, please come in touch !

 

If you want use a set of (a, b, c,...):

m_patterns_usage=(1<<a)|(1<<b)|(1<<c)...;

 
Don't hard code shifts, use an enum with powers of two.
Note the use of Bitwise Operations and (&) not the Boolean Operations and (&&)


Enum Pattern = {A=1, B=2, C=4 ...};

Pattenn a = A,
        b = B,
        c = C,
        aAndb = A+B;
        all3  = A+B+C;

if(aAndb & C == 0) Print("No C");
if(aAndb & B)      Print("aAndB includes B");
 

whroeder1:

Don't hard code shifts, use an enum with powers of two.
Note the use ofBitwise Operationsand(&) not theBoolean Operationsand(&&)


Enum Pattern = {A=1, B=2, C=4 ...};

Pattenn a = A,
        b = B,
        c = C,
        aAndb = A+B;
        all3  = A+B+C;

if(aAndb & C == 0) Print("No C");
if(aAndb & B)      Print("aAndB includes B");

You can't add enums together that way (A+B) if you plan to store the result in an enum variable.

You would need to use more parentheses on the first 'if' statement to avoid a compiler warning, and cast the second 'if' statement as a bool to avoid another compiler warning.

 
Nguyen Nhu Phan:

If you want use a set of (a, b, c,...):

m_patterns_usage=(1<<a)|(1<<b)|(1<<c)...;

thanks!