
Build Self Optimizing Expert Advisors in MQL5 (Part 6): Self Adapting Trading Rules (II)
In our last discussion on self adapting trading rules, linked here, we considered the problems faced by an algorithmic trader attempting to follow best practices on how to follow the RSI indicator.
We found that the standardized results aren’t always generated by the indicator, depending on several factors, such as the period, time frame and also the particular market in question.
To resolve this issue, we postulated that algorithmic traders could rather study the true range of the indicator, so they can readjust the midpoint of the indicator, to the middle of its observed range, and not its total possible range. Doing so, gives us some guarantees about the generation of trading signals, that we cannot obtain from the traditional rules of the RSI. We gained additional control over the new signal by recording an average deviation from the mid-point, and only registering signals generated by multiples of the average deviation.
We shall now advance beyond our initial attempt to build a practical solution. There are several improvements we can make over our last attempt. The integral improvement we seek is the ability to try to estimate the value of our chosen RSI levels.
In our last discussion, we simply assumed that deviations significantly larger than the average deviation could tend to be more profitable. However, we did not attempt to measure if this was true. We made no attempt to quantify the value of the new levels we are proposing and compare them against the value of the traditional levels, 70 and 30.
Additionally, our last discussion considered the case where the RSI period was fixed. This simplifying assumption made our framework easier to understand. Today, we turn our attention to the opposite end of the problem, when the practitioner is uncertain of the right period to use.
Visualizing The Problem
In case we have new readers joining us, In Fig 1 below, we have attached a screenshot of the Daily EURUSD chart with a 2 period RSI.
Fig 1: Visualizing our RSI the quality of signals generated by a short period.
Underneath Fig 1, we have applied a 70 period RSI to the same section of the chart in Fig 2. We can observe the RSI signal is slowly becoming a flat line centered along the 50 RSI level. What comments could we make when we compare the 2 pictures? Well, one comment worth mentioning is that, over the period of the EURUSD captured in both figures, the exchange rate simply fell, from the 1.121 price level on 18 September 2024 to lows of 1.051 by 2 December 2024. However, the 2 period RSI changed levels too frequently over the same time, and the 70 period RSI didn’t change levels at all.
Does this mean that traders should be forever limited to only using a narrow range of periods when employing the RSI? What will it take for us, to design algorithms that automatically select a good RSI period, without human intervention? Additionally, how can we write algorithms, to help us find good trading levels, no matter which period we initially start with?
Fig 2: Visualizing our RSI indicator operating with a long period.
Getting Started in MQL5
There are many ways we could approach this problem. We could use libraries in Python, to generate RSI readings with different periods, and optimize our period and RSI levels that way. However, this introduces possible drawbacks. The greatest limitation could potentially come from slight differences behind the computation being implemented for calculating these technical indicator values.To avoid this, we will implement our solution in MQL5. By building an RSI class, we can quickly record multiple instances of RSI values into one CSV file, and use these values to perform our numerical analysis in Python to estimate an optimal RSI period, and alternative levels to use besides 70 and 30.
We'll start off by creating a script, that allows us to first manually retrieve RSI values and calculate the change in RSI levels. Then, we will create a class, that encapsulates the functionality we need. We want to create a grid of RSI readings with periods incrementing in steps of 5, from 5 until 70. But before we can achieve this, we need to implement and test our class.
Building the class in a script will allow us to quickly test the output from the class, against the standard output manually obtained from the indicator. If we have specified the class well, the output generated by both methods should be the same. This will give us a useful class for generating and 14 RSI indicators with different periods, while keeping track of the change in each instance of the RSI, over any other symbol we may wish to trade.
Given the use we want to make of this RSI class, it makes sense to start making sure that the class has a mechanism that prevents us from attempting to read the indicator's value before we have set the buffer accordingly. We will start by building this part of our class first. We need to declare private members of our class. These private boolean flags will prevent us from reading RSI values, before we copy from the indicator buffer.
//+------------------------------------------------------------------+ //| ProjectName | //| Copyright 2020, CompanyName | //| http://www.companyname.net | //+------------------------------------------------------------------+ #property copyright "Copyright 2024, MetaQuotes Ltd." #property link "https://www.mql5.com" #property version "1.00" #property script_show_inputs //--- The RSI class will manage our indicator settings and provide useful transformations we need class RSI { //--- Private members private: //--- Have the indicator values been copied to the buffer? bool indicator_values_initialized; bool indicator_differenced_values_initialized;
I have also included a method that returns strings to prompt the user what is going on within the object, and how to resolve issues. The method takes an integer parameter which informs where in the code the error was generated. Thus, the solutions are normally easy to suggest as a printed message in the terminal.
//--- Give the user feedback string user_feedback(int flag) { string message; //--- Check if the RSI indicator loaded correctly if(flag == 0) { //--- Check the indicator was loaded correctly if(IsValid()) message = "RSI Indicator Class Loaded Correcrtly \n"; return(message); //--- Something went wrong message = "Error loading RSI Indicator: [ERROR] " + (string) GetLastError(); return(message); } //--- User tried getting indicator values before setting them if(flag == 1) { message = "Please set the indicator values before trying to fetch them from memory"; return(message); } //--- We sueccessfully set our differenced indicator values if(flag == 2) { message = "Succesfully set differenced indicator values."; return(message); } //--- Failed to set our differenced indicator values if(flag == 3) { message = "Failed to set our differenced indicator values: [ERROR] " + (string) GetLastError(); return(message); } //--- The user is trying to retrieve an index beyond the buffer size and must update the buffer size first if(flag == 4) { message = "The user is attempting to use call an index beyond the buffer size, update the buffer size first"; return(message); } //--- No feedback else return(""); }
Now we shall define protected members of our class. These members will constitute the moving parts needed to initialize an instance of the iRSI() class, and interact with the indicator buffer.
//--- Protected members protected: //--- The handler for our RSI int rsi_handler; //--- The Symbol our RSI should be applied on string rsi_symbol; //--- Our RSI period int rsi_period; //--- How far into the future we wish to forecast int forecast_horizon; //--- The buffer for our RSI indicator double rsi_reading[]; vector rsi_differenced_values; //--- The current size of the buffer the user last requested int rsi_buffer_size; int rsi_differenced_buffer_size; //--- The time frame our RSI should be applied on ENUM_TIMEFRAMES rsi_time_frame; //--- The price should the RSI be applied on ENUM_APPLIED_PRICE rsi_price;
Moving on to the public class members. The first and function we will require, informs us if our indicator handler is valid. If our indicator handler is not set up correctly, we can inform the user right away.
//--- Now, we can define public members: public: //--- Check if our indicator handler is valid bool IsValid(void) { return((this.rsi_handler != INVALID_HANDLE)); }
Our default constructor will create an iRSI object set to the EURUSD on the Daily chart, for a period of 5 Days. To make sure this is the user's intended choice, our class prints out which market and period it is working with. Additionally, the default constructor specifically prints out to the user that the current instance of the RSI object was built by the default constructor.
//--- Our default constructor void RSI(void): indicator_values_initialized(false), rsi_symbol("EURUSD"), rsi_time_frame(PERIOD_D1), rsi_period(5), rsi_price(PRICE_CLOSE), rsi_handler(iRSI(rsi_symbol,rsi_time_frame,rsi_period,rsi_price)) { //--- Give the user feedback on initilization Print(user_feedback(0)); //--- Remind the user they called the default constructor Print("Default Constructor Called: ",__FUNCSIG__," ",&this); }
Otherwise, we expect the user to call the parametric constructor for the RSI object and specify all the necessary parameters you need.
//--- Parametric constructor void RSI(string user_symbol,ENUM_TIMEFRAMES user_time_frame,int user_period,ENUM_APPLIED_PRICE user_price) { indicator_values_initialized = false; rsi_symbol = user_symbol; rsi_time_frame = user_time_frame; rsi_period = user_period; rsi_price = user_price; rsi_handler = iRSI(rsi_symbol,rsi_time_frame,rsi_period,rsi_price); //--- Give the user feedback on initilization Print(user_feedback(0)); }
We will also need a destructor to free up the system resources we no longer need, allowing us to clean up after ourselves.
//--- Destructor void ~RSI(void) { //--- Free up resources we don't need and reset our flags if(IndicatorRelease(rsi_handler)) { indicator_differenced_values_initialized = false; indicator_values_initialized = false; Print("RSI System logging off"); } }
Now the methods needed to interact with the indicator buffer, are the key component of our class. We ask the user to specify how many values should be copied from the buffer, and whether the values should be arranged as series. We then test to make sure the RSI values are not returning to us null, before the termination of the method call.
//--- Copy readings for our RSI indicator bool SetIndicatorValues(int buffer_size,bool set_as_series) { rsi_buffer_size = buffer_size; CopyBuffer(this.rsi_handler,0,0,buffer_size,rsi_reading); if(set_as_series) ArraySetAsSeries(this.rsi_reading,true); indicator_values_initialized = true; //--- Did something go wrong? vector rsi_test; rsi_test.CopyIndicatorBuffer(rsi_handler,0,0,buffer_size); if(rsi_test.Sum() == 0) return(false); //--- Everything went fine. return(true); }
A simple function to get the current RSI reading.
//--- Get the current RSI reading double GetCurrentReading(void) { double temp[]; CopyBuffer(this.rsi_handler,0,0,1,temp); return(temp[0]); }
We may also need to fetch values at a specific index, not just the most recent value. This function, GetReadingAt(), renders this utility for us. The function first checks to make sure we are not trying to go beyond the size of the buffer we copied from the indicator. If used correctly, the function will return the indicator reading at the specified index. Otherwise, an error message will be given.
//--- Get a specific RSI reading double GetReadingAt(int index) { //--- Is the user trying to call indexes beyond the buffer? if(index > rsi_buffer_size) { Print(user_feedback(4)); return(-1e10); } //--- Get the reading at the specified index if((indicator_values_initialized) && (index < rsi_buffer_size)) return(rsi_reading[index]); //--- User is trying to get values that were not set prior else { Print(user_feedback(1)); return(-1e10); } }
Our interest also lies in the change in RSI values. It is not enough for us to have access to the current RSI reading. We also want access to the change in RSI levels that is happening over any arbitrary window size we specify. As before, we simply call the CopyBuffer functions for the user behind the scenes to calculate the growth in RSI levels, but the class also includes an additional check to make sure that the output of the calculation is not a vector of 0 before returning the answer it found to the user.
//--- Let's set the conditions for our differenced data bool SetDifferencedIndicatorValues(int buffer_size,int differencing_period,bool set_as_series) { //--- Internal variables rsi_differenced_buffer_size = buffer_size; rsi_differenced_values = vector::Zeros(rsi_differenced_buffer_size); //--- Prepare to record the differences in our RSI readings double temp_buffer[]; int fetch = (rsi_differenced_buffer_size + (2 * differencing_period)); CopyBuffer(rsi_handler,0,0,fetch,temp_buffer); if(set_as_series) ArraySetAsSeries(temp_buffer,true); //--- Fill in our values iteratively for(int i = rsi_differenced_buffer_size;i > 1; i--) { rsi_differenced_values[i-1] = temp_buffer[i-1] - temp_buffer[i-1+differencing_period]; } //--- If the norm of a vector is 0, the vector is empty! if(rsi_differenced_values.Norm(VECTOR_NORM_P) != 0) { Print(user_feedback(2)); indicator_differenced_values_initialized = true; return(true); } indicator_differenced_values_initialized = false; Print(user_feedback(3)); return(false); }
Lastly, we need a method to fetch the differenced RSI value at a specific index value. Again, our function makes sure the user is not trying to call beyond the range of the buffer copied. In such a case, the user should first update the buffer size, then copy their desired index value accordingly.
//--- Get a differenced value at a specific index double GetDifferencedReadingAt(int index) { //--- Make sure we're not trying to call values beyond our index if(index > rsi_differenced_buffer_size) { Print(user_feedback(4)); return(-1e10); } //--- Make sure our values have been set if(!indicator_differenced_values_initialized) { //--- The user is trying to use values before they were set in memory Print(user_feedback(1)); return(-1e10); } //--- Return the differenced value of our indicator at a specific index if((indicator_differenced_values_initialized) && (index < rsi_differenced_buffer_size)) return(rsi_differenced_values[index]); //--- Something went wrong. return(-1e10); } };
Building out the rest of our test is straightforward. We will manually control a similar instance of the RSI indicator, initialized with the same settings. If we write out both readings to the same file, we should observe duplicated information. Otherwise, we would've made a mistake in our implementation of the class.
//--- How far we want to forecast #define HORIZON 10 //--- Our handlers for our indicators int rsi_5_handle; //--- Data structures to store the readings from our indicators double rsi_5_reading[]; //--- File name string file_name = Symbol() + " Testing RSI Class.csv"; //--- Amount of data requested input int size = 3000;
For the rest of our script, we need only to initialize our RSI class and set it up with the same parameters we will use with a duplicated but manually controlled version of the RSI.
//+------------------------------------------------------------------+ //| Our script execution | //+------------------------------------------------------------------+ void OnStart() { //--- Testing the RSI Class //--- Initialize the class RSI my_rsi(Symbol(),PERIOD_CURRENT,5,PRICE_CLOSE); my_rsi.SetIndicatorValues(size,true); my_rsi.SetDifferencedIndicatorValues(size,10,true); //---Setup our technical indicators rsi_5_handle = iRSI(Symbol(),PERIOD_CURRENT,5,PRICE_CLOSE); int fetch = size + (2 * HORIZON); //---Set the values as series CopyBuffer(rsi_5_handle,0,0,fetch,rsi_5_reading); ArraySetAsSeries(rsi_5_reading,true); //---Write to file int file_handle=FileOpen(file_name,FILE_WRITE|FILE_ANSI|FILE_CSV,","); for(int i=size;i>=1;i--) { if(i == size) { FileWrite(file_handle,"Time","RSI 5","RSI 5 Class","RSI 5 Difference","RSI 5 Class Difference"); } else { FileWrite(file_handle, iTime(_Symbol,PERIOD_CURRENT,i), rsi_5_reading[i], my_rsi.GetReadingAt(i), rsi_5_reading[i] - rsi_5_reading[i + HORIZON], my_rsi.GetDifferencedReadingAt(i) ); } } //--- Close the file FileClose(file_handle); } //+------------------------------------------------------------------+ #undef HORIZON
If our class has been implemented correctly, our test script will produce a file titled "EURUSD Testing RSI Class" that will contain duplicated RSI readings. As you can see from Fig 3, our RSI class passed our test. This class saves us time, from not having to implement the same methods over multiple times, across many projects. We can simply import our RSI class, and call the methods we need.
Fig 3: Our RSI class has passed our test, using it is identical to manually working with the RSI indicator class.
Now that we are confident in our implementation of the class, let us write out the class in a dedicated include file, so we can share the class with our Expert Advisor and any other exercises we conduct in future that may require the same set of functionality. When fully composed, this is what our class looks like in its current form.
//+------------------------------------------------------------------+ //| RSI.mqh | //| Gamuchirai Ndawana | //| https://www.mql5.com/en/users/gamuchiraindawa | //+------------------------------------------------------------------+ #property copyright "Gamuchirai Ndawana" #property link "https://www.mql5.com/en/users/gamuchiraindawa" #property version "1.00" //+------------------------------------------------------------------+ //| This class will provide us with usefull functionality | //+------------------------------------------------------------------+ class RSI { private: //--- Have the indicator values been copied to the buffer? bool indicator_values_initialized; bool indicator_differenced_values_initialized; //--- Give the user feedback string user_feedback(int flag); protected: //--- The handler for our RSI int rsi_handler; //--- The Symbol our RSI should be applied on string rsi_symbol; //--- Our RSI period int rsi_period; //--- How far into the future we wish to forecast int forecast_horizon; //--- The buffer for our RSI indicator double rsi_reading[]; vector rsi_differenced_values; //--- The current size of the buffer the user last requested int rsi_buffer_size; int rsi_differenced_buffer_size; //--- The time frame our RSI should be applied on ENUM_TIMEFRAMES rsi_time_frame; //--- The price should the RSI be applied on ENUM_APPLIED_PRICE rsi_price; public: RSI(); RSI(string user_symbol,ENUM_TIMEFRAMES user_time_frame,int user_period,ENUM_APPLIED_PRICE user_price); ~RSI(); bool SetIndicatorValues(int buffer_size,bool set_as_series); bool IsValid(void); double GetCurrentReading(void); double GetReadingAt(int index); bool SetDifferencedIndicatorValues(int buffer_size,int differencing_period,bool set_as_series); double GetDifferencedReadingAt(int index); }; //+------------------------------------------------------------------+ //| Our default constructor for our RSI class | //+------------------------------------------------------------------+ void RSI::RSI() { indicator_values_initialized = false; rsi_symbol = "EURUSD"; rsi_time_frame = PERIOD_D1; rsi_period = 5; rsi_price = PRICE_CLOSE; rsi_handler = iRSI(rsi_symbol,rsi_time_frame,rsi_period,rsi_price); //--- Give the user feedback on initilization Print(user_feedback(0)); //--- Remind the user they called the default constructor Print("Default Constructor Called: ",__FUNCSIG__," ",&this); } //+------------------------------------------------------------------+ //| Our parametric constructor for our RSI class | //+------------------------------------------------------------------+ void RSI::RSI(string user_symbol,ENUM_TIMEFRAMES user_time_frame,int user_period,ENUM_APPLIED_PRICE user_price) { indicator_values_initialized = false; rsi_symbol = user_symbol; rsi_time_frame = user_time_frame; rsi_period = user_period; rsi_price = user_price; rsi_handler = iRSI(rsi_symbol,rsi_time_frame,rsi_period,rsi_price); //--- Give the user feedback on initilization Print(user_feedback(0)); } //+------------------------------------------------------------------+ //| Our destructor for our RSI class | //+------------------------------------------------------------------+ void RSI::~RSI() { //--- Free up resources we don't need and reset our flags if(IndicatorRelease(rsi_handler)) { indicator_differenced_values_initialized = false; indicator_values_initialized = false; Print(user_feedback(5)); } } //+------------------------------------------------------------------+ //| Get our current reading from the RSI indicator | //+------------------------------------------------------------------+ double RSI::GetCurrentReading(void) { double temp[]; CopyBuffer(this.rsi_handler,0,0,1,temp); return(temp[0]); } //+------------------------------------------------------------------+ //| Set our indicator values and our buffer size | //+------------------------------------------------------------------+ bool RSI::SetIndicatorValues(int buffer_size,bool set_as_series) { rsi_buffer_size = buffer_size; CopyBuffer(this.rsi_handler,0,0,buffer_size,rsi_reading); if(set_as_series) ArraySetAsSeries(this.rsi_reading,true); indicator_values_initialized = true; //--- Did something go wrong? vector rsi_test; rsi_test.CopyIndicatorBuffer(rsi_handler,0,0,buffer_size); if(rsi_test.Sum() == 0) return(false); //--- Everything went fine. return(true); } //+--------------------------------------------------------------+ //| Let's set the conditions for our differenced data | //+--------------------------------------------------------------+ bool RSI::SetDifferencedIndicatorValues(int buffer_size,int differencing_period,bool set_as_series) { //--- Internal variables rsi_differenced_buffer_size = buffer_size; rsi_differenced_values = vector::Zeros(rsi_differenced_buffer_size); //--- Prepare to record the differences in our RSI readings double temp_buffer[]; int fetch = (rsi_differenced_buffer_size + (2 * differencing_period)); CopyBuffer(rsi_handler,0,0,fetch,temp_buffer); if(set_as_series) ArraySetAsSeries(temp_buffer,true); //--- Fill in our values iteratively for(int i = rsi_differenced_buffer_size;i > 1; i--) { rsi_differenced_values[i-1] = temp_buffer[i-1] - temp_buffer[i-1+differencing_period]; } //--- If the norm of a vector is 0, the vector is empty! if(rsi_differenced_values.Norm(VECTOR_NORM_P) != 0) { Print(user_feedback(2)); indicator_differenced_values_initialized = true; return(true); } indicator_differenced_values_initialized = false; Print(user_feedback(3)); return(false); } //--- Get a differenced value at a specific index double RSI::GetDifferencedReadingAt(int index) { //--- Make sure we're not trying to call values beyond our index if(index > rsi_differenced_buffer_size) { Print(user_feedback(4)); return(-1e10); } //--- Make sure our values have been set if(!indicator_differenced_values_initialized) { //--- The user is trying to use values before they were set in memory Print(user_feedback(1)); return(-1e10); } //--- Return the differenced value of our indicator at a specific index if((indicator_differenced_values_initialized) && (index < rsi_differenced_buffer_size)) return(rsi_differenced_values[index]); //--- Something went wrong. return(-1e10); } //+------------------------------------------------------------------+ //| Get a reading at a specific index from our RSI buffer | //+------------------------------------------------------------------+ double RSI::GetReadingAt(int index) { //--- Is the user trying to call indexes beyond the buffer? if(index > rsi_buffer_size) { Print(user_feedback(4)); return(-1e10); } //--- Get the reading at the specified index if((indicator_values_initialized) && (index < rsi_buffer_size)) return(rsi_reading[index]); //--- User is trying to get values that were not set prior else { Print(user_feedback(1)); return(-1e10); } } //+------------------------------------------------------------------+ //| Check if our indicator handler is valid | //+------------------------------------------------------------------+ bool RSI::IsValid(void) { return((this.rsi_handler != INVALID_HANDLE)); } //+------------------------------------------------------------------+ //| Give the user feedback on the actions he is performing | //+------------------------------------------------------------------+ string RSI::user_feedback(int flag) { string message; //--- Check if the RSI indicator loaded correctly if(flag == 0) { //--- Check the indicator was loaded correctly if(IsValid()) message = "RSI Indicator Class Loaded Correcrtly \nSymbol: " + (string) rsi_symbol + "\nPeriod: " + (string) rsi_period; return(message); //--- Something went wrong message = "Error loading RSI Indicator: [ERROR] " + (string) GetLastError(); return(message); } //--- User tried getting indicator values before setting them if(flag == 1) { message = "Please set the indicator values before trying to fetch them from memory, call SetIndicatorValues()"; return(message); } //--- We sueccessfully set our differenced indicator values if(flag == 2) { message = "Succesfully set differenced indicator values."; return(message); } //--- Failed to set our differenced indicator values if(flag == 3) { message = "Failed to set our differenced indicator values: [ERROR] " + (string) GetLastError(); return(message); } //--- The user is trying to retrieve an index beyond the buffer size and must update the buffer size first if(flag == 4) { message = "The user is attempting to use call an index beyond the buffer size, update the buffer size first"; return(message); } //--- The class has been deactivated by the user if(flag == 5) { message = "Goodbye."; return(message); } //--- No feedback else return(""); } //+------------------------------------------------------------------+
Now let us fetch the market data we need using a collection of instances of our RSI class. We will store pointers to each instance of our class in an array of the custom type we have defined. MQL5 allows us to automatically generate objects on the fly as and when we need them. However, this flexibility comes at the price of having to always clean up after ourselves to prevent issues related to memory leakage.
//+------------------------------------------------------------------+ //| ProjectName | //| Copyright 2020, CompanyName | //| http://www.companyname.net | //+------------------------------------------------------------------+ #property copyright "Copyright 2024, MetaQuotes Ltd." #property link "https://www.mql5.com" #property version "1.00" #property script_show_inputs //+------------------------------------------------------------------+ //| System constants | //+------------------------------------------------------------------+ #define HORIZON 10 //+------------------------------------------------------------------+ //| Libraries | //+------------------------------------------------------------------+ #include <VolatilityDoctor\Indicators\RSI.mqh> //+------------------------------------------------------------------+ //| Global variables | //+------------------------------------------------------------------+ RSI *my_rsi_array[14]; string file_name = Symbol() + " RSI Algorithmic Input Selection.csv"; //+------------------------------------------------------------------+ //| Inputs | //+------------------------------------------------------------------+ input int size = 3000; //+------------------------------------------------------------------+ //| Our script execution | //+------------------------------------------------------------------+ void OnStart() { //--- How much data should we store in our indicator buffer? int fetch = size + (2 * HORIZON); //--- Store pointers to our RSI objects for(int i = 0; i <= 13; i++) { //--- Create an RSI object my_rsi_array[i] = new RSI(Symbol(),PERIOD_CURRENT,((i+1) * 5),PRICE_CLOSE); //--- Set the RSI buffers my_rsi_array[i].SetIndicatorValues(fetch,true); my_rsi_array[i].SetDifferencedIndicatorValues(fetch,HORIZON,true); } //---Write to file int file_handle=FileOpen(file_name,FILE_WRITE|FILE_ANSI|FILE_CSV,","); for(int i=size;i>=1;i--) { if(i == size) { FileWrite(file_handle,"Time","True Close","Open","High","Low","Close","RSI 5","RSI 10","RSI 15","RSI 20","RSI 25","RSI 30","RSI 35","RSI 40","RSI 45","RSI 50","RSI 55","RSI 60","RSI 65","RSI 70","Diff RSI 5","Diff RSI 10","Diff RSI 15","Diff RSI 20","Diff RSI 25","Diff RSI 30","Diff RSI 35","Diff RSI 40","Diff RSI 45","Diff RSI 50","Diff RSI 55","Diff RSI 60","Diff RSI 65","Diff RSI 70"); } else { FileWrite(file_handle, iTime(_Symbol,PERIOD_CURRENT,i), iClose(_Symbol,PERIOD_CURRENT,i), iOpen(_Symbol,PERIOD_CURRENT,i) - iOpen(Symbol(),PERIOD_CURRENT,i + HORIZON), iHigh(_Symbol,PERIOD_CURRENT,i) - iHigh(Symbol(),PERIOD_CURRENT,i + HORIZON), iLow(_Symbol,PERIOD_CURRENT,i) - iLow(Symbol(),PERIOD_CURRENT,i + HORIZON), iClose(_Symbol,PERIOD_CURRENT,i) - iClose(Symbol(),PERIOD_CURRENT,i + HORIZON), my_rsi_array[0].GetReadingAt(i), my_rsi_array[1].GetReadingAt(i), my_rsi_array[2].GetReadingAt(i), my_rsi_array[3].GetReadingAt(i), my_rsi_array[4].GetReadingAt(i), my_rsi_array[5].GetReadingAt(i), my_rsi_array[6].GetReadingAt(i), my_rsi_array[7].GetReadingAt(i), my_rsi_array[8].GetReadingAt(i), my_rsi_array[9].GetReadingAt(i), my_rsi_array[10].GetReadingAt(i), my_rsi_array[11].GetReadingAt(i), my_rsi_array[12].GetReadingAt(i), my_rsi_array[13].GetReadingAt(i), my_rsi_array[0].GetDifferencedReadingAt(i), my_rsi_array[1].GetDifferencedReadingAt(i), my_rsi_array[2].GetDifferencedReadingAt(i), my_rsi_array[3].GetDifferencedReadingAt(i), my_rsi_array[4].GetDifferencedReadingAt(i), my_rsi_array[5].GetDifferencedReadingAt(i), my_rsi_array[6].GetDifferencedReadingAt(i), my_rsi_array[7].GetDifferencedReadingAt(i), my_rsi_array[8].GetDifferencedReadingAt(i), my_rsi_array[9].GetDifferencedReadingAt(i), my_rsi_array[10].GetDifferencedReadingAt(i), my_rsi_array[11].GetDifferencedReadingAt(i), my_rsi_array[12].GetDifferencedReadingAt(i), my_rsi_array[13].GetDifferencedReadingAt(i) ); } } //--- Close the file FileClose(file_handle); //--- Delete our RSI object pointers for(int i = 0; i <= 13; i++) { delete my_rsi_array[i]; } } //+------------------------------------------------------------------+ #undef HORIZON
Analyzing The Data In Python
We can now start analyzing the data we have collected from our MetaTrader 5 Terminal. Our first step will be to load the standard libraries we need.
import pandas as pd import numpy as np import seaborn as sns import matplotlib.pyplot as plt import plotly
Now read in the market data. Also, let us create a flag to denote if the current price level is greater than or less than the price that was offered 10 days ago. Recall that 10 days is the same period we used in our script to calculate the change in RSI levels.
#Let's read in our market data data = pd.read_csv("EURUSD RSI Algorithmic Input Selection.csv") data['Bull'] = np.NaN data.loc[data['True Close'] > data['True Close'].shift(10),'Bull'] = 1 data.loc[data['True Close'] < data['True Close'].shift(10),'Bull'] = 0 data.dropna(inplace=True) data.reset_index(inplace=True,drop=True)
We need to calculate the actual market returns as well.
#Estimate the market returns #Define our forecast horizon HORIZON = 10 data['Target'] = 0 data['Return'] = data['True Close'].shift(-HORIZON) - data['True Close'] data.loc[data['Return'] > 0,'Target'] = 1 #Drop missing values data.dropna(inplace=True)
Most importantly, we need to delete all the data that overlaps with our intended back test period.
#No cheating boys. _ = data.iloc[((-365 * 3) + 95):,:] data = data.iloc[:((-365 * 3) + 95),:] data
Fig 4: Our dataset no longer contains any of the dates that overlap with our back test period.
We can visualize the distribution of EURUSD 10 Day market returns, and we can quickly see that the market returns are fixed around 0. This general distribution shape comes as little surprise and is not unique to the EURUSD pair.
plt.title('Distribution of EURUSD 10 Day Returns') plt.grid() sns.histplot(data['Return'],color='black')
Fig 5: Visualizing the 10 Day EURUSD return distribution.
This exercise offers us a unique opportunity to visually see the difference between the distribution of RSI levels on a short period and RSI levels on a long period. The dashed vertical red lines, mark the standardized 30 and 70 levels. The black bars mark the distribution of the RSI levels when its period is set to 5. We can see the 5 period RSI will generate many signals past the standardized levels. However, the white bars, represent the distribution of RSI levels when the period is set to 70. We can visually see almost no signals will be generated at all. It is this change in the shape of the distribution that makes it challenging for algorithmic traders to always follow "best practices" for using an indicator.
plt.title('Comapring The Distribution of RSI Changes Across Different RSI Periods') sns.histplot(data['RSI 5'],color='black') sns.histplot(data['RSI 70'],color='white') plt.xlabel('RSI Level') plt.legend(['RSI 5','RSI 70']) plt.axvline(30,color='red',linestyle='--') plt.axvline(70,color='red',linestyle='--') plt.grid()
Fig 6: Comparing the distribution of RSI levels across different RSI periods.
Creating a scatter plot with the 10-period change in the 60 period RSI on both the x and y-axis, allows us to visualize whether there is a relationship between the change in the indicator and the target. It appears that a change of 10 RSI levels, may be a reasonable trading signal to go for short positions if the RSI reading fell by 10 levels. Or enter long positions if the RSI level increased by 10.
plt.title('Scatter Plot of 10 Day Change in 50 Period RSI & EURUSD 10 Day Return') sns.scatterplot(data,y='Diff RSI 60',x='Diff RSI 60',hue='Target') plt.xlabel('50 Period RSI') plt.ylabel('50 Period RSI') plt.grid() plt.axvline(-10,color='red',linestyle='--') plt.axvline(10,color='red',linestyle='--')
Fig 7: Visualizing the relationship between the change in the 60 Period RSI and the target.
Trying to combine the signals generated by changes on RSI with different periods may sound like a reasonable enough idea. However, it appears to do little in the way of better separating our two classes of interest.
plt.title('Scatter Plot of 10 Day Change in 5 Period RSI & EURUSD 10 Day Return') sns.scatterplot(data,y='Diff RSI 60',x='Diff RSI 5',hue='Target') plt.xlabel('5 Period RSI') plt.ylabel('5 Period RSI') plt.grid()
Fig 8: It appears that simply using RSI indicators with different periods is a poor source of confirmation.
We have 14 different RSI indicators to choose from. Instead of running 14 back tests to decide which period may serve us best, we can estimate the optimal period by evaluating the performance of a model trained with all 14 RSI indicators as its inputs and then evaluating the importance of features the statistical model learned from the data it was trained with.
Notice that, we always have the choice of employing statistical models for prediction accuracy, or for interpretations and insights. Today we are performing the latter. We will fit a Ridge model on the differences in all 14 RSI indicator inputs. The Ridge model has tuning parameters of its own. We will therefore perform a grid search over the input space for the Ridge statistical model. In particular, we will be adjusting the tuning parameters:
- Alpha: The ridge model requires that a penalty added to control the model's coefficients.
- Tolerance: Determines the smallest change necessary to set stopping conditions / other conditions for subroutines depending on the solver the user has selected.
For our discussion, we will be using a ridge model with the 'sparse_cg' solver. The reader is free to also consider tuning the model if they chose to.
The Ridge model is particularly helpful for us because it shrinks its coefficients down to 0 to reduce the model's loss. Therefore, we will search a wide space of initial settings for our model and then focus on the initial settings that brought about the lowest error. The configuration of the model's weights in its best performing mode can quickly inform us which RSI period our model depended on the most. In our particular example today, it was the 10-period change in the 55 period RSI that obtained the largest coefficients from our best performing model.
#Set the max levels we wish to check ALPHA_LEVELS = 10 TOL_LEVELS = 10 #DataFrame labels r_c = 'TOL_LEVEL_' r_r = 'ALHPA_LEVEL_' results_columns = [] results_rows = [] for c in range(TOL_LEVELS): n_c = r_c + str(c) n_r = r_r + str(c) results_columns.append(n_c) results_rows.append(n_r) #Create a DataFrame to store our results results = pd.DataFrame(columns=results_columns,index=results_rows) #Cross validate our model for i in range(TOL_LEVELS): tol = 10 ** (-i) error = [] for j in range(ALPHA_LEVELS): #Set alpha alpha = 10 ** (-j) #Its good practice to generally check the 0 case if(i == 0 & j == 0): model = Ridge(alpha=j,tol=i,solver='sparse_cg') #Otherwise use a float model = Ridge(alpha=alpha,tol=tol,solver='sparse_cg') #Store the error levels error.append(np.mean(np.abs(cross_val_score(model,data.loc[:,['Diff RSI 5', 'Diff RSI 10', 'Diff RSI 15', 'Diff RSI 20', 'Diff RSI 25', 'Diff RSI 30', 'Diff RSI 35', 'Diff RSI 40', 'Diff RSI 45', 'Diff RSI 50', 'Diff RSI 55', 'Diff RSI 60', 'Diff RSI 65', 'Diff RSI 70',]],data['Return'],cv=tscv)))) #Record the error levels results.iloc[:,i] = error results
The table below summarizes our results. We observe that, our lowest error rates were obtained when we set our model's initial parameters both set to 0.
Tuning Parameter | Model Error |
---|---|
ALHPA_LEVEL_0 | 0.053509 |
ALHPA_LEVEL_1 | 0.056245 |
ALHPA_LEVEL_2 | 0.060158 |
ALHPA_LEVEL_3 | 0.062230 |
ALHPA_LEVEL_4 | 0.061521 |
ALHPA_LEVEL_5 | 0.064312 |
ALHPA_LEVEL_6 | 0.073248 |
ALHPA_LEVEL_7 | 0.079310 |
ALHPA_LEVEL_8 | 0.081914 |
ALHPA_LEVEL_9 | 0.085171 |
It is also possible to visualize our findings using a contour plot. We want to use models in the region of the plot associated with low error, the blue regions. These are our best performing models so far. Let us now visualize the size of each coefficient in the model. The largest coefficient will be assigned to the input our model depended on the most.
import plotly.graph_objects as go fig = go.Figure(data = go.Contour( z=results, colorscale='bluered' )) fig.update_layout( width = 600, height = 400, title='Contour Plot Of Our Error Forecasting EURUSD Using Grid Search ' ) fig.show()
Fig 9: We have found optimal input settings for our Ridge model predicting the 10 day EURUSD return.
Plotting the data visually, we can quickly see the coefficient associated with the 55 Period RSI was assigned the largest absolute value. This gives us some confidence to narrow our focus down to that particular RSI setup.
#Let's visualize the importance of each column model = Ridge(alpha=0,tol=0,solver='sparse_cg') model.fit(data.loc[:,['Diff RSI 5', 'Diff RSI 10', 'Diff RSI 15', 'Diff RSI 20', 'Diff RSI 25', 'Diff RSI 30', 'Diff RSI 35', 'Diff RSI 40', 'Diff RSI 45', 'Diff RSI 50', 'Diff RSI 55', 'Diff RSI 60', 'Diff RSI 65', 'Diff RSI 70',]],data['Return']) #Clearly our model relied on the 25 Period RSI the most, from all the data it had available at training sns.barplot(np.abs(model.coef_),color='black') plt.title('Rleative Feature Importance') plt.ylabel('Coefficient Value') plt.xlabel('Coefficient Index') plt.grid()
Fig 10: The coefficient associated with the Difference in the 55 Period RSI was assigned the largest value.
Now that we have identified our period of interest, let us also evaluate how the error of our model changes, as we cycle through increasing RSI levels of 10. We will create 3 additional columns in our data frame. Column 1 will have the value 1 if the RSI reading is greater than the first value we want to check. Otherwise, if the RSI reading is less than some particular value we wish to check, Column 2 will have the value 1. In any other case, column 3 will have the value 1.
def objective(x): data = pd.read_csv("EURUSD RSI Algorithmic Input Selection.csv") data['0'] = 0 data['1'] = 0 data['2'] = 0 HORIZON = 10 data['Return'] = data['True Close'].shift(-HORIZON) - data['True Close'] data.dropna(subset=['Return'],inplace=True) data.iloc[data['Diff RSI 55'] > x[0],12] = 1 data.iloc[data['Diff RSI 55'] < x[1],13] = 1 data.iloc[(data['Diff RSI 55'] < x[0]) & (data['RSI 55'] > x[1]),14] = 1 #Calculate or RMSE When using those levels model = Ridge(alpha=0,tol=0,solver='sparse_cg') error = np.mean(np.abs(cross_val_score(model,data.iloc[:,12:15],data['Return'],cv=tscv))) return(error)
Let us evaluate the error produced by our model if we set 0 to be our critical level.
#Bad rules for using the RSI objective([0,0])
0.026897725573317266
If we input the 70 and 30 levels into our function, our error increases. We will perform a grid search in steps of 10, to find changes in RSI levels that are better suited for the 55 Period RSI. Our results found that the optimal change level appears close to 10 RSI levels.
#Bad rules for using the RSI objective([70,30])
0.031258730612736006
LEVELS = 10 results = [] for i in np.arange(0,(LEVELS)): results.append(objective([i * 10,-(i * 10)])) plt.plot(results,color='black') plt.ylabel('Error Rate') plt.xlabel('Change in RSI as multiples of 10') plt.grid() plt.scatter(results.index(min(results)),min(results),color='red') plt.title('Measuring The Strength of Changes In RSI Levels')
Fig 11: Visualizing the optimal change level in our RSI indicator.
Let us now perform another finer search, between the interval of RSI changes in the 0 and 20 range. Upon closer inspection, we observe that the true optima appears to be on the value 9. However, we do not conduct such exercises to fit historical data perfectly, that is called over fitting and is a bad practice. Our goal is not to perform an exercise in curve fitting. Instead of taking the optimal value exactly where it appears in our analysis of historical data, we embrace the fact that the location of the optima may change, and rather we aim to be within a certain fraction of a standard deviation away from the optimal value on either side to serve as our confidence intervals.
LEVELS = 20 coef = 0.5 results = [] for i in np.arange(0,(LEVELS),1): results.append(objective([i,-(i)])) plt.plot(results) plt.ylabel('Error Rate') plt.xlabel('Change in RSI') plt.grid() plt.scatter(results.index(min(results)),min(results),color='red') plt.title('Measuring The Strength of Changes In RSI Levels') plt.axvline(results.index(min(results)),color='red') plt.axvline(results.index(min(results)) - (coef * np.std(data['Diff RSI 55'])),linestyle='--',color='red') plt.axvline(results.index(min(results)) + (coef * np.std(data['Diff RSI 55'])),linestyle='--',color='red')
Fig 12: Visualizing the error rate associated with setting different thresholds for RSI level changes.
We can visually see the region we believe may be optimal overlaid on top of the historical distribution of change in the RSI changes.
sns.histplot(data['Diff RSI 55'],color='black') coef = 0.5 plt.axvline((results.index(min(results))),linestyle='--',color='red') plt.axvline(results.index(min(results)) - (coef * np.std(data['Diff RSI 55'])),color='red') plt.axvline(results.index(min(results)) + (coef * np.std(data['Diff RSI 55'])),color='red') plt.axvline(-(results.index(min(results))),linestyle='--',color='red') plt.axvline(-(results.index(min(results)) - (coef * np.std(data['Diff RSI 55']))),color='red') plt.axvline(-(results.index(min(results)) + (coef * np.std(data['Diff RSI 55']))),color='red') plt.title("Visualizing our Optimal Point in The Distribution")
Fig 13: Visualizing the optimal regions we have selected for our RSI trading signals.
Let us get the values of our estimation of good confidence intervals, these will serve as the critical values in our Expert Advisor that trigger long and short signals.
results.index(min(results)) + ( coef * np.std(data['Diff RSI 55']))
10.822857254027287
And our lower bound.
results.index(min(results)) - (coef * np.std(data['Diff RSI 55']))
7.177142745972713
Let's obtain an explanation from our Ridge model to interpret the RSI indicator in a way we may not have intuitively thought of.
def explanation(x): data = pd.read_csv("EURUSD RSI Algorithmic Input Selection.csv") data['0'] = 0 data['1'] = 0 data['2'] = 0 HORIZON = 10 data['Return'] = data['True Close'].shift(-HORIZON) - data['True Close'] data.dropna(subset=['Return'],inplace=True) data.iloc[data['Diff RSI 55'] > x[0],12] = 1 data.iloc[data['Diff RSI 55'] < x[1],13] = 1 data.iloc[(data['Diff RSI 55'] < x[0]) & (data['RSI 55'] > x[1]),14] = 1 #Calculate or RMSE When using those levels model = Ridge(alpha=0,tol=0,solver='sparse_cg') model.fit(data.iloc[:,12:15],data['Return']) return(model.coef_.copy())
We see that, when the RSI indicator changes by more than 9, our optimal value, our model learned positive coefficients, which imply we should enter long positions. Otherwise, the model asserts we should sell under any other conditions.
opt = 9 print(explanation([opt,-opt]))
[ 1.97234840e-04 -1.64215118e-04 -7.55222156e-05]
Building Our Expert Advisor
Assuming our knowledge of the past, is a good model of the future, can we build an application to trade the EURUSD profitably using what we have now learned about the market? To get started, we will first define important system constants that we will need throughout our program and across any other versions we may build.
//+------------------------------------------------------------------+ //| Algorithmic Input Selection.mq5 | //| Gamuchirai Ndawana | //| https://www.mql5.com/en/users/gamuchiraindawa | //+------------------------------------------------------------------+ #property copyright "Gamuchirai Ndawana" #property link "https://www.mql5.com/en/users/gamuchiraindawa" #property version "1.00" //+------------------------------------------------------------------+ //| System constants | //+------------------------------------------------------------------+ #define RSI_PERIOD 55 #define RSI_TIME_FRAME PERIOD_D1 #define SYSTEM_TIME_FRAME PERIOD_D1 #define RSI_PRICE PRICE_CLOSE #define RSI_BUFFER_SIZE 20 #define TRADING_VOLUME SymbolInfoDouble(Symbol(),SYMBOL_VOLUME_MIN)
Let's load our libraries.
//+------------------------------------------------------------------+ //| Load our RSI library | //+------------------------------------------------------------------+ #include <VolatilityDoctor\Indicators\RSI.mqh> #include <Trade\Trade.mqh>
We will need a few global variables.
//+------------------------------------------------------------------+ //| Global variables | //+------------------------------------------------------------------+ CTrade Trade; RSI rsi_55(Symbol(),RSI_TIME_FRAME,RSI_PERIOD,RSI_PRICE); double last_value; int count; int ma_o_handler,ma_c_handler; double ma_o[],ma_c[]; double trade_sl;
Our event handlers will each call their own dedicated method to handle the sub processes needed to complete their tasks.
//+------------------------------------------------------------------+ //| Expert initialization function | //+------------------------------------------------------------------+ int OnInit() { //--- setup(); //--- return(INIT_SUCCEEDED); } //+------------------------------------------------------------------+ //| Expert deinitialization function | //+------------------------------------------------------------------+ void OnDeinit(const int reason) { //--- IndicatorRelease(ma_c_handler); IndicatorRelease(ma_o_handler); } //+------------------------------------------------------------------+ //| Expert tick function | //+------------------------------------------------------------------+ void OnTick() { //--- update(); } //+------------------------------------------------------------------+
The update function updates all our system variables and checks if we have to either open a position, or manage our open positions.
//+------------------------------------------------------------------+ //| Update our system variables | //+------------------------------------------------------------------+ void update(void) { static datetime time_stamp; datetime current_time = iTime(Symbol(),SYSTEM_TIME_FRAME,0); if(time_stamp != current_time) { time_stamp = current_time; CopyBuffer(ma_c_handler,0,0,1,ma_c); CopyBuffer(ma_o_handler,0,0,1,ma_o); if((count == 0) && (PositionsTotal() == 0)) { rsi_55.SetIndicatorValues(RSI_BUFFER_SIZE,true); last_value = rsi_55.GetReadingAt(RSI_BUFFER_SIZE - 1); count = 1; } if(PositionsTotal() == 0) check_signal(); if(PositionsTotal() > 0) manage_setup(); } }
Our positions need to have their stop losses constantly adjusted to ensure that we reduce our risk levels whenever possible.
//+------------------------------------------------------------------+ //| Manage our open trades | //+------------------------------------------------------------------+ void manage_setup(void) { double bid = SymbolInfoDouble(Symbol(),SYMBOL_BID); double ask = SymbolInfoDouble(Symbol(),SYMBOL_ASK); if(PositionSelect(Symbol())) { double current_sl = PositionGetDouble(POSITION_SL); double current_tp = PositionGetDouble(POSITION_TP); double new_sl = (current_tp > current_sl) ? (bid-trade_sl) : (ask+trade_sl); double new_tp = (current_tp < current_sl) ? (bid+trade_sl) : (ask-trade_sl); //--- Buy setup if((current_tp > current_sl) && (new_sl < current_sl)) Trade.PositionModify(Symbol(),new_sl,new_tp); //--- Sell setup if((current_tp < current_sl) && (new_sl > current_sl)) Trade.PositionModify(Symbol(),new_sl,new_tp); } }
Our setup function is responsible for getting our system started up from the ground. It will prepare the indicators we need, and resets our counters.
//+------------------------------------------------------------------+ //| Setup our global variables | //+------------------------------------------------------------------+ void setup(void) { ma_c_handler = iMA(Symbol(),SYSTEM_TIME_FRAME,2,0,MODE_EMA,PRICE_CLOSE); ma_o_handler = iMA(Symbol(),SYSTEM_TIME_FRAME,2,0,MODE_EMA,PRICE_OPEN); count = 0; last_value = 0; trade_sl = 1.5e-2; }
Lastly, our trading rules have been generated thanks to the coefficient values our model learned from the training data. This is the last function we will define before we undefine the system variables we created at the beginning of our program.
//+------------------------------------------------------------------+ //| Check if we have a trading setup | //+------------------------------------------------------------------+ void check_signal(void) { double current_reading = rsi_55.GetCurrentReading(); Comment("Last Reading: ",last_value,"\nDifference in Reading: ",(last_value - current_reading)); double bid = SymbolInfoDouble(Symbol(),SYMBOL_BID); double ask = SymbolInfoDouble(Symbol(),SYMBOL_ASK); double cp_lb = 7.17; double cp_ub = 10.82; if((((last_value - current_reading) <= -(cp_lb)) && ((last_value - current_reading) > (cp_ub)))|| ((((last_value - current_reading) > -(cp_lb))) && ((last_value - current_reading) < (cp_lb)))) { if(ma_o[0] > ma_c[0]) { if(PositionsTotal() == 0) { Trade.Sell(TRADING_VOLUME,Symbol(),bid,(ask+trade_sl),(ask-trade_sl)); count = 0; } } } if(((last_value - current_reading) >= (cp_lb)) < ((last_value - current_reading) < cp_ub)) { if(ma_c[0] < ma_o[0]) { if(PositionsTotal() == 0) { Trade.Buy(TRADING_VOLUME,Symbol(),ask,(bid-trade_sl),(bid+trade_sl)); count = 0; } } } } //+------------------------------------------------------------------+ #undef RSI_BUFFER_SIZE #undef RSI_PERIOD #undef RSI_PRICE #undef RSI_TIME_FRAME #undef SYSTEM_TIME_FRAME #undef TRADING_VOLUME //+------------------------------------------------------------------+
Let us now get started with our back test of the trading system. Recall that in Fig 4, we deleted all the data that overlaps with our back test. Therefore, this may serve us as a close approximation of how our strategy may perform in real time. We will perform a 3-year back test of our strategy on Daily data, from 1 January 2022 until March 2025.
Fig 14: The dates we will use for our back test of the trading strategy.
For best results, we always use "Every tick based on real ticks" because it renders the most accurate simulation of past market conditions based on the historical ticks the MetaTrader 5 Terminal collected on your behalf from your broker.
Fig 15: The conditions which we will perform our tests under matter a lot, and change the profitability of our strategy.
The equity curve produced by our strategy tester appears promising. But, let's interpret the detailed statistics together, to obtain a complete view of our strategy's performance.
Fig 16: The equity curve produced by EURUSD 55 Period RSI trading strategy.
Running our strategy produced the following statistics in our strategy tester:
- Sharpe Ratio: 0.92
- Expected payoff: 2.49
- Total Net Profit: $151.87
- Profit trades: 57.38%
However, pay attention to the fact that from the 61 total trades placed, only 4 were long trades. Why is our strategy so disproportionally biased towards selling? How can we correct this bias? Let us take another look at our EURUSD chart and try to make sense of this together.
Fig 17: The detailed statistics of our Expert Advisor's historical performance on the Daily EURUSD Exchange Rate.
Improving Our Expert Advisor
In Fig 18, I have attached a screenshot of the Monthly EURUSD exchange rate. The 2 red vertical lines mark the beginning of the year 2009 and the end of the year 2021 respectively. The green vertical line, represents the beginning of the training data we used for our statistical model. One can quickly see why the model learned a bias for short positions, given the sustained bear trend that started in 2008.
Fig 18: Understanding why our model learned such bearish sentiment.
We do not always need more historical data, to try and correct for this. Rather, we can fit a more flexible model than the Ridge model we started with. The stronger learner, will then supply our strategy with additional long signals.
It is possible for us to give our Expert Advisor a reasonable idea of the probability that the exchange rate will rise over the next 10 Days. We can train a Random Forest Regressor to predict the probability that we will observe bullish price action. When the expected probability exceeds 0.5, our Expert Advisor will enter a long position. Otherwise, we will follow the strategy we learned from our Ridge model.
Modelling Probabilities in Python
To get started with our corrections, we will first import a few libraries.
from sklearn.metrics import accuracy_score from sklearn.ensemble import RandomForestRegressor
Then we shall define our dependent and independent variables.
#Independent variable X = data[['Diff RSI 55']] #Dependent variable y = data['Target']
Fit the model.
model = RandomForestRegressor() model.fit(X,y)
Get ready to export to ONNX.
import onnx from skl2onnx import convert_sklearn from skl2onnx.common.data_types import FloatTensorType
Define the input shapes.
inital_params = [("float_input",FloatTensorType([1,1]))]
Create ONNX prototype and save it to disk.
onnx_proto = convert_sklearn(model=model,initial_types=inital_params,target_opset=12) onnx.save(onnx_proto,"EURUSD Diff RSI 55 D1 1 1.onnx")
You can also visualize your model using the netron library, to ensure the right input and output attributes were given to your ONNX model.
import netron
netron.start("EURUSD Diff RSI 55 D1 1 1.onnx")
This is a graphical representation of our Random Forest Regressor and the attributes the model has. Netron can also be used to visualize Neural Networks and many other types of ONNX models.
Fig 19: Visualizing our ONNX Random Forest Regressor model.
The inputs and output shapes were correctly specified by ONNX, so we can now move on to applying the Random Forest Regressor to help our Expert Advisor predict the probability of Bullish Price action happening over the next 10 days.
Fig 20: The details of our ONNX model match the expected specifications we wanted to check for.
Improving Our Expert Advisor
Now that we have exported a probabilistic model of the EURUSD market, we can import our ONNX model into our Expert Advisor.
//+------------------------------------------------------------------+ //| Algorithmic Input Selection.mq5 | //| Gamuchirai Ndawana | //| https://www.mql5.com/en/users/gamuchiraindawa | //+------------------------------------------------------------------+ #property copyright "Gamuchirai Ndawana" #property link "https://www.mql5.com/en/users/gamuchiraindawa" #property version "1.00" //+------------------------------------------------------------------+ //| System resources | //+------------------------------------------------------------------+ #resource "\\Files\\EURUSD Diff RSI 55 D1 1 1.onnx" as uchar onnx_model_buffer[];
We will also need a few new macros specifying the shape of our ONNX model.
#define ONNX_INPUTS 1 #define ONNX_OUTPUTS 1
Additionally, the ONNX model deserves a few global variables because we may quickly need them in several parts of our code.
long onnx_model; vectorf onnx_output(1); vectorf onnx_input(1);
In the code snippet below, we have excluded parts of the code base that hasn't changed, and only show the changes made to initialize the ONNX model and set its parameter shapes.
//+------------------------------------------------------------------+ //| Setup our global variables | //+------------------------------------------------------------------+ bool setup(void) { //--- Setup our technical indicators ... //--- Create our ONNX model onnx_model = OnnxCreateFromBuffer(onnx_model_buffer,ONNX_DATA_TYPE_FLOAT); //--- Validate the ONNX model if(onnx_model == INVALID_HANDLE) { return(false); } //--- Define the I/O signature ulong onnx_param[] = {1,1}; if(!OnnxSetInputShape(onnx_model,0,onnx_param)) return(false); if(!OnnxSetOutputShape(onnx_model,0,onnx_param)) return(false); return(true); }
Additionally, our check for valid trading has been trimmed to only highlight the additional code added to it, and avoid duplicating the same code.
//+------------------------------------------------------------------+ //| Check if we have a trading setup | //+------------------------------------------------------------------+ void check_signal(void) { rsi_55.SetDifferencedIndicatorValues(RSI_BUFFER_SIZE,HORIZON,true); onnx_input[0] = (float) rsi_55.GetDifferencedReadingAt(RSI_BUFFER_SIZE - 1); //--- Our Random forest model if(!OnnxRun(onnx_model,ONNX_DATA_TYPE_FLOAT,onnx_input,onnx_output)) Comment("Failed to obtain a forecast from our model!"); else { if(onnx_output[0] > 0.5) if(ma_o[0] < ma_c[0]) Trade.Buy(TRADING_VOLUME,Symbol(),ask,(bid-trade_sl),(bid+trade_sl)); Print("Model Bullish Probabilty: ",onnx_output); } }
All in all, this is what our second version of the trading strategy looks like when fully composed.
//+------------------------------------------------------------------+ //| Algorithmic Input Selection.mq5 | //| Gamuchirai Ndawana | //| https://www.mql5.com/en/users/gamuchiraindawa | //+------------------------------------------------------------------+ #property copyright "Gamuchirai Ndawana" #property link "https://www.mql5.com/en/users/gamuchiraindawa" #property version "1.00" //+------------------------------------------------------------------+ //| System resources | //+------------------------------------------------------------------+ #resource "\\Files\\EURUSD Diff RSI 55 D1 1 1.onnx" as uchar onnx_model_buffer[]; //+------------------------------------------------------------------+ //| System constants | //+------------------------------------------------------------------+ #define RSI_PERIOD 55 #define RSI_TIME_FRAME PERIOD_D1 #define SYSTEM_TIME_FRAME PERIOD_D1 #define RSI_PRICE PRICE_CLOSE #define RSI_BUFFER_SIZE 20 #define TRADING_VOLUME SymbolInfoDouble(Symbol(),SYMBOL_VOLUME_MIN) #define ONNX_INPUTS 1 #define ONNX_OUTPUTS 1 #define HORIZON 10 //+------------------------------------------------------------------+ //| Load our RSI library | //+------------------------------------------------------------------+ #include <VolatilityDoctor\Indicators\RSI.mqh> #include <Trade\Trade.mqh> //+------------------------------------------------------------------+ //| Global variables | //+------------------------------------------------------------------+ CTrade Trade; RSI rsi_55(Symbol(),RSI_TIME_FRAME,RSI_PERIOD,RSI_PRICE); double last_value; int count; int ma_o_handler,ma_c_handler; double ma_o[],ma_c[]; double trade_sl; long onnx_model; vectorf onnx_output(1); vectorf onnx_input(1); //+------------------------------------------------------------------+ //| Expert initialization function | //+------------------------------------------------------------------+ int OnInit() { //--- setup(); //--- return(INIT_SUCCEEDED); } //+------------------------------------------------------------------+ //| Expert deinitialization function | //+------------------------------------------------------------------+ void OnDeinit(const int reason) { //--- IndicatorRelease(ma_c_handler); IndicatorRelease(ma_o_handler); } //+------------------------------------------------------------------+ //| Expert tick function | //+------------------------------------------------------------------+ void OnTick() { //--- update(); } //+------------------------------------------------------------------+ //+------------------------------------------------------------------+ //| Update our system variables | //+------------------------------------------------------------------+ void update(void) { static datetime time_stamp; datetime current_time = iTime(Symbol(),SYSTEM_TIME_FRAME,0); if(time_stamp != current_time) { time_stamp = current_time; CopyBuffer(ma_c_handler,0,0,1,ma_c); CopyBuffer(ma_o_handler,0,0,1,ma_o); if((count == 0) && (PositionsTotal() == 0)) { rsi_55.SetIndicatorValues(RSI_BUFFER_SIZE,true); rsi_55.SetDifferencedIndicatorValues(RSI_BUFFER_SIZE,HORIZON,true); last_value = rsi_55.GetReadingAt(RSI_BUFFER_SIZE - 1); count = 1; } if(PositionsTotal() == 0) check_signal(); if(PositionsTotal() > 0) manage_setup(); } } //+------------------------------------------------------------------+ //| Manage our open trades | //+------------------------------------------------------------------+ void manage_setup(void) { double bid = SymbolInfoDouble(Symbol(),SYMBOL_BID); double ask = SymbolInfoDouble(Symbol(),SYMBOL_ASK); if(PositionSelect(Symbol())) { double current_sl = PositionGetDouble(POSITION_SL); double current_tp = PositionGetDouble(POSITION_TP); double new_sl = (current_tp > current_sl) ? (bid-trade_sl) : (ask+trade_sl); double new_tp = (current_tp < current_sl) ? (bid+trade_sl) : (ask-trade_sl); //--- Buy setup if((current_tp > current_sl) && (new_sl < current_sl)) Trade.PositionModify(Symbol(),new_sl,new_tp); //--- Sell setup if((current_tp < current_sl) && (new_sl > current_sl)) Trade.PositionModify(Symbol(),new_sl,new_tp); } } //+------------------------------------------------------------------+ //| Setup our global variables | //+------------------------------------------------------------------+ bool setup(void) { //--- Setup our technical indicators ma_c_handler = iMA(Symbol(),SYSTEM_TIME_FRAME,2,0,MODE_EMA,PRICE_CLOSE); ma_o_handler = iMA(Symbol(),SYSTEM_TIME_FRAME,2,0,MODE_EMA,PRICE_OPEN); count = 0; last_value = 0; trade_sl = 1.5e-2; //--- Create our ONNX model onnx_model = OnnxCreateFromBuffer(onnx_model_buffer,ONNX_DATA_TYPE_FLOAT); //--- Validate the ONNX model if(onnx_model == INVALID_HANDLE) { return(false); } //--- Define the I/O signature ulong onnx_param[] = {1,1}; if(!OnnxSetInputShape(onnx_model,0,onnx_param)) return(false); if(!OnnxSetOutputShape(onnx_model,0,onnx_param)) return(false); return(true); } //+------------------------------------------------------------------+ //| Check if we have a trading setup | //+------------------------------------------------------------------+ void check_signal(void) { rsi_55.SetDifferencedIndicatorValues(RSI_BUFFER_SIZE,HORIZON,true); last_value = rsi_55.GetReadingAt(RSI_BUFFER_SIZE - 1); double current_reading = rsi_55.GetCurrentReading(); Comment("Last Reading: ",last_value,"\nDifference in Reading: ",(last_value - current_reading)); double bid = SymbolInfoDouble(Symbol(),SYMBOL_BID); double ask = SymbolInfoDouble(Symbol(),SYMBOL_ASK); double cp_lb = 7.17; double cp_ub = 10.82; onnx_input[0] = (float) rsi_55.GetDifferencedReadingAt(RSI_BUFFER_SIZE - 1); //--- Our Random forest model if(!OnnxRun(onnx_model,ONNX_DATA_TYPE_FLOAT,onnx_input,onnx_output)) Comment("Failed to obtain a forecast from our model!"); else { if(onnx_output[0] > 0.5) if(ma_o[0] < ma_c[0]) Trade.Buy(TRADING_VOLUME,Symbol(),ask,(bid-trade_sl),(bid+trade_sl)); Print("Model Bullish Probabilty: ",onnx_output); } //--- The trading rules we learned from our Ridge Regression Model //--- Ridge Regression Sell if((((last_value - current_reading) <= -(cp_lb)) && ((last_value - current_reading) > (cp_ub)))|| ((((last_value - current_reading) > -(cp_lb))) && ((last_value - current_reading) < (cp_lb)))) { if(ma_o[0] > ma_c[0]) { if(PositionsTotal() == 0) { Trade.Sell(TRADING_VOLUME,Symbol(),bid,(ask+trade_sl),(ask-trade_sl)); count = 0; } } } //--- Ridge Regression Buy else if(((last_value - current_reading) >= (cp_lb)) < ((last_value - current_reading) < cp_ub)) { if(ma_c[0] < ma_o[0]) { if(PositionsTotal() == 0) { Trade.Buy(TRADING_VOLUME,Symbol(),ask,(bid-trade_sl),(bid+trade_sl)); count = 0; } } } } //+------------------------------------------------------------------+ #undef RSI_BUFFER_SIZE #undef RSI_PERIOD #undef RSI_PRICE #undef RSI_TIME_FRAME #undef SYSTEM_TIME_FRAME #undef TRADING_VOLUME #undef ONNX_INPUTS #undef ONNX_OUTPUTS //+------------------------------------------------------------------+
We will be testing the strategy under the same conditions highlighted in Fig 14 and Fig 15. Here are the results we obtained with the second version of our trading strategy. The equity curve produced by our second version of the trading appears to be quite the same as the first equity curve we produced.
Fig 21: Visualizing the profitability of our second trading strategy.
It is only when we view the detailed statistics, that the differences between the two strategies start to surface. Our Sharpe ratio and expected payoff fell. Our new strategy placed 85 trades that's 39% more than the 61 trades placed by our initial strategy. Additionally, our total number of long positions increased from just 4 in the initial test, to 42 in our second test. That is an increase of 950%. So when we consider the amount of additional risk we are taking on is significant and yet, our accuracy and profitability statistics are only dropping marginally, then we start to build positive expectations about using the strategy. In our previous test, 57.38 of all our trades were profitable, and now 56.47%, that's a reduction of in accuracy of approximately 1.59%.
Fig 22: Our detailed statistics of the performance of our revised version of our trading strategy.
Conclusion
After reading this article, the reader walks away with an understanding of how they could employ grid search techniques alongside statistical models to help them select an optimal period to use for indicators without having to perform multiple back tests and manually search every single possible period. Additionally, the reader has also learned one possible way of estimating and comparing the value of new RSI levels they wish to trade, against the value of the traditional 70 and 30 levels, allowing the reader to trade adversarial market conditions with renewed confidence in their ability.
File Name | Description |
---|---|
Algorithmic Inputs Selection.ipynb | The Jupyter Notebook we used to perform numerical analysis using Python. |
EURUSD Testing RSI Class.mql5 | The MQL5 script we used to test our implementation of our custom RSI Class. |
EURUSD RSI Algorithmic Input Selection.mql5 | The script we used to fetch historical market data. |
Algorithmic Input Selection.mql5 | Our initial version of the Trading Strategy we built. |
Algorithmic Input Selection 2.mql5 | Our refined version of the Trading Strategy that corrected the bias learned by our initial trading strategy. |





- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use