Before reading this article, I recommend that you study the second article in the series "Developing a self-adapting algorithm (Part II): Improving efficiency". The methodology applied in the current article differs significantly from everything discussed earlier, but it will be useful to read the previous articles to understand the topic.
Opting for MQL5's matrix and vector capabilities over alternatives like ONNX and Python has considerable benefits. Using an ONNX model would require separate model instances for each trading symbol and new models for any minor parameter changes, such as time frame adjustments. MQL5, however, offers adaptability without the need to manage numerous models for varying conditions.
This article demonstrates how we can intelligently achieve our goal by using a transition matrix to model market behavior and determine whether to employ trend-following or mean-reverting strategies. We start by developing a high-level understanding of transition matrices. Then, we explore how these mathematical tools can be used to create intelligent trading algorithms with enhanced decision-making abilities.
In this article, we will discuss how we can build Expert Advisors capable of autonomously selecting and changing trading strategies based on prevailing market conditions. We will learn about Markov Chains and how they can be helpful to us as algorithmic traders.
Members of our community are keen to integrate AI into their trading strategies, which requires tuning AI models for specific markets. Each AI model has adjustable parameters that significantly influence its performance; optimal settings for one market may not work for another. This article will show how to customize AI models to outperform default settings using optimization algorithms, specifically the Nelder-Mead algorithm. We'll apply this algorithm to fine-tune a deep neural network using data from the MetaTrader5 terminal and then export the optimized model in ONNX format for use in an Expert Advisor. For those unfamiliar with these concepts, we will provide detailed explanations throughout the article.
We will analyze all of Deriv’s synthetic markets individually, starting with their best known synthetic market, the Boom 1000. The Boom 1000 is notorious for its volatile and unpredictable behavior. The market is characterized by slow, short and equally sized bear candles that are randomly followed by violent, skyscraper sized bull candles. The bull candles are especially challenging to mitigate because the ticks associated with the candle normally aren’t sent to the client terminal, meaning that all stop losses are breached with guaranteed slippage every time.
Therefore, most successful traders have created strategies loosely based on only taking buy opportunities when trading the Boom 1000. Recall that the Boom 1000 could fall for 20 mins on the M1 chart, and retrace that entire movement in 1 candle! Therefore, given its overpowered bullish nature, successful traders look to use this to their advantage by attributing more weight to buy setups on the Boom 1000, than they would to a sell setup.
Get ready to be introduced into the wonderful world of auto-optimizing forex trading algorithms. It can allow your Expert Advisor (EA) to adjust itself for the next iteration of trading based on how the market conditions are after a trade is done.
In this series of articles, we will discuss different ways of building trading applications capable of dynamically adjusting themselves to evolving market conditions. There are potentially infinite ways we can approach this problem but, it is unlikely that all possible solutions will be valid. Therefore, our goal today is to demonstrate and empirically analyze the merits and shortcomings of different possible solutions, to help you improve your trading strategies.
In our previous discussion on Markov Chains, linked here, we demonstrated how to use a transition matrix to understand the probabilistic behavior of the market. Our transition matrix summarized a lot of information for us. It not only guided us on when to buy and sell, it also informed us whether our market had strong trends or was mostly mean reverting. In today's discussion, we shall change our definition of the system state from the moving averages we used in our first discussion to the Relative Strength Indicator (RSI) indicator instead.
Overfitting in machine learning can take on many different forms. Most commonly, it happens when an AI model learns too much of the noise in the data, and fails to make any useful generalizations. This leads to dismal performance when we assess the model on data it has not seen before. There are many techniques that have been developed to mitigate overfitting, but such methods can often prove challenging to implement, especially when you are just getting started on your journey. However, a recent paper, published by a group of diligent Harvard Alumni, suggests that on certain tasks, overfitting may be a problem of the past. This article will walk you through the research paper, and demonstrate how you can build world-class AI models, inline with the world's leading research.
There are many reasons we can use to explain this behavior in trading applications. However, one key understanding is that once our applications have been developed and deployed, their behavior remains fixed and cannot usually be modified without human intervention. This means our strategies are vulnerable to repeating the same mistakes over and over again without ever profiting from failure or learning from past mistakes.
Financial markets are unpredictable, and trading strategies that look profitable in the past often collapse in real market conditions. This happens because most strategies are fixed once deployed and cannot adapt or learn from their mistakes. By borrowing ideas from control theory, we can use feedback controllers to observe how our strategies interact with markets and adjust their behavior toward profitability. Our results show that adding a feedback controller to a simple moving average strategy improved profits, reduced risk, and increased efficiency, proving that this approach has strong potential for trading applications.
Forum on trading, automated trading systems and testing trading strategies
How to start with MetaTrader and forex, the beginning
Sergey Golubev, 2021.03.12 09:56
Self-adapting algorithm (Part III): Abandoning optimization
Before reading this article, I recommend that you study the second article in the series "Developing a self-adapting algorithm (Part II): Improving efficiency". The methodology applied in the current article differs significantly from everything discussed earlier, but it will be useful to read the previous articles to understand the topic.
Build Self Optmising Expert Advisors in MQL5
Build Self Optimizing Expert Advisors With MQL5 And Python
Forum on trading, automated trading systems and testing trading strategies
MetaTrader 5 Python User Group - the summary
Sergey Golubev, 2024.08.03 09:21
Build Self Optimizing Expert Advisors With MQL5 And Python (Part II): Tuning Deep Neural Networks
Forum on trading, automated trading systems and testing trading strategies
MetaTrader 5 Python User Group - the summary
Sergey Golubev, 2024.09.10 08:27
Self Optimizing Expert Advisor with MQL5 And Python (Part III): Cracking The Boom 1000 Algorithm
Therefore, most successful traders have created strategies loosely based on only taking buy opportunities when trading the Boom 1000. Recall that the Boom 1000 could fall for 20 mins on the M1 chart, and retrace that entire movement in 1 candle! Therefore, given its overpowered bullish nature, successful traders look to use this to their advantage by attributing more weight to buy setups on the Boom 1000, than they would to a sell setup.
Forum on trading, automated trading systems and testing trading strategies
All (not yet) about Strategy Tester, Optimization and Cloud
Sergey Golubev, 2024.09.14 07:43
How to Implement Auto Optimization in MQL5 Expert Advisors
Forum on trading, automated trading systems and testing trading strategies
MetaTrader 5 Python User Group - the summary
Sergey Golubev, 2024.09.20 06:36
Self Optimizing Expert Advisor With MQL5 And Python (Part IV): Stacking Models
Forum on trading, automated trading systems and testing trading strategies
MetaTrader 5 Python User Group - the summary
Sergey Golubev, 2024.10.10 12:34
Self Optimizing Expert Advisor With MQL5 And Python (Part V): Deep Markov Models
Forum on trading, automated trading systems and testing trading strategies
MetaTrader 5 Python User Group - the summary
Sergey Golubev, 2024.11.05 18:56
Self Optimizing Expert Advisor With MQL5 And Python (Part VI): Taking Advantage of Deep Double Descent
Overfitting in machine learning can take on many different forms. Most commonly, it happens when an AI model learns too much of the noise in the data, and fails to make any useful generalizations. This leads to dismal performance when we assess the model on data it has not seen before. There are many techniques that have been developed to mitigate overfitting, but such methods can often prove challenging to implement, especially when you are just getting started on your journey. However, a recent paper, published by a group of diligent Harvard Alumni, suggests that on certain tasks, overfitting may be a problem of the past. This article will walk you through the research paper, and demonstrate how you can build world-class AI models, inline with the world's leading research.
Self Optimizing Expert Advisors in MQL5 (Part 13): A Gentle Introduction To Control Theory Using Matrix Factorization