Discussing the article: "Neural networks made easy (Part 47): Continuous action space"

[MetaQuotes]

Check out the new article: Neural networks made easy (Part 47): Continuous action space.

In this article, we expand the range of tasks of our agent. The training process will include some aspects of money and risk management, which are an integral part of any trading strategy.

In our previous article, we trained the agent only to determine the trading direction. The Agent's range of actions was limited to only 4 options:

• buy, 
• sell, 
• hold/wait,
• close all positions.

Here we do not see capital and risk management functions. We used the minimum lot in all trading operations. This is enough to evaluate training approaches, but not enough to build a trading strategy. A profitable trading strategy simply must have a money management algorithm.

In addition, to create a stable trading strategy, we need to manage risks. This block is also missing from our designs. The EA assessed the market situation at each new trading candle and made a decision on a trading operation. But every upcoming bar carries risks for our account. Price movement within a bar can be detrimental to our balance. This is why it is always recommended to use stop losses. This simple approach allows us to limit risks per trade.

After about 3000 passes, I was able to get a model that was able to generate profit on the training set. During the training period of 5 months, the model made 334 transactions. More than 84% of them were profitable. The result was the profit of 33% of the initial capital. At the same time, the drawdown on balances was less than 1%, and by Equity - 7.6%. The profit factor exceeded 26 and the recovery factor amounted to 3.16. The graph below shows an upward trend in the balance. The balance line is almost always below the Equity line, which indicates that positions are being opened in the right direction. At the same time, the load on the deposit is about 20%. This is a fairly high figure, but does not exceed the accumulated profit.

Unfortunately, the results of the EA's work turned out to be more modest outside the training set.

Author: Dmitriy Gizlyk

[Too Chee Ng]

Dmitriy Gizlyk Many thanks for the detailed article. I retrained the model between 1st Jan 2023 and 31st May 2023 to produced DDPGAct.nnw and DDPGCrt.nnw. However, when testing the EA with test.ex5, there wan't a single trade.

I have taken the following steps:

1. Download, unzip and compile Study.mq5 and test.mq5 from https://www.mql5.com/en/articles/download/12853.zip
2. In Strategy Tester, run Study.ex5 once as indicated in https://c.mql5.com/2/55/Study.png
3. In Strategy Tester, Optimize Study.ex5 as indicated in https://c.mql5.com/2/55/Study_opt.png; ref. to the attached Optimized.png
   
4. In Strategy Tester, run test.ex5 (Optimization: Disabled) for the same period between 1st Jan 2023 and 31st May 2023
5. (No error and no trade at all!)

Trying to debug with the following PrintFormat lines:

...
   double sell_sl = NormalizeDouble(Symb.Bid() + ActorResult[5], Symb.Digits());
   PrintFormat("ActorResult[0]=%f  ActorResult[1]=%f ActorResult[2]=%f buy_sl=%f",ActorResult[0], ActorResult[1],  ActorResult[2], buy_sl);
   PrintFormat("ActorResult[3]=%f  ActorResult[4]=%f ActorResult[5]=%f sell_tp=%f",ActorResult[0], ActorResult[1],  ActorResult[2], sell_tp);
//---
   if(ActorResult[0] > 0 && ActorResult[1] > 0 && ActorResult[2] > 0 && buy_sl > 0)
      Trade.Buy(buy_lot, Symb.Name(), Symb.Ask(), buy_sl, buy_tp);
   if(ActorResult[3] > 0 && ActorResult[4] > 0 && ActorResult[5] > 0 && sell_tp > 0)
      Trade.Sell(sell_lot, Symb.Name(), Symb.Bid(), sell_sl, sell_tp);
...

reviewed the following:

...
2023.12.01 23:15:18.641	Core 01	2023.05.30 19:00:00   ActorResult[0]=0.085580  ActorResult[1]=-0.000476 ActorResult[2]=-0.000742 buy_sl=1.072910
2023.12.01 23:15:18.641	Core 01	2023.05.30 19:00:00   ActorResult[3]=0.085580  ActorResult[4]=-0.000476 ActorResult[5]=-0.000742 sell_tp=1.070290
2023.12.01 23:15:18.641	Core 01	2023.05.30 20:00:00   ActorResult[0]=0.085580  ActorResult[1]=-0.000476 ActorResult[2]=-0.000742 buy_sl=1.072830
2023.12.01 23:15:18.641	Core 01	2023.05.30 20:00:00   ActorResult[3]=0.085580  ActorResult[4]=-0.000476 ActorResult[5]=-0.000742 sell_tp=1.070210
2023.12.01 23:15:18.641	Core 01	2023.05.30 21:00:00   ActorResult[0]=0.085580  ActorResult[1]=-0.000476 ActorResult[2]=-0.000742 buy_sl=1.072450
2023.12.01 23:15:18.641	Core 01	2023.05.30 21:00:00   ActorResult[3]=0.085580  ActorResult[4]=-0.000476 ActorResult[5]=-0.000742 sell_tp=1.069830
2023.12.01 23:15:18.641	Core 01	2023.05.30 22:00:00   ActorResult[0]=0.085580  ActorResult[1]=-0.000476 ActorResult[2]=-0.000742 buy_sl=1.072710
2023.12.01 23:15:18.641	Core 01	2023.05.30 22:00:00   ActorResult[3]=0.085580  ActorResult[4]=-0.000476 ActorResult[5]=-0.000742 sell_tp=1.070090
2023.12.01 23:15:18.641	Core 01	2023.05.30 23:00:00   ActorResult[0]=0.085580  ActorResult[1]=-0.000476 ActorResult[2]=-0.000742 buy_sl=1.073750
2023.12.01 23:15:18.641	Core 01	2023.05.30 23:00:00   ActorResult[3]=0.085580  ActorResult[4]=-0.000476 ActorResult[5]=-0.000742 sell_tp=1.071130
...

May I know what has gone wrong or missed please?

Many thanks.