Developing a Replay System (Part 65): Playing the service (VI)

Introduction

In the previous article "Developing a Replay System (Part 64): Playing the service (V)", we fixed two bugs in the replay/simulation application. However, not everything was resolved. At least not to the extent that we can move forward with new developments in this article. There are still a few minor problems that continue to affect the system. These issues did not exist back when we were using global terminal variables. But since we've moved away from that approach and adopted new techniques and methods to make the replay/simulation application work, we now need to adapt and build a new implementation. That said, as you, dear reader, may have noticed, we're not starting from scratch. In fact, we are adapting and refining the existing code to ensure that the work previously done using global terminal variables is not entirely lost.

With that, we are now almost at the same level of functionality we had before. However, to reach that same point, we still need to iron out a few more details. I'll attempt to do that in this article, as the remaining issues are relatively simple. This is different from the memory dump bug we addressed earlier. That particular issue was quite complex and required a detailed explanation of why the bug occurred, even when the code appeared to be completely correct. Simply showing the line that needed to be added wouldn't have sufficed, and many readers would have been left confused. Alternatively, if I had just fixed the code without any explanation, that would have been equally disappointing. You would never have been exposed to such problems and might have developed a false sense of security. And then you would become frustrated when similar issues arose with no clear source of guidance. Worse yet, you might start doubting your own abilities as a professional. I want to avoid that. Even seasoned professionals make mistakes. Though you might not always see them, they do happen. What sets them apart is their ability to quickly identify and resolve these issues. That's why I wish every aspiring developer to grow into a true professional. And not just any professional, but an outstanding one in their field. With that in mind, let's tackle the first of the remaining issues.

Adding the Fast-Forward Feature (Basic Model)

This feature existed in the past and was implemented during the period when we relied on global terminal variables. Since we are no longer using those variables, we'll need to adapt the code to reintroduce the fast-forward functionality. I will maintain the same fast-forwarding logic we used previously. So you will be able to more easily understand how the legacy implementation is being adapted to fit the new system.

To begin, we need to make a small modification to the code presented in the last article. This change ensures that the control indicator will function correctly. You can see the changes below:

35. //+------------------------------------------------------------------+
36. inline void UpdateIndicatorControl(void)
37.          {
38.             double Buff[];
39.                                  
40.             if (m_IndControl.Handle == INVALID_HANDLE) return;
41.             if (m_IndControl.Memory._16b[C_Controls::eCtrlPosition] == m_IndControl.Position)
42.             {
43.                if (CopyBuffer(m_IndControl.Handle, 0, 0, 1, Buff) == 1)
44.                   m_IndControl.Memory.dValue = Buff[0];
45.                if ((C_Controls::eObjectControl)m_IndControl.Memory._16b[C_Controls::eCtrlStatus] != C_Controls::eTriState)
46.                   if ((m_IndControl.Mode = (C_Controls::eObjectControl)m_IndControl.Memory._16b[C_Controls::eCtrlStatus]) == C_Controls::ePlay)
47.                      m_IndControl.Position = m_IndControl.Memory._16b[C_Controls::eCtrlPosition];
48.             }else if (m_IndControl.Mode == C_Controls::ePause)
49.             {
50.                m_IndControl.Memory._16b[C_Controls::eCtrlPosition] = m_IndControl.Position;
51.                m_IndControl.Memory._16b[C_Controls::eCtrlStatus] = (ushort)m_IndControl.Mode;
52.                m_IndControl.Memory._8b[7] = 'D';
53.                m_IndControl.Memory._8b[6] = 'M';
54.                EventChartCustom(m_Infos.IdReplay, evCtrlReplayInit, 0, m_IndControl.Memory.dValue, "");
55.             }
56.          }
57. //+------------------------------------------------------------------+

Code snippet from file C_Replay.mqh

The change, or more precisely the addition, was made specifically on line 48. But why was this change necessary? The reason lies in the code within the LoopEventOnTime function. Wait, now that sounds confusing. Why make a change in the UpdateIndicatorControl procedure if the issue stems from what happens inside LoopEventOnTime? That doesn't make sense. Indeed, it wouldn't make sense if it weren't for the fact that the LoopEventOnTime function reads from and writes to the control indicator by sending and receiving messages. Without the conditional check found on line 48, something unusual occurs when you attempt to fast-forward and then immediately hit play. This is before we even implement the actual fast-forwarding logic.

If you fast-forward and then press play, you will be unable to send a pause command to the service. What? That sounds absurd. Surely pressing the pause button would send the appropriate update to the service, right? It does send the update instructing the system to pause. However, the effect is not immediate. Why do you think? The reason lies in line 41. The issue is that the control indicator's buffer and the memory space being referenced are out of sync. If you start the service, press play, and then fast-forward through time, nothing will appear to go wrong. But if you pause the service and try to fast-forward while the check on line 48 is missing, the control indicator's lock bar will move along with the fast-forward. This would prevent the user from manually adjusting the position.

Now, if you start the replay/simulation service, move forward by a single position, and then hit play, you won't be able to pause the service using the pause button. It will only stop when the condition in line 41 becomes true and the buffer indicates that pause mode is active. This could take quite some time. Perhaps the explanation seems a bit tangled, but that's because there are three distinct scenarios we need to consider. Each one has its own challenges due to the fact that LoopEventOnTime constantly reads from and sends messages to the control indicator throughout the normal execution of the replay/simulation process.

However, by simply adding a single conditional test implemented in line 48 as shown in the snippet, all of these issues are resolved. We eliminate the problems associated with the LoopEventOnTime function. This allows us to fully focus on making the fast-forward feature work as intended.

Implementing fast-forward isn't actually a complex task. In fact, we can achieve it by simply adding the following code snippets to the C_Replay.mqh file.

001. //+------------------------------------------------------------------+
002. #property copyright "Daniel Jose"
003. //+------------------------------------------------------------------+
004. #include "C_ConfigService.mqh"
005. #include "C_Controls.mqh"
006. //+------------------------------------------------------------------+
007. #define def_IndicatorControl   "Indicators\\Market Replay.ex5"
008. #resource "\\" + def_IndicatorControl
009. //+------------------------------------------------------------------+
010. #define def_CheckLoopService ((!_StopFlag) && (ChartSymbol(m_Infos.IdReplay) != ""))
011. //+------------------------------------------------------------------+
012. #define def_ShortNameIndControl "Market Replay Control"
013. //+------------------------------------------------------------------+
014. class C_Replay : public C_ConfigService
015. {
016.    private   :

              ...

035. //+------------------------------------------------------------------+
036. inline void UpdateIndicatorControl(void)
037.          {
038.             double Buff[];
039.                                  
040.             if (m_IndControl.Handle == INVALID_HANDLE) return;
041.             if (m_IndControl.Memory._16b[C_Controls::eCtrlPosition] == m_IndControl.Position)
042.             {
043.                if (CopyBuffer(m_IndControl.Handle, 0, 0, 1, Buff) == 1)
044.                   m_IndControl.Memory.dValue = Buff[0];
045.                if ((C_Controls::eObjectControl)m_IndControl.Memory._16b[C_Controls::eCtrlStatus] != C_Controls::eTriState)
046.                   if ((m_IndControl.Mode = (C_Controls::eObjectControl)m_IndControl.Memory._16b[C_Controls::eCtrlStatus]) == C_Controls::ePlay)
047.                      m_IndControl.Position = m_IndControl.Memory._16b[C_Controls::eCtrlPosition];
048.             }else if (m_IndControl.Mode == C_Controls::ePause)
049.             {
050.                m_IndControl.Memory._16b[C_Controls::eCtrlPosition] = m_IndControl.Position;
051.                m_IndControl.Memory._16b[C_Controls::eCtrlStatus] = (ushort)m_IndControl.Mode;
052.                m_IndControl.Memory._8b[7] = 'D';
053.                m_IndControl.Memory._8b[6] = 'M';
054.                EventChartCustom(m_Infos.IdReplay, evCtrlReplayInit, 0, m_IndControl.Memory.dValue, "");
055.             }
056.          }
057. //+------------------------------------------------------------------+

              ...

068. //+------------------------------------------------------------------+
069. inline void CreateBarInReplay(bool bViewTick)
070.          {
071.             bool    bNew;
072.             double dSpread;
073.             int    iRand = rand();
074. 
075.             if (BuildBar1Min(m_Infos.CountReplay, m_Infos.Rate[0], bNew))
076.             {
077.                m_Infos.tick[0] = m_MemoryData.Info[m_Infos.CountReplay];
078.                if (m_MemoryData.ModePlot == PRICE_EXCHANGE)
079.                {                  
080.                   dSpread = m_Infos.PointsPerTick + ((iRand > 29080) && (iRand < 32767) ? ((iRand & 1) == 1 ? m_Infos.PointsPerTick : 0 ) : 0 );
081.                   if (m_Infos.tick[0].last > m_Infos.tick[0].ask)
082.                   {
083.                      m_Infos.tick[0].ask = m_Infos.tick[0].last;
084.                      m_Infos.tick[0].bid = m_Infos.tick[0].last - dSpread;
085.                   }else if (m_Infos.tick[0].last < m_Infos.tick[0].bid)
086.                   {
087.                      m_Infos.tick[0].ask = m_Infos.tick[0].last + dSpread;
088.                      m_Infos.tick[0].bid = m_Infos.tick[0].last;
089.                   }
090.                }
091.                if (bViewTick) CustomTicksAdd(def_SymbolReplay, m_Infos.tick);
092.                CustomRatesUpdate(def_SymbolReplay, m_Infos.Rate);
093.             }
094.             m_Infos.CountReplay++;
095.          }
096. //+------------------------------------------------------------------+

              ...

123. //+------------------------------------------------------------------+
124.       void AdjustPositionToReplay(void)
125.          {
126.             int nPos;
127.             
128.             if (m_IndControl.Position == (int)((m_Infos.CountReplay * def_MaxPosSlider * 1.0) / m_MemoryData.nTicks)) return;
129.             nPos = (int)(m_MemoryData.nTicks * ((m_IndControl.Position * 1.0) / (def_MaxPosSlider + 1)));
130.             while ((nPos > m_Infos.CountReplay) && def_CheckLoopService)
131.                CreateBarInReplay(false);
132.          }
133. //+------------------------------------------------------------------+
134.    public   :
135. //+------------------------------------------------------------------+

              ...

200. //+------------------------------------------------------------------+
201.       bool LoopEventOnTime(void)
202.          {         
203.             int iPos;
204. 
205.             while ((def_CheckLoopService) && (m_IndControl.Mode != C_Controls::ePlay))
206.             {
207.                UpdateIndicatorControl();
208.                Sleep(200);
209.             }
210.             m_MemoryData = GetInfoTicks();
211.             AdjustPositionToReplay();
212.             iPos = 0;
213.             while ((m_Infos.CountReplay < m_MemoryData.nTicks) && (def_CheckLoopService))
214.             {
215.                if (m_IndControl.Mode == C_Controls::ePause) return true;
216.                iPos += (int)(m_Infos.CountReplay < (m_MemoryData.nTicks - 1) ? m_MemoryData.Info[m_Infos.CountReplay + 1].time_msc - m_MemoryData.Info[m_Infos.CountReplay].time_msc : 0);
217.                CreateBarInReplay(true);
218.                while ((iPos > 200) && (def_CheckLoopService))
219.                {
220.                   Sleep(195);
221.                   iPos -= 200;
222.                   m_IndControl.Position = (ushort)((m_Infos.CountReplay * def_MaxPosSlider) / m_MemoryData.nTicks);
223.                   UpdateIndicatorControl();
224.                }
225.             }
226. 
227.             return ((m_Infos.CountReplay == m_MemoryData.nTicks) && (def_CheckLoopService));
228.          }
229. //+------------------------------------------------------------------+
230. };
231. //+------------------------------------------------------------------+
232. #undef macroRemoveSec
233. #undef def_SymbolReplay
234. #undef def_CheckLoopService
235. //+------------------------------------------------------------------+

Code snippets from file C_Replay.mqh

As shown in the code snippets above from the C_Replay.mqh file, we can clearly see everything necessary to implement the basic fast-forward functionality. I say basic because there are still a few minor issues that I'll need to explain. However, it's important for you to understand this simpler approach, as it forms the foundation for the more complete implementation we'll be building shortly. Notice the reference made earlier regarding line 48. Now, take a look at line 207. This is the line that causes issues with the control indicator if the condition at line 48 is not present. You can observe the problems by disabling both the check on line 48 and line 211. But since the control indicator is already functioning correctly in our current version, we'll leave those as-is. Let's focus on understanding how fast-forwarding works in this basic model.

When the service code calls the LoopEventOnTime function, we are in pause mode. At this point, the loop starting at line 205 and ending at line 209 runs continuously, allowing the user to adjust the control indicator position. Here's what we do. Once the user presses the play button in the replay/simulator interface, we capture the asset data at line 210 for quicker access. Then, at line 211, we call the procedure defined at line 124. This is the procedure responsible for fast-forwarding from the current position to the one specified by the user.

At line 128, we check if the requested position matches the current one. If it does, the procedure ends, and the loop between lines 213 and 225 begins execution. If the desired position is ahead, we calculate the target offset at line 129. Nothing particularly magical so far, but the real trick happens at line 130, where we enter a loop that triggers line 131 repeatedly. This line calls the procedure responsible for generating bars. This procedure runs as quickly as possible until the position counter matches the calculated target. However, you'll notice that the procedure called in line 131 will execute the code between lines 69 and 95. Note that we pass a false parameter to this bar generation procedure. Because of this, the check at line 91 prevents the CustomTicksAdd function from being called. This means that no ticks are pushed to the symbol. Yet, at line 92, the bars are still rendered on the chart, one by one, as they're built.

Overall, the process works very well, except for two specific aspects that introduce noticeable delays during fast-forward execution. If you as the user request a large enough skip forward, you will actually see the bars being drawn on the chart. The main sources of this delay are lines 75 and 92, in addition to the procedure call itself. Though the latter's overhead is relatively small and can be ignored. Line 75, in particular, is a significant contributor to the delay.

This, then, is the basic way of implementing fast-forwarding. But there is a considerably faster alternative. If you want to see the bars being generated, this simpler approach is perfectly fine. By modifying the C_Replay.mqh file and including the code discussed above, basic fast-forwarding will already be functional. It's up to you whether to use this method or opt for the more advanced and significantly faster version that we'll explore next. To separate the information, let's move on to a new section.

Adding the Fast-Forward Feature (Dynamic Model)

If you take a closer look at the code, you'll notice that the C_FileTicks class already generates one-minute bars during tick loading. So why waste time rebuilding something that's already been created? Instead, we'll leverage those pre-built bars to get us as close as possible to the target point. If needed, we can then proceed to fast-forward to the exact calculated position. This makes the fast-forward process feel almost instantaneous.

Of course, not everything is perfect right out of the box. To achieve the desired level of efficiency, we'll need to introduce some kind of index or reference link for rapid lookups. Fortunately, we can repurpose part of the data structure that isn't particularly useful in the context of the replay/simulation system: the Spread field within the MqlRates structure. To understand the changes, take a look at the following code snippet:

126. //+------------------------------------------------------------------+
127.       bool BarsToTicks(const string szFileNameCSV, int MaxTickVolume)
128.          {
129.             C_FileBars     *pFileBars;
130.             C_Simulation   *pSimulator = NULL;
131.             int            iMem = m_Ticks.nTicks,
132.                            iRet = -1;
133.             MqlRates       rate[1];
134.             MqlTick        local[];
135.             bool           bInit = false;
136.             
137.             pFileBars = new C_FileBars(szFileNameCSV);
138.             ArrayResize(local, def_MaxSizeArray);
139.             Print("Converting bars to ticks. Please wait...");
140.             while ((*pFileBars).ReadBar(rate) && (!_StopFlag))
141.             {
142.                if (!bInit)
143.                {
144.                   m_Ticks.ModePlot = (rate[0].real_volume > 0 ? PRICE_EXCHANGE : PRICE_FOREX);
145.                   pSimulator = new C_Simulation(SetSymbolInfos());
146.                   bInit = true;
147.                }
148.                ArrayResize(m_Ticks.Rate, (m_Ticks.nRate > 0 ? m_Ticks.nRate + 3 : def_BarsDiary), def_BarsDiary);
149.                m_Ticks.Rate[++m_Ticks.nRate] = rate[0];
150.                if (pSimulator == NULL) iRet = -1; else iRet = (*pSimulator).Simulation(rate[0], local, MaxTickVolume);
151.                if (iRet < 0) break;
152.                rate[0].spread = m_Ticks.nTicks;
153.                for (int c0 = 0; c0 <= iRet; c0++)
154.                {
155.                   ArrayResize(m_Ticks.Info, (m_Ticks.nTicks + 1), def_MaxSizeArray);
156.                   m_Ticks.Info[m_Ticks.nTicks++] = local[c0];
157.                }
158.                m_Ticks.Rate[++m_Ticks.nRate] = rate[0];
159.             }
160.             ArrayFree(local);
161.             delete pFileBars;
162.             delete pSimulator;
163.             m_Ticks.bTickReal = false;
164.             
165.             return ((!_StopFlag) && (iMem != m_Ticks.nTicks) && (iRet > 0));
166.          }
167. //+------------------------------------------------------------------+
168.       datetime LoadTicks(const string szFileNameCSV, const bool ToReplay, const int MaxTickVolume)
169.          {
170.             int      MemNRates,
171.                      MemNTicks,
172.                      nDigits,
173.                      nShift;
174.             datetime dtRet = TimeCurrent();
175.             MqlRates RatesLocal[],
176.                      rate;
177.             MqlTick  TicksLocal[];
178.             bool     bNew;
179.             
180.             MemNRates = (m_Ticks.nRate < 0 ? 0 : m_Ticks.nRate);
181.             nShift = MemNTicks = m_Ticks.nTicks;
182.             if (!Open(szFileNameCSV)) return 0;
183.             if (!ReadAllsTicks()) return 0;         
184.             rate.time = 0;
185.             nDigits = SetSymbolInfos(); 
186.             m_Ticks.bTickReal = true;
187.             for (int c0 = MemNTicks, c1, MemShift = nShift; c0 < m_Ticks.nTicks; c0++, nShift++)
188.             {
189.                if (nShift != c0) m_Ticks.Info[nShift] = m_Ticks.Info[c0];
190.                if (!BuildBar1Min(c0, rate, bNew)) continue;
191.                if (bNew)
192.                {
193.                   if ((m_Ticks.nRate >= 0) && (ToReplay)) if (m_Ticks.Rate[m_Ticks.nRate].tick_volume > MaxTickVolume)
194.                   {
195.                      nShift = MemShift;
196.                      ArrayResize(TicksLocal, def_MaxSizeArray);
197.                      C_Simulation *pSimulator = new C_Simulation(nDigits);
198.                      if ((c1 = (*pSimulator).Simulation(m_Ticks.Rate[m_Ticks.nRate], TicksLocal, MaxTickVolume)) > 0)
199.                         nShift += ArrayCopy(m_Ticks.Info, TicksLocal, nShift, 0, c1);
200.                      delete pSimulator;
201.                      ArrayFree(TicksLocal);
202.                      if (c1 < 0) return 0;
203.                   }
204.                   rate.spread = MemShift;
205.                   MemShift = nShift;
206.                   ArrayResize(m_Ticks.Rate, (m_Ticks.nRate > 0 ? m_Ticks.nRate + 2 : def_BarsDiary), def_BarsDiary);
207.                };
208.                m_Ticks.Rate[(m_Ticks.nRate += (bNew ? 1 : 0))] = rate;
209.             }
210.             if (!ToReplay)
211.             {
212.                ArrayResize(RatesLocal, (m_Ticks.nRate - MemNRates));
213.                ArrayCopy(RatesLocal, m_Ticks.Rate, 0, 0);
214.                CustomRatesUpdate(def_SymbolReplay, RatesLocal, (m_Ticks.nRate - MemNRates));
215.                dtRet = m_Ticks.Rate[m_Ticks.nRate].time;
216.                m_Ticks.nRate = (MemNRates == 0 ? -1 : MemNRates);
217.                m_Ticks.nTicks = MemNTicks;
218.                ArrayFree(RatesLocal);
219.             }else   m_Ticks.nTicks = nShift;
220.                            
221.             return dtRet;
222.          };
223. //+------------------------------------------------------------------+

Code snippet from file C_FileTicks.mqh

This snippet highlights exactly which lines need to be modified in the C_FileTicks.mqh file. Take note that line 149 should be removed, or more accurately, relocated to a new position: line 158. But why make this change? Patience, dear reader. All will be explained in due course. Now, observe that a new line has been added: line 152. Pay close attention to this: the function here is transforming bars into ticks via simulation. At line 152, we capture the index value where a new bar is set to begin. This value is then stored in the spread field of the bar.

Let's move on to the next function, where we'll be doing something very similar. You' ll notice that only one new line has been added - line 204 - within the function responsible for reading ticks. But remember this key point: even though we're reading ticks from a file, there are cases where these ticks may need to be discarded and replaced with simulated ticks. This was discussed in previous articles, where I explained the reasoning behind such an approach. Because of that, the value we're truly interested in is the memory index MemShift, which tells us where the new bar begins. Just as we did in the previous function, we now store this value in the spread field of the MqlRates structure.

So, why are we doing this? What's the practical purpose? Let's clarify this now. During the simulation of ticks into bars, which occurs in the previous function, we know exactly when and where each bar begins. That's because we have the index pointing directly to the bar starting position before it's even simulated. The same holds true here, in the function that loads ticks from a file. At line 190, for example, ticks are converted into a bar, just like they would be inside the C_Replay class. As a result, every time a new bar is created here, we know precisely where it begins. That means we no longer need the C_Replay class to determine this starting point manually. And since the spread field holds no practical use in our replay/simulation context, we're repurposing it to store something valuable: the exact starting index of each bar.

If you recall from the previous section, the procedure that performs fast-forwarding includes a calculation at line 129. That calculation determines the exact index to which we need to jump in order to fast-forward the simulation efficiently. Starting to see the importance of this value, generated during tick loading? It's critical, as it allows us to significantly accelerate the fast-forwarding process. So, there's no need to reconstruct each bar one by one. We can jump directly to the correct point, then ask MetaTrader 5 to render and update the bars between the previous and current positions. In other words, we now have a new way of handling things.

To make effective use of this data, we'll need to modify a few aspects of the C_Replay class. Fortunately, these changes are minimal compared to what we addressed in the previous section. Next, you'll need to include the changes shown in the snippet below, by updating the C_Replay.mqh file accordingly.

013. #define def_MaxSlider             (def_MaxPosSlider + 1)

              ...

124. //+------------------------------------------------------------------+
125.       void AdjustPositionToReplay(void)
126.          {
127.             int nPos, nCount;
128.             
129.             if (m_IndControl.Position == (int)((m_Infos.CountReplay * def_MaxSlider) / m_MemoryData.nTicks)) return;
130.             nPos = (int)((m_MemoryData.nTicks * m_IndControl.Position) / def_MaxSlider);
131.             for (nCount = 0; m_MemoryData.Rate[nCount].spread < nPos; m_Infos.CountReplay = m_MemoryData.Rate[nCount++].spread);
132.             if (nCount > 0) CustomRatesUpdate(def_SymbolReplay, m_MemoryData.Rate, nCount - 1);
133.             while ((nPos > m_Infos.CountReplay) && def_CheckLoopService)
134.                CreateBarInReplay(false);
135.          }
136. //+------------------------------------------------------------------+

Code snippet from file C_Replay.mqh

As you can see, we had to add a new line to the C_Replay.mqh header file. This is line 13, where we apply a small correction to the overall offset calculation. And once again, everything here has a reason and purpose. If you don't make this small adjustment, you'll either encounter another issue or need to approach things differently. To avoid having to redo a large part of the data modeling, I prefer simply making the adjustment here. Now, why is this adjustment on line 13 important for us? The reason is that without it, you'll be subject to an offset where the final position of the slider in the control indicator would prematurely end the simulation or replay. By simply adding a position here, we ensure that a few extra ticks can still be applied to the chart. This is actually beneficial, as it guarantees a smoother ending for the replay or simulation process.

But the part that really matters is found in lines 131 and 132. By adding just these two lines, we achieve a significantly faster fast-forward than before. Still, there may be a few ticks left unprocessed, which must then be handled as previously. These ticks will use the loop beginning at line 133. However, since these remaining ticks are usually minimal, the process remains quite fast.

So, what are we actually doing here? At line 131, we search for the index of the bar whose value is immediately below our target position. This is done entirely within the for loop. Although this construction may seem unusual to most of you, it works perfectly well. Its unusual appearance is due to the fact that Iэm placing the assignment of the CountReplay value directly within the loop declaration. But you could certainly move this assignment outside the loop if you prefer.

Then at line 132, we must check the value of nCount. This is because I don't want to risk a failure in the CustomRatesUpdate call from the MQL5 library, which might not correctly interpret the number of data points or bars to be processed. The remainder of the function has already been explained in the previous section. What's interesting about these latest changes is that, as a result, the final code in the C_Replay.mqh file had to be updated again. Since these modifications are straightforward and don't require further explanation, Iэll simply show you the final version of the code (at least as it stands for now). You can find the complete code belowЖ

001. //+------------------------------------------------------------------+
002. #property copyright "Daniel Jose"
003. //+------------------------------------------------------------------+
004. #include "C_ConfigService.mqh"
005. #include "C_Controls.mqh"
006. //+------------------------------------------------------------------+
007. #define def_IndicatorControl   "Indicators\\Market Replay.ex5"
008. #resource "\\" + def_IndicatorControl
009. //+------------------------------------------------------------------+
010. #define def_CheckLoopService ((!_StopFlag) && (ChartSymbol(m_Infos.IdReplay) != ""))
011. //+------------------------------------------------------------------+
012. #define def_ShortNameIndControl    "Market Replay Control"
013. #define def_MaxSlider              (def_MaxPosSlider + 1)
014. //+------------------------------------------------------------------+
015. class C_Replay : public C_ConfigService
016. {
017.    private   :
018.       struct st00
019.       {
020.          C_Controls::eObjectControl Mode;
021.          uCast_Double               Memory;
022.          ushort                     Position;
023.          int                        Handle;
024.       }m_IndControl;
025.       struct st01
026.       {
027.          long     IdReplay;
028.          int      CountReplay;
029.          double   PointsPerTick;
030.          MqlTick  tick[1];
031.          MqlRates Rate[1];
032.       }m_Infos;
033.       stInfoTicks m_MemoryData;
034. //+------------------------------------------------------------------+
035. inline bool MsgError(string sz0) { Print(sz0); return false; }
036. //+------------------------------------------------------------------+
037. inline void UpdateIndicatorControl(void)
038.          {
039.             double Buff[];
040.                                  
041.             if (m_IndControl.Handle == INVALID_HANDLE) return;
042.             if (m_IndControl.Memory._16b[C_Controls::eCtrlPosition] == m_IndControl.Position)
043.             {
044.                if (CopyBuffer(m_IndControl.Handle, 0, 0, 1, Buff) == 1)
045.                   m_IndControl.Memory.dValue = Buff[0];
046.                if ((m_IndControl.Mode = (C_Controls::eObjectControl)m_IndControl.Memory._16b[C_Controls::eCtrlStatus]) == C_Controls::ePlay)
047.                   m_IndControl.Position = m_IndControl.Memory._16b[C_Controls::eCtrlPosition];
048.             }else
049.             {
050.                m_IndControl.Memory._16b[C_Controls::eCtrlPosition] = m_IndControl.Position;
051.                m_IndControl.Memory._16b[C_Controls::eCtrlStatus] = (ushort)m_IndControl.Mode;
052.                m_IndControl.Memory._8b[7] = 'D';
053.                m_IndControl.Memory._8b[6] = 'M';
054.                EventChartCustom(m_Infos.IdReplay, evCtrlReplayInit, 0, m_IndControl.Memory.dValue, "");
055.             }
056.          }
057. //+------------------------------------------------------------------+
058.       void SweepAndCloseChart(void)
059.          {
060.             long id;
061.             
062.             if ((id = ChartFirst()) > 0) do
063.             {
064.                if (ChartSymbol(id) == def_SymbolReplay)
065.                   ChartClose(id);
066.             }while ((id = ChartNext(id)) > 0);
067.          }
068. //+------------------------------------------------------------------+
069. inline void CreateBarInReplay(bool bViewTick)
070.          {
071.             bool    bNew;
072.             double dSpread;
073.             int    iRand = rand();
074. 
075.             if (BuildBar1Min(m_Infos.CountReplay, m_Infos.Rate[0], bNew))
076.             {
077.                m_Infos.tick[0] = m_MemoryData.Info[m_Infos.CountReplay];
078.                if (m_MemoryData.ModePlot == PRICE_EXCHANGE)
079.                {                  
080.                   dSpread = m_Infos.PointsPerTick + ((iRand > 29080) && (iRand < 32767) ? ((iRand & 1) == 1 ? m_Infos.PointsPerTick : 0 ) : 0 );
081.                   if (m_Infos.tick[0].last > m_Infos.tick[0].ask)
082.                   {
083.                      m_Infos.tick[0].ask = m_Infos.tick[0].last;
084.                      m_Infos.tick[0].bid = m_Infos.tick[0].last - dSpread;
085.                   }else if (m_Infos.tick[0].last < m_Infos.tick[0].bid)
086.                   {
087.                      m_Infos.tick[0].ask = m_Infos.tick[0].last + dSpread;
088.                      m_Infos.tick[0].bid = m_Infos.tick[0].last;
089.                   }
090.                }
091.                if (bViewTick) CustomTicksAdd(def_SymbolReplay, m_Infos.tick);
092.                CustomRatesUpdate(def_SymbolReplay, m_Infos.Rate);
093.             }
094.             m_Infos.CountReplay++;
095.          }
096. //+------------------------------------------------------------------+
097.       void AdjustViewDetails(void)
098.          {
099.             MqlRates rate[1];
100. 
101.             ChartSetInteger(m_Infos.IdReplay, CHART_SHOW_ASK_LINE, GetInfoTicks().ModePlot == PRICE_FOREX);
102.             ChartSetInteger(m_Infos.IdReplay, CHART_SHOW_BID_LINE, GetInfoTicks().ModePlot == PRICE_FOREX);
103.             ChartSetInteger(m_Infos.IdReplay, CHART_SHOW_LAST_LINE, GetInfoTicks().ModePlot == PRICE_EXCHANGE);
104.             m_Infos.PointsPerTick = SymbolInfoDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_SIZE);
105.             CopyRates(def_SymbolReplay, PERIOD_M1, 0, 1, rate);
106.             if ((m_Infos.CountReplay == 0) && (GetInfoTicks().ModePlot == PRICE_EXCHANGE))
107.                for (; GetInfoTicks().Info[m_Infos.CountReplay].volume_real == 0; m_Infos.CountReplay++);
108.             if (rate[0].close > 0)
109.             {
110.                if (GetInfoTicks().ModePlot == PRICE_EXCHANGE)
111.                   m_Infos.tick[0].last = rate[0].close;
112.                else
113.                {
114.                   m_Infos.tick[0].bid = rate[0].close;
115.                   m_Infos.tick[0].ask = rate[0].close + (rate[0].spread * m_Infos.PointsPerTick);
116.                }               
117.                m_Infos.tick[0].time = rate[0].time;
118.                m_Infos.tick[0].time_msc = rate[0].time * 1000;
119.             }else
120.                m_Infos.tick[0] = GetInfoTicks().Info[m_Infos.CountReplay];
121.             CustomTicksAdd(def_SymbolReplay, m_Infos.tick);
122.          }
123. //+------------------------------------------------------------------+
124.       void AdjustPositionToReplay(void)
125.          {
126.             int nPos, nCount;
127.             
128.             if (m_IndControl.Position == (int)((m_Infos.CountReplay * def_MaxSlider) / m_MemoryData.nTicks)) return;
129.             nPos = (int)((m_MemoryData.nTicks * m_IndControl.Position) / def_MaxSlider);
130.             for (nCount = 0; m_MemoryData.Rate[nCount].spread < nPos; m_Infos.CountReplay = m_MemoryData.Rate[nCount++].spread);
131.             if (nCount > 0) CustomRatesUpdate(def_SymbolReplay, m_MemoryData.Rate, nCount - 1);
132.             while ((nPos > m_Infos.CountReplay) && def_CheckLoopService)
133.                CreateBarInReplay(false);
134.          }
135. //+------------------------------------------------------------------+
136.    public   :
137. //+------------------------------------------------------------------+
138.       C_Replay()
139.          :C_ConfigService()
140.          {
141.             Print("************** Market Replay Service **************");
142.             srand(GetTickCount());
143.             SymbolSelect(def_SymbolReplay, false);
144.             CustomSymbolDelete(def_SymbolReplay);
145.             CustomSymbolCreate(def_SymbolReplay, StringFormat("Custom\\%s", def_SymbolReplay));
146.             CustomSymbolSetDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_SIZE, 0);
147.             CustomSymbolSetDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_VALUE, 0);
148.             CustomSymbolSetDouble(def_SymbolReplay, SYMBOL_VOLUME_STEP, 0);
149.             CustomSymbolSetString(def_SymbolReplay, SYMBOL_DESCRIPTION, "Symbol for replay / simulation");
150.             CustomSymbolSetInteger(def_SymbolReplay, SYMBOL_DIGITS, 8);
151.             SymbolSelect(def_SymbolReplay, true);
152.             m_Infos.CountReplay = 0;
153.             m_IndControl.Handle = INVALID_HANDLE;
154.             m_IndControl.Mode = C_Controls::ePause;
155.             m_IndControl.Position = 0;
156.             m_IndControl.Memory._16b[C_Controls::eCtrlPosition] = C_Controls::eTriState;
157.          }
158. //+------------------------------------------------------------------+
159.       ~C_Replay()
160.          {
161.             SweepAndCloseChart();
162.             IndicatorRelease(m_IndControl.Handle);
163.             SymbolSelect(def_SymbolReplay, false);
164.             CustomSymbolDelete(def_SymbolReplay);
165.             Print("Finished replay service...");
166.          }
167. //+------------------------------------------------------------------+
168.       bool OpenChartReplay(const ENUM_TIMEFRAMES arg1, const string szNameTemplate)
169.          {
170.             if (SymbolInfoDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_SIZE) == 0)
171.                return MsgError("Asset configuration is not complete, it remains to declare the size of the ticket.");
172.             if (SymbolInfoDouble(def_SymbolReplay, SYMBOL_TRADE_TICK_VALUE) == 0)
173.                return MsgError("Asset configuration is not complete, need to declare the ticket value.");
174.             if (SymbolInfoDouble(def_SymbolReplay, SYMBOL_VOLUME_STEP) == 0)
175.                return MsgError("Asset configuration not complete, need to declare the minimum volume.");
176.             SweepAndCloseChart();
177.             m_Infos.IdReplay = ChartOpen(def_SymbolReplay, arg1);
178.             if (!ChartApplyTemplate(m_Infos.IdReplay, szNameTemplate + ".tpl"))
179.                Print("Failed apply template: ", szNameTemplate, ".tpl Using template default.tpl");
180.             else
181.                Print("Apply template: ", szNameTemplate, ".tpl");
182. 
183.             return true;
184.          }
185. //+------------------------------------------------------------------+
186.       bool InitBaseControl(const ushort wait = 1000)
187.          {
188.             Print("Waiting for Mouse Indicator...");
189.             Sleep(wait);
190.             while ((def_CheckLoopService) && (ChartIndicatorGet(m_Infos.IdReplay, 0, "Indicator Mouse Study") == INVALID_HANDLE)) Sleep(200);
191.             if (def_CheckLoopService)
192.             {
193.                AdjustViewDetails();
194.                Print("Waiting for Control Indicator...");
195.                if ((m_IndControl.Handle = iCustom(ChartSymbol(m_Infos.IdReplay), ChartPeriod(m_Infos.IdReplay), "::" + def_IndicatorControl, m_Infos.IdReplay)) == INVALID_HANDLE) return false;
196.                ChartIndicatorAdd(m_Infos.IdReplay, 0, m_IndControl.Handle);
197.                UpdateIndicatorControl();
198.             }
199.             
200.             return def_CheckLoopService;
201.          }
202. //+------------------------------------------------------------------+
203.       bool LoopEventOnTime(void)
204.          {         
205.             int iPos;
206. 
207.             while ((def_CheckLoopService) && (m_IndControl.Mode != C_Controls::ePlay))
208.             {
209.                UpdateIndicatorControl();
210.                Sleep(200);
211.             }
212.             m_MemoryData = GetInfoTicks();
213.             AdjustPositionToReplay();
214.             iPos = 0;
215.             while ((m_Infos.CountReplay < m_MemoryData.nTicks) && (def_CheckLoopService))
216.             {
217.                if (m_IndControl.Mode == C_Controls::ePause) return true;
218.                iPos += (int)(m_Infos.CountReplay < (m_MemoryData.nTicks - 1) ? m_MemoryData.Info[m_Infos.CountReplay + 1].time_msc - m_MemoryData.Info[m_Infos.CountReplay].time_msc : 0);
219.                CreateBarInReplay(true);
220.                while ((iPos > 200) && (def_CheckLoopService))
221.                {
222.                   Sleep(195);
223.                   iPos -= 200;
224.                   m_IndControl.Position = (ushort)((m_Infos.CountReplay * def_MaxSlider) / m_MemoryData.nTicks);
225.                   UpdateIndicatorControl();
226.                }
227.             }
228. 
229.             return ((m_Infos.CountReplay == m_MemoryData.nTicks) && (def_CheckLoopService));
230.          }
231. };
232. //+------------------------------------------------------------------+
233. #undef macroRemoveSec
234. #undef def_SymbolReplay
235. #undef def_CheckLoopService
236. #undef def_MaxSlider
237. //+------------------------------------------------------------------+

Final source code of C_Replay.mqh file

Updating Bar Time and Quote Percentage

This issue is relatively simple to resolve. All we need to do is send messages to the mouse indicator so that it can properly interpret and display the relevant information. The task at hand is to have the indicator show how much time remains until a new bar begins and what the percentage change is between the previous day's close and the current quoted price.

To make things easier, we'll start by handling the percentage adjustment first If you prefer, you're welcome to create your own method based on the one I'll present here. Feel free to adapt it in whatever way best suits your needs. Let's begin by understanding the issue with the percentage. If you take a look at the mouse indicator, you'll notice that the percentage change between the previous close and the current quote is not being displayed correctly. However, the value based on the mouse position is accurate. Why is there such a discrepancy? At first glance, you might assume this is because the mouse indicator doesn't understand where the historical data ends and where the simulation or replay begins. But that's not actually the case. The mouse indicator is able to correctly read and interpret the data, which we can check by observing the variation while moving the mouse. What's really happening is that something is causing the mouse indicator to misinterpret data and occasionally display an odd value. That said, it sometimes does show the correct value. That's the problem we need to solve.

The solution is actually very simple. However, I want to give a word of caution: you should avoid overusing the approach I'm about to show. If you're not careful, you may lose control over the logic and flow of your development. With that said, let's see how the issue was resolved.

The first step is to modify the code of the mouse indicator as shown below:

09. #property indicator_chart_window
10. #property indicator_plots 0
11. #property indicator_buffers 1
12. //+------------------------------------------------------------------+
13. double GL_PriceClose;
14. //+------------------------------------------------------------------+
15. #include <Market Replay\Auxiliar\Study\C_Study.mqh>
16. //+------------------------------------------------------------------+
17. C_Study *Study       = NULL;
18. //+------------------------------------------------------------------+
19. input color user02   = clrBlack;                         //Price Line
20. input color user03   = clrPaleGreen;                     //Positive Study
21. input color user04   = clrLightCoral;                    //Negative Study
22. //+------------------------------------------------------------------+
23. C_Study::eStatusMarket m_Status;
24. int m_posBuff = 0;
25. double m_Buff[];
26. //+------------------------------------------------------------------+
27. int OnInit()
28. {
29.    ResetLastError();
30.    Study = new C_Study(0, "Indicator Mouse Study", user02, user03, user04);
31.    if (_LastError != ERR_SUCCESS) return INIT_FAILED;
32.    if ((*Study).GetInfoTerminal().szSymbol != def_SymbolReplay)
33.    {
34.       MarketBookAdd((*Study).GetInfoTerminal().szSymbol);
35.       OnBookEvent((*Study).GetInfoTerminal().szSymbol);
36.       m_Status = C_Study::eCloseMarket;
37.    }else
38.       m_Status = C_Study::eInReplay;
39.    SetIndexBuffer(0, m_Buff, INDICATOR_DATA);
40.    ArrayInitialize(m_Buff, EMPTY_VALUE);
41.    
42.    return INIT_SUCCEEDED;
43. }
44. //+------------------------------------------------------------------+
45. int OnCalculate(const int rates_total, const int prev_calculated, const datetime& time[], const double& open[],
46.                 const double& high[], const double& low[], const double& close[], const long& tick_volume[], 
47.                 const long& volume[], const int& spread[]) 
48. //int OnCalculate(const int rates_total, const int prev_calculated, const int begin, const double &price[])
49. {
50.    GL_PriceClose = close[rates_total - 1];
51.    m_posBuff = rates_total;
52.    (*Study).Update(m_Status);   
53.    
54.    return rates_total;
55. }
56. //+------------------------------------------------------------------+

Mouse pointer code snippet

Note that on line 13, I added a variable. This variable is global. And I don't mean "global" simply because it's declared outside of a function or procedure, but it's truly global due to its scope and placement in the code. Now, this variable doesn't do anything particularly magical. However, in line 50, it receives a value provided by MetaTrader 5. Also note that line 48, which previously held the declaration of the OnCalculate event function, has been replaced with a new version. It is very important. Now we can make use of the variable declared on line 13 to solve the percentage issue we discussed earlier. The next change should be made in the code within the C_Study.mqh header file. It is shown below:

41. //+------------------------------------------------------------------+
42.       void Draw(void)
43.          {
44.             double v1;
45.             
46.             if (m_Info.bvT)
47.             {
48.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn1, OBJPROP_YDISTANCE, GetInfoMouse().Position.Y_Adjusted - 18);
49.                ObjectSetString(GetInfoTerminal().ID, m_Info.szBtn1, OBJPROP_TEXT, m_Info.szInfo);
50.             }
51.             if (m_Info.bvD)
52.             {
53.                v1 = NormalizeDouble((((GetInfoMouse().Position.Price - m_Info.Rate.close) / m_Info.Rate.close) * 100.0), 2);
54.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn2, OBJPROP_YDISTANCE, GetInfoMouse().Position.Y_Adjusted - 1);
55.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn2, OBJPROP_BGCOLOR, (v1 < 0 ? m_Info.corN : m_Info.corP));
56.                ObjectSetString(GetInfoTerminal().ID, m_Info.szBtn2, OBJPROP_TEXT, StringFormat("%.2f%%", MathAbs(v1)));
57.             }
58.             if (m_Info.bvP)
59.             {
60.                v1 = NormalizeDouble((((GL_PriceClose - m_Info.Rate.close) / m_Info.Rate.close) * 100.0), 2);
61.                v1 = NormalizeDouble((((iClose(GetInfoTerminal().szSymbol, PERIOD_D1, 0) - m_Info.Rate.close) / m_Info.Rate.close) * 100.0), 2);
62.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn3, OBJPROP_YDISTANCE, GetInfoMouse().Position.Y_Adjusted - 1);
63.                ObjectSetInteger(GetInfoTerminal().ID, m_Info.szBtn3, OBJPROP_BGCOLOR, (v1 < 0 ? m_Info.corN : m_Info.corP));
64.                ObjectSetString(GetInfoTerminal().ID, m_Info.szBtn3, OBJPROP_TEXT, StringFormat("%.2f%%", MathAbs(v1)));
65.             }
66.          }
67. //+------------------------------------------------------------------+

Code snippet from file C_Study.mqh

You'll see that line 61, which previously held the old indicator logic, has been replaced by new logic in line 60. Notice that the global variable declared in the indicator file is being referenced here. How is this possible? The reason is that we declared the variable as global. Or more precisely, within the global scope of the file. This allows it to be accessed from any point in the code currently being built. This kind of global access can sometimes cause problems. That is why you must always be cautious when working with global variables.

When I need to use global variables (usually for a very specific reason) I do so consciously, knowing that they may cause trouble if not carefully handled. That's why I typically define them before any #include statements and prefix them with GL_ to help identify them clearly. Even though we also have a globally scoped variable on line 25, that one doesn’t worry me much. It serves a very specific purpose and is unlikely to be unintentionally altered. Pay attention to the variable in line 13 - that one does require extra care, as it's easy to accidentally modify it without realizing.

Through this simple change, we are resolving the issue of the percentage occasionally displaying incorrect values. In addition, we've also gained some performance improvement. This is because we no longer need to use iClose to fetch the closing price. MetaTrader 5 now provides this information directly, saving us the overhead of retrieving it manually.

Conclusion

Although we haven't yet addressed how to resolve the issue of tracking the time remaining until a bar closes, especially in the context of the replay/simulation, we've made great progress in this article. The application is now closely aligned with the behavior we had when we were relying on global terminal variables. I think that many of you are surprised by how much can be accomplished using plain MQL5, without external dependencies. However, this is just the beginning. There's a lot more to do, and each new step will bring both challenges and new opportunities to learn.

While I haven't yet explained how to get MetaTrader 5 to inform us of the remaining bar time when using the replay/simulation mode, I'll cover that right at the start of the next article. Don't miss it as we will also begin refining another aspect of the system that needs improvement to work seamlessly within the current setup.

Unfortunately, this still isn't the article where non-programmers will be able to fully use the application. That's simply because the time-to-close detail remains unresolved. But if you're a programmer and have been following along, applying the changes I've laid out, you'll see the replay/simulation feature already behaving as shown in the demo video below. See you in the next article. 

Demo video