Libraries: TimeGMT library for the strategy tester - page 2

[sap51]

I forgot to verify that the rest was compiling correctly on MT4.

Now I get the following errors, as some functions don't exist on MT4.

'TimeTradeServer' - function not defined                      TimeGMT.mqh    62    13
'SymbolExist' - function not defined                          TimeGMT.mqh    135    11
'SYMBOL_CALC_MODE_FOREX' - undeclared identifier              TimeGMT.mqh    146    25
'SYMBOL_CALC_MODE_FOREX_NO_LEVER…' - undeclared identifier    TimeGMT.mqh    146    63
'new_size' - variable already defined                         HashMap.mqh    604    14

Do you have any suggestion on how to add those missing functions/identifiers ?

I saw that fxsaber suggested a possible workaround to recreate the function TimeTradeServer() (see here).

I will try to find how to fix the other errors too.

[sap51]

Hello !

Thanks for your great libraries !

I've been working on making this library compatible with MQL4, as I want to be able to detect the correct GMT time for both MT4 and MT5.

While I was verifying the behavior of the current MQL5 code, I found a bug on the brokers that show a GOLD symbol that is not really Gold<->USD, and this breaks the GMT calculation.

An example of the screenshot below, where on my broker the real gold symbol is XAUUSD.p, and the GOLD symbol is something else that is not working correctly to detect the GMT time.

On my broker, the library will use the GOLD symbol and will fail to detect the GMT time, (logs below):

2024.07.06 15:27:12.781  Core 1  GOLD: history ticks synchronized from 2023.12.04 to 2024.06.28
2024.07.06 15:27:12.781  Core 1  2022.01.01 00:00:00  >> CTimeGMT::HistoryBrokerOffset() FAILED: could not calculate broker offset for the requested date Sat, 2022.01.01 00:00. 

I was able to confirm the behavior by adding an INPUT variable in the Expert file to allow or forbid the usage of GOLD as the symbol for Gold<->USD.

$ cat TimeGMT_TestEA.mq5 | grep Input_Force
input bool Input_ForceXAUforGold = false; // Force the Gold symbol to start with XAU

$ diff TimeGMT-original.mqh TimeGMT.mqh
<             if(StringFind(SymbolName(i,0),"XAU")==0||StringFind(SymbolName(i,0),"GOLD")==0)
---
>             if(StringFind(SymbolName(i,0),"XAU")==0||(StringFind(SymbolName(i,0),"GOLD")==0 && Input_ForceXAUforGold == false))

When I re-run the test by setting the INPUT variable to True, to force the usage of a symbol that starts with XAU, the GMT detection works better.

2024.07.06 15:40:25.254	Core 1	XAUEUR.p: history ticks synchronized from 2023.01.03 to 2024.06.28
2024.07.06 15:40:43.566	Core 1	2022.03.15 00:00:00  TimeCurrent() = Fri, 2022.03.11 23:56  | TimeTradeServer() = Mon, 2022.03.14 00:00  | TimeGMT() = Sun, 2022.03.13 22:00 | BrokerOffset = 7200 (GMT+2) 

We can see that the first symbol that starts with XAU is XAUEUR.p, so the library took this one instead of XAUUSD.

It's probably fine, as I'm assuming that XAUEUR and XAUUSD will open at the same time.

I suggest a small improvement in the code, in order to tell the library to first try to find the Gold symbol using the XAUUSD prefix, then if not found use XAU, and if still not found use the GOLD symbol:

      //--- Forex pairs start Sun 17:00 NY. Gold usually starts an hour later.
      string symbol = "XAUUSD";
      bool is_custom = false;
      bool symbol_found = false; // Store if we have found a valid symbol or not yet
      if(!SymbolExist(symbol,is_custom))
         for(int i=0; i < SymbolsTotal(0); i++)
            if(StringFind(SymbolName(i,0),"XAUUSD")==0) // First try to find a symbol that starts with XAUUSD
              {
               symbol=SymbolName(i,0); // symbol may have suffix
               symbol_found = true;
               break;
              }
         if(symbol_found == false)
            for(int i=0; i < SymbolsTotal(0); i++)
               if(StringFind(SymbolName(i,0),"XAU")==0) // Then try to find a symbol that starts with XAU (XAUEUR, etc)
               {
                  symbol=SymbolName(i,0); // symbol may have suffix
                  symbol_found = true;
                  break;
               }
         if(symbol_found == false)
            for(int i=0; i < SymbolsTotal(0); i++)
               if(StringFind(SymbolName(i,0),"GOLD")==0) // And finally try to find a symbol that starts with GOLD
               {
                  symbol=SymbolName(i,0); // symbol may have suffix
                  symbol_found = true;
                  break;
               }
      //--- Few brokers do not provide gold trading, so we must use forex symbol 

On a side note, I saw some broker using lowercase letters for the Gold symbol, so maybe the code should be updated to ignore the case of the symbols when trying to find a symbol that match.

I will continue to work on converting this library to MQL4, and I will let you know if I see other things that could be improved.

Thanks

Thomas

[fxsaber]

SYMBOL_DESCRIPTION

[sap51]

fxsaber #:

SYMBOL_DESCRIPTION

Good point, we could be using the Symbol Description to detect the right gold symbol.

I'm also curious why most people seems to want to use the Sunday bar to detect the broker time settings, where we could in fact use SymbolInfoSessionTrade to detect which days the market is open, and use that to detect the broker time settings.

[amrali]

sap51 #:

While I was verifying the behavior of the current MQL5 code, I found a bug on the brokers that show a GOLD symbol that is not really Gold<->USD, and this breaks the GMT calculation.

Thank you for reporting this bug..

I will update the code with your fix.

[amrali]

sap51 #:
I'm also curious why most people seems to want to use the Sunday bar to detect the broker time settings, where we could in fact use SymbolInfoSessionTrade to detect which days the market is open, and use that to detect the broker time settings.

SymbolInfoSessionTrade returns session times for the current week only. So, how to calculate GMT offset for any previous week?

[sap51]

amrali #:
SymbolInfoSessionTrade returns session times for the current week only. So, how to calculate GMT offset for any previous week?

Do we know if the session times actually changes in the SymbolInfoSessionTrade through the year ?

I have the feeling it's not changing, i.e. it's always the same value all year long, but I could be wrong.

[sap51]

I have also found that if we update the Expert Advisor code to show exactly at what minute the Offset change, we can see that it's switching on/off the Daylight Savings on the right day, but on the wrong time.

According to the page https://www.timeanddate.com/time/zone/usa/new-york, the time changes for the NewYork timezone is 02:00 AM local time.




If we convert that back to UTC it's 07:00 AM UTC.

The last minute that has the "old" offset should be 01:59 on the NewYork timezone, and the next minute's offset should either be on at 02:00 AM NewYork if we are reducing 1 hour, or at 03:00 NewYork if we are adding one hour.
In UTC timezone it should be +4h or +5h, depending on if we are entering DST or exiting.

See this page https://www.timeanddate.com/worldclock/converter.html?iso=20200308T065900&p1=179&p2=1440

I think the problem is that you are only providing the day of the DST, and not the actual time in hours, so maybe the following code need to be improved to be more precise ?

datetime dst_start = StringToTime((string)dt.year+".03.08");
datetime dst_end   = StringToTime((string)dt.year+".11.01");

Attached the custom version of the Expert that shows the exact minute where the time is changing.

I will try to see how this can be fixed, but I'm not sure just yet

[sap51]

I just did one more verification regarding the usage of SymbolInfoSessionTrade.

This week's Thursday was a special day for the US, as it was independence day.

I received an email from the FXView broker to let me know that the trading times will be modified.



But when we look on the Symbol specifications, it doesn't look like it has changed.



So either the specifications are never changing, no matter if there are some special days or not, or they changed on Thursday and reverted to the default on Friday already.

[amrali]

Update 7 July 2024 - version 1.36

Fixed issue in proper detection of the GOLD symbol on some brokers. Thanks @sap51 for reporting this bug.