How to visualize multicurrency trading history based on HTML and CSV reports

Since its introduction, MetaTrader 5 provides multicurrency testing options. This possibility is often used by traders. However the function is not universal. In particular, after running a test, the user can open a chart with performed trading operations. But this is only a chart of one traded symbol selected in the strategy tester settings. The entire trading history of all used symbols cannot be viewed after testing, while visual examination is not always efficient. Additional analysis may be required after some time after testing. Also, a report can be provided by another person. Therefore, a tool for visualizing trading on multiple working symbols based on the HTML testing report would be very useful.

This task is closely related to another similar MetaTrader application. Many of the trading signals available on mql5.com involve multicurrency trading. It would be convenient to display the CSV files with the signals history in charts.

Let us develop an indicator which can perform the aforementioned functions.

To enable the parallel analysis of multiple working symbols, several indicator instances (one per symbol) will be created in chart subwindows. The main graphical objects will be the "quotes" of the selected symbol (which is usually different from the chart symbol), synchronized with the main window bars. Trend lines corresponding to trading orders (positions) will be applied to these "quotes".

There is an alternative approach: deals are displayed in the main window, but only one symbol is analyzed on the chart in this case. This approach requires another indicator without buffers, with the possibility to switch to any of the symbols included in the report.

The previous article provided a description of the HTML parser based on CSS selectors[1]. The parser extracts the list of deals from the HTML report, based on which we can trades can be formed (graphical objects). Parsing of CSV files from the Signals section is a bit easier, while the file format for the MetaTrader 4 (*.history.csv) and MetaTrader 5 (*.positions.csv) signals is supported by the built-in MQL functions.

The SubChart indicator

The first step in the implementation is to create a simple indicator, which displays "quotes" of an external symbol in a subwindow of any chart. This will be the SubChart indicator.

To display data with OHLC (Open, High, Low, Close) values, the MQL provides multiple display styles, including DRAW_CANDLES and DRAW_BARS. Each of them uses four indicator buffers. Instead of selecting one of the styles, we will support both options. The style will be selected dynamically based on the current window settings. A groups of radio buttons is available in chart settings, under the Common tab: "Bars", "Japanese candlesticks" and "Line". For quick access, the same buttons are available as buttons on the toolbar. The settings can be obtained from MQL using the following call:

(ENUM_CHART_MODE)ChartGetInteger(0, CHART_MODE)

The ENUM_CHART_MODE enumeration contains elements with a similar purpose: CHART_CANDLES, CHART_BARS, CHART_LINE.

Support for the last CHART_LINE point will also be implemented in the indicator. Thus the indicator view will be changed in accordance with the main window, when the display mode in the UI is switched. DRAW_LINE is suitable for the last mode; it uses one buffer.

Let's start the implementation. Declare the number of indicator buffers and of displayed graphical object types:

#property indicator_separate_window
#property indicator_buffers 4
#property indicator_plots   1

Add input variables:

input string SubSymbol = ""; // Symbol
input bool Exact = false;

Using SubSymbol, it will be possible to set for the indicator any symbol other than the current symbol in the main window.

The Exact parameter determines actions for the cases where bars in the main window do not have matching bars of another symbol with exactly the same time. The parameter will be used in the iBarShift function call. Its visual effect will be as follows:

• if Exact is equal to 'false', the function returns the number of the nearest suitable bar, which is close to the specified time, and thus if there are no quotes (due to holidays or for other reasons), the indicator will display the previous bar;
• if Exact = true, the function returns -1, and thus the indicator chart at this position will be empty;

The SubSymbol parameter is equal to an empty string by default. This means that quotes are identical to those displayed in the main window. In this case, you should edit the actual variable value and set it to _Symbol. However since 'input' is a read-only variable in MQL, we will have to enter the intermediate variable 'Symbol' and fill it in the OnInit handler.

string Symbol;

int OnInit()
{
  Symbol = SubSymbol;
  if(Symbol == "") Symbol = _Symbol;
  else SymbolSelect(Symbol, true);
  ...

Please note that this symbol should also be added to the Market Watch, since it may be absent it the Market Watch list.

Use the 'mode' variable to control the current display mode:

ENUM_CHART_MODE mode = 0;

The following four indicator buffers will be used:

// OHLC
double open[];
double high[];
double low[];
double close[];

Let us add a small function to initialize buffers in the usual way, (with the setting of the "series" property):

void InitBuffer(int index, double &buffer[], ENUM_INDEXBUFFER_TYPE style)
{
  SetIndexBuffer(index, buffer, style);
  ArraySetAsSeries(buffer, true);
}

Initialization of graphics is also performed in one auxiliary function (this indicator has only one graphical construction; the function can save time if multiple constructions are used):

void InitPlot(int index, string name, int style, int width = -1, int colorx = -1)
{
  PlotIndexSetInteger(index, PLOT_DRAW_TYPE, style);
  PlotIndexSetDouble(index, PLOT_EMPTY_VALUE, 0);
  PlotIndexSetString(index, PLOT_LABEL, name);
  if(width != -1) PlotIndexSetInteger(index, PLOT_LINE_WIDTH, width);
  if(colorx != -1) PlotIndexSetInteger(index, PLOT_LINE_COLOR, colorx);
}

The following function will switch chart display mode into the buffer style:

int Mode2Style(/*global ENUM_CHART_MODE mode*/)
{
  switch(mode)
  {
    case CHART_CANDLES: return DRAW_CANDLES;
    case CHART_BARS: return DRAW_BARS;
    case CHART_LINE: return DRAW_LINE;
  }
  return 0;
}

The function uses the aforementioned global variable 'mode' which should be filled with a relevant value in OnInit, along with the calls of all auxiliary functions.

  InitBuffer(0, open, INDICATOR_DATA);
  string title = "# Open;# High;# Low;# Close";
  StringReplace(title, "#", Symbol);
  mode = (ENUM_CHART_MODE)ChartGetInteger(0, CHART_MODE);
  InitPlot(0, title, Mode2Style());

  InitBuffer(1, high, INDICATOR_DATA);
  InitBuffer(2, low, INDICATOR_DATA);
  InitBuffer(3, close, INDICATOR_DATA);

This is still not enough for the proper indicator operation. Line color should be changed depending on the current mode (the mode variable): the colors are provided by the chart settings.

void SetPlotColors()
{
  if(mode == CHART_CANDLES)
  {
    PlotIndexSetInteger(0, PLOT_COLOR_INDEXES, 3);
    PlotIndexSetInteger(0, PLOT_LINE_COLOR, 0, (int)ChartGetInteger(0, CHART_COLOR_CHART_LINE));  // rectangle
    PlotIndexSetInteger(0, PLOT_LINE_COLOR, 1, (int)ChartGetInteger(0, CHART_COLOR_CANDLE_BULL)); // up
    PlotIndexSetInteger(0, PLOT_LINE_COLOR, 2, (int)ChartGetInteger(0, CHART_COLOR_CANDLE_BEAR)); // down
  }
  else
  {
    PlotIndexSetInteger(0, PLOT_COLOR_INDEXES, 1);
    PlotIndexSetInteger(0, PLOT_LINE_COLOR, (int)ChartGetInteger(0, CHART_COLOR_CHART_LINE));
  }
}

The addition of the SetPlotColors() call in OnInit and setting of the value accuracy guarantee the correct indicator display after the launch.

  SetPlotColors();

  IndicatorSetString(INDICATOR_SHORTNAME, "SubChart (" + Symbol + ")");
  IndicatorSetInteger(INDICATOR_DIGITS, (int)SymbolInfoInteger(Symbol, SYMBOL_DIGITS));
  
  return INIT_SUCCEEDED;
}

However, if the user changes the chart mode while the indicator is running, it is necessary to track this event and modify the properties of the buffers. This will be done by the OnChartEvent handler.

void OnChartEvent(const int id,
                  const long& lparam,
                  const double& dparam,
                  const string& sparam)
{
  if(id == CHARTEVENT_CHART_CHANGE)
  {
    mode = (ENUM_CHART_MODE)ChartGetInteger(0, CHART_MODE);
    PlotIndexSetInteger(0, PLOT_DRAW_TYPE, Mode2Style());
    SetPlotColors();
    ChartRedraw();
  }
}

Now we need to write the most important indicator function, the OnCalculate handler. The specific feature of this handler is that the indicator actually uses third-party symbol quotes instead of the chart symbol. Therefore, all standard programming techniques based on the rates_total and prev_calculated values, which are usually passed from the kernel, are not suitable or are only partially suitable. Quotes of the third-party symbol will be downloaded asynchronously, and therefore a new batch of bars may "arrive" at any time - this situation requires full recalculation. Therefore, let is create variables, which control the number of bars on that symbol (lastAvailable) and the editable "clone" of the constant prev_calculated argument.

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime& time[],
                const double& op[],
                const double& hi[],
                const double& lo[],
                const double& cl[],
                const long& tick_volume[],
                const long& volume[],
                const int& spread[])
{
  static int lastAvailable = 0;
  static bool initialized = false;

  int _prev_calculated = prev_calculated;

  if(iBars(Symbol, _Period) - lastAvailable > 1) // bar gap filled
  {
    _prev_calculated = 0;
    lastAvailable = 0;
    initialized = false;
  }

  if(_prev_calculated == 0)
  {
    for(int i = 0; i < rates_total; ++i)
    {
      open[i] = 0;
      high[i] = 0;
      low[i] = 0;
      close[i] = 0;
    }
  }

If the indicator symbol is different from the window symbol, use the iBarShift function to find synchronous bars and to copy their OHLC values.

  if(_Symbol != Symbol)
  {
    for(int i = 0; i < MathMax(rates_total - _prev_calculated, 1); ++i)
    {
      datetime dt = iTime(_Symbol, _Period, i);
      int x = iBarShift(Symbol, _Period, dt, Exact);
      if(x != -1)
      {
        open[i] = iOpen(Symbol, _Period, x);
        high[i] = iHigh(Symbol, _Period, x);
        low[i] = iLow(Symbol, _Period, x);
        close[i] = iClose(Symbol, _Period, x);
      }
    }
  }

If the indicator symbol matches the window symbol, simply use the passed arrays-arguments:

  else
  {
    ArraySetAsSeries(op, true);
    ArraySetAsSeries(hi, true);
    ArraySetAsSeries(lo, true);
    ArraySetAsSeries(cl, true);
    for(int i = 0; i < MathMax(rates_total - _prev_calculated, 1); ++i)
    {
      open[i] = op[i];
      high[i] = hi[i];
      low[i] = lo[i];
      close[i] = cl[i];
    }
  }

Finally, we need to provide data uploading. Let us use the implementation of the RefreshHistory function from the MetaQuotes' example (we will include this code as the Refresh.mqh header file).

The 'initialized' static variable contains a sign of the update completion. Set it to true if RefreshHistory returns a sign of success or if the number of third-party symbol bars remains constant and non-zero (in case there is no history for the required number of bars).

  if(lastAvailable == iBars(Symbol, _Period) && lastAvailable != 0)
  {
    if(!initialized)
    {
      Print("Updated ", Symbol, " ", iBars(Symbol, _Period), " bars");
      initialized = true;
    }
    return rates_total;
  }

  if(!initialized)
  {
    if(_Symbol != Symbol)
    {
      Print("Updating ", Symbol, " ", lastAvailable, " -> ", iBars(Symbol, _Period), " bars up to ", (string)time[0], "... Please wait");
      int result = RefreshHistory(Symbol, time[0]);
      if(result >= 0 && result <= 2)
      {
        _prev_calculated = rates_total;
      }
      if(result >= 0)
      {
        initialized = true;
        ChartSetSymbolPeriod(0, _Symbol, _Period);
      }
    }
    else
    {
      initialized = true;
    }
  }
  
  lastAvailable = iBars(Symbol, _Period);
  
  return _Symbol != Symbol ? _prev_calculated : rates_total;
}

If the data loading process takes too much time, the manual chart refreshing can be required.

After initialization completion, the new bars will be calculated in the resource-saving mode, i.e. taking into account _prev_calculated and rates_total. If the number of lastAvailable bars changes by more than 1, all bars should be redrawn anew. For this purpose 'initialized' should be reset to false.

Attach the indicator to a chart, let's say EURUSD and enter another symbol in the parameters, for example UKBrent (Brent CFD; it is interesting because when 'Exact' is true, the absence of night bars on intraday timeframes is obvious). Click the mode change buttons and make sure that the indicator drawing is correct.

Switching the indicator display mode

Note that in the linear display mode, the indicator uses the first buffer (index 0) open. This is different from the main chart, which is based on close prices. This approach allows to avoid complete indicator redrawing when switching to or from the linear representation: to display the close price based line, Close prices need to be copied to the first (the only one) buffer, while in candlestick and bar modes (when 4 buffers are used) the first buffer stores Open prices. The current implementation is based on the fact that the open buffer is the first one in OHLC, and therefore a change of style immediately leads to a change in the representation without recalculation. The line mode will hardly be used much in history analysis, therefore this feature is not critical.

Run the indicator in the visual testing mode.

The SubChart indicator in visual testing mode

Now we can proceed to trading history visualization based on this indicator.

The SubChartReporter indicator

Let's name the new indicator SubChartReporter. Add to the previously created code the possibility to read HTML and CSV reports. The name of the file for the analysis will be passed in the ReportFile input variable. Let us also provide input parameters for specifying the time shift, as well as for specifying symbol prefixes and suffixes when a report is received from another user (from a different trading environment).

input string ReportFile = ""; // · ReportFile
input string Prefix = ""; // · Prefix
input string Suffix = ""; // · Suffix
input int  TimeShift = 0; // · TimeShift

Special classes in the SubChartReporter indicator will process receiving of data and generation of graphical objects (trendlines).

Since we want to analyze HTML along with CSV files, a general basic class was designed for all report types - Processor. Specific implementations for HTML and CSV were inherited from it: ReportProcessor and HistoryProcessor.

The following variables are described in Processor:

class Processor
{
  protected:
    string symbol;
    string realsymbol;
    IndexMap *data;
    ulong timestamp;
    string prefix;

• symbol — the name of the current working symbol from the report;
• realsymbol — the name of the real available working symbol, which may differ from those used in third-party reports due to prefixes and suffixes;
• data — an array of trading operations (the IndexMap class is already familiar from article [1], it is connected from the header file with the same name);
• timestamp and prefix — auxiliary variables for the unique naming of on-chart graphical objects which will be generated by the indicator;

To enable switching between different report symbols, let us add interface buttons. The currently pressed button ID (corresponds to the selected symbol) will be stored in a variable:

    string pressed;

The following auxiliary methods for object generation are implemented in the Processor class:

    void createTrend(const long dealIn, const long dealOut, const int type, const datetime time1, const double price1, const datetime time2, const double price2, const string &description)
    void createButton(const int x, const int y, const int dx, const int dy, const string text, const bool selected)
    void controlPanel(const IndexMap &symbols)

createTrend creates the visual representation for a separate trade (trendline and two arrows), createButton crates a working symbol button, controlPanel is the full set of buttons for all symbols presented in the report. The buttons are displayed in the lower left corner of the subwindow.

The public interface of the Processor class includes two groups of methods:

• virtual methods, which are subject to a specific implementation in child classes;
• non-virtual methods, providing single standard functionality;

    virtual IndexMap *load(const string file) = 0;
    virtual int getColumnCount() = 0;
    virtual int getSymbolColumn() = 0;
    virtual datetime getStart() = 0;
    virtual bool applyInit() { return true; }
    virtual void makeTrade(IndexMap *row) = 0;
    virtual int render() = 0;

    bool attach(const string file)
    bool apply(const string _s = NULL)
    bool isEmpty() const
    string findrealsymbol()
    string _symbol() const
    string _realsymbol() const
    void onChartEvent(const int id, const long& lparam, const double& dparam, const string& sparam)

The full source code is attached below. Here I will only provide a brief description.

• load — uploads the specified data file
• getColumnCount — returns the number of columns in the data table
• getSymbolColumn — returns the index of the column with the name of trading operation symbols
• getStart — returns the date of the first trading operation
• applyInit — method for optional initialization of internal data structures before processing
• makeTrade — method for the registration of one record from a report in internal data structures
• render — method for the generation of graphical objects based on records in internal data structures

The last two methods exist separately, because some report formats, such as HTML reports in MetaTrader 5 include records of deals, however we need to display positions. Therefore, we need an additional conversion of one type of entities to another, with the need to view an array of deals.

Non-virtual methods are presented below in a simplified form. The attach method receives the name of the file for analysis, loads it using the virtual 'load' method, creates a list of unique symbols and creates a "panel" of buttons for these symbols.

    bool attach(const string file)
    {
      data = load(file);
      
      IndexMap symbols;
      
      for(int i = 0; i < data.getSize(); ++i)
      {
        IndexMap *row = data[i];
        // collect all unique symbols
        string s = row[getSymbolColumn()].get<string>();
        StringTrimLeft(s);
        if(StringLen(s) > 0) symbols.set(s);
      }

      if(symbols.getSize() > 0)
      {
        controlPanel(symbols);
      }
      return true;
    }

The 'apply' method activates in the indicator the selected working symbol from among those available in the report. First, old objects are deleted from the chart (if there are any), then a matching real symbol is searched (for example, EURUSD provided by your broker instead of EURUSD.m, which is specified in the report). Here, child classes can reset old internal arrays using applyInit. Then trades are formed based on table entries with the matching symbol (call of makeTrade). Graphical objects are drawn based on these trades (call of render) and the interface is updated (active button selection, indicator name change, call of ChartRedraw).

                                                                                                                                          
    bool apply(const string _s = NULL)
    {
      ObjectsDeleteAll(0, "SCR", ChartWindowFind(), OBJ_TREND);

      if(_s != NULL && _s != "") symbol = _s;
      if(symbol == NULL)
      {
        Print("No symbol selected");
        return false;
      }
      
      string real = findrealsymbol();
      if(real == NULL)
      {
        Print("No suitable symbol found");
        return false;
      }
      
      SymbolSelect(real, true);

      if(!applyInit()) return false;
      
      int selected = 0;
  
      for(int i = 0; i < data.getSize(); ++i)
      {
        IndexMap *row = data[i];
        
        string s = row[getSymbolColumn()].get<string>();
        StringTrimLeft(s);
        
        if(s == symbol)
        {
          selected++;
          makeTrade(row);
        }
      }
      
      pressed = prefix + "#" + symbol;
      ObjectSetInteger(0, pressed, OBJPROP_BGCOLOR, clrGreen);
      
      int trends = render();
      Print(data.getSize(), " records in total");
      Print(selected, " trades for ", symbol);
      
      string title = CHART_REPORTER_TITLE + " (" + symbol + ", " + (string)selected + " records, " + (string)trends + " trades)";
      IndicatorSetString(INDICATOR_SHORTNAME, title);
      
      ChartRedraw();
      return true;
    }

The findrealsymbol method uses multiple approaches to search for matching symbols. It checks availability of market data (bid prices) for the symbol. If the data is found, the symbol is considered real. If there is no market data, the program tries to match the Suffix and/or Prefix parameters (if the suffix or prefix is specified), i.e. it adds or removes them from the symbol name. If the price is received after modification, it means that a valid alias has been found for the symbol.

Methods _symbol and _realsymbol return the name of the current symbol from the report and return this symbol or its valid counterpart for your account. When analyzing your own reports, you will receive the same symbol name, unless the broker excludes your symbol.

The onChartEvent method handles OnChartEvent, i.e. button click events. The previous selected button (if there is any) becomes gray like all other buttons. The important part is the call of the virtual 'apply' method, to which the name of the new symbol extracted from the button identifier is passed.

    void onChartEvent(const int id, const long& lparam, const double& dparam, const string& sparam)
    {
      if(id == CHARTEVENT_OBJECT_CLICK)
      {
        int x = StringFind(sparam, "_#");
        if(x != -1)
        {
          string s = StringSubstr(sparam, x + 2);
          Print(s, " ", sparam, " ", pressed);
          
          ObjectSetInteger(0, sparam, OBJPROP_STATE, false);
          ObjectSetInteger(0, pressed, OBJPROP_STATE, false);
          ObjectSetInteger(0, pressed, OBJPROP_BGCOLOR, clrGray);
          pressed = "";
          
          if(apply(s)) // will set pressed and other properties
          {
            ChartSetSymbolPeriod(0, _Symbol, _Period);
          }
        }
      }
    }

Now let's move on to implementing the main functionality in the virtual methods of child classes. Let's start with ReportProcessor.

The HTML pages will be parsed using WebDataExtractor from article [1]. The parser will be used as the include file WebDataExtractor.mqh. Compared to the original source code, all working methods in this file are wrapped in the HTMLConverter class in order not to litter the global context. We are first of all interested in the HTMLConverter::convertReport2Map method, which almost completely coincides with the 'process' function discussed in [1]. The report file name from ReportFile is input into convertReport2Map. The function output is the IndexMap with strings corresponding to trading operations in the report table.

All settings required for HTML report parsing, such as RowSelector and ColumnSettingsFile, are already specified in the header file. But they can be edited since they are described as input parameters. Default parameters apply to MetaTrader 5 reports, from which a table of deals is parsed and further displayed positions are calculated based on these deals. Each deal is described by an instance of the special 'Deal' class. The 'Deal' constructor receives one entry from the report table, wrapped in IndexMap. The following fields are stored inside 'Deal': deal time and price, its type and direction, volume and other properties.

class ReportProcessor: public Processor
{
  private:
    class Deal   // if MQL5 could respect private access specifier for classes,
    {            // Trades will be unreachable from outer world, so it would be fine to have
      public:    // fields made public for direct access from Processor only
        datetime time;
        double price;
        int type;      // +1 - buy, -1 - sell
        int direction; // +1 - in, -1 - out, 0 - in/out
        double volume;
        double profit;
        long deal;
        long order;
        string comment;
        
      public:
        Deal(const IndexMap *row) // this is MT5 deal
        {
          time = StringToTime(row[COLUMN_TIME].get<string>()) + TimeShift;
          price = StringToDouble(row[COLUMN_PRICE].get<string>());
          string t = row[COLUMN_TYPE].get<string>();
          type = t == "buy" ? +1 : (t == "sell" ? -1 : 0);
          t = row[COLUMN_DIRECTION].get<string>();
          direction = 0;
          if(StringFind(t, "in") > -1) ++direction;
          if(StringFind(t, "out") > -1) --direction;
          volume = StringToDouble(row[COLUMN_VOLUME].get<string>());
          t = row[COLUMN_PROFIT].get<string>();
          StringReplace(t, " ", "");
          profit = StringToDouble(t);
          deal = StringToInteger(row[COLUMN_DEAL].get<string>());
          order = StringToInteger(row[COLUMN_ORDER].get<string>());
          comment = row[COLUMN_COMMENT].get<string>();
        }
    
        bool isIn() const
        {
          return direction >= 0;
        }
        
        bool isOut() const
        {
          return direction <= 0;
        }
        
        bool isOpposite(const Deal *t) const
        {
          return type * t.type < 0;
        }
        
        bool isActive() const
        {
          return volume > 0;
        }
    };

All deals are added into 'array'. In the process of history analysis, market entry deals will be added to queue and will be removed from it when a corresponding opposite exit deal is found. If the queue is non-empty after passing through the whole history, then there is an open position.

    RubbArray<Deal *> array;
    RubbArray<Deal *> queue;

The RubbArray class is a wrapper for the dynamic array which is automatically expanded to fit incoming data.

The implementation of some virtual methods is shown below:

    virtual IndexMap *load(const string file) override
    {
      return HTMLConverter::convertReport2Map(file, true);
    }

    virtual int getColumnCount() override
    {
      return COLUMNS_COUNT;
    }

    virtual int getSymbolColumn() override
    {
      return COLUMN_SYMBOL;
    }

The last two methods use macro definitions for the standard table of deals from the MetaTrader 5 HTML report.

#define COLUMNS_COUNT 13
#define COLUMN_TIME 0
#define COLUMN_DEAL 1
#define COLUMN_SYMBOL 2
...

The applyInit method clears the 'array' and 'queue' arrays.

    virtual bool applyInit() override
    {
      ((BaseArray<Deal *> *)&queue).clear();
      array.clear();
      return true;
    }

Deal objects are created and added to 'array' in the makeTrade method.

    virtual void makeTrade(IndexMap *row) override
    {
      array << new Deal(row);
    }

Finally, the most interesting and also the most difficult part is the analysis of the deals list and the generation of trade objects on their basis.

    virtual int render() override
    {
      int count = 0;
      
      for(int i = 0; i < array.size(); ++i)
      {
        Deal *current = array[i];
        
        if(!current.isActive()) continue;
        
        if(current.isOut())
        {
          // first try to find exact match
          for(int j = 0; j < queue.size(); ++j)
          {
            if(queue[j].isIn() && queue[j].isOpposite(current) && queue[j].volume == current.volume)
            {
              string description;
              StringConcatenate(description, (float)queue[j].volume, "[", queue[j].deal, "/", queue[j].order, "-", current.deal, "/", current.order, "] ", (current.profit < 0 ? "-" : ""), current.profit, " ", current.comment);
              createTrend(queue[j].deal, current.deal, queue[j].type, queue[j].time, queue[j].price, current.time, current.price, description);
              current.volume = 0;
              queue >> j; // remove from queue
              ++count;
              break;
            }
          }

          if(!current.isActive()) continue;
          
          // second try to perform partial close
          for(int j = 0; j < queue.size(); ++j)
          {
            if(queue[j].isIn() && queue[j].isOpposite(current))
            {
              string description;
              if(current.volume >= queue[j].volume)
              {
                StringConcatenate(description, (float)queue[j].volume, "[", queue[j].deal, "/", queue[j].order, "-", current.deal, "/", current.order, "] ", (current.profit < 0 ? "-" : ""), current.profit, " ", current.comment);
                createTrend(queue[j].deal, current.deal, queue[j].type, queue[j].time, queue[j].price, current.time, current.price, description);

                current.volume -= queue[j].volume;
                queue[j].volume = 0;
                ++count;
              }
              else
              {
                StringConcatenate(description, (float)current.volume, "[", queue[j].deal, "/", queue[j].order, "-", current.deal, "/", current.order, "] ", (current.profit < 0 ? "-" : ""), current.profit, " ", current.comment);
                createTrend(queue[j].deal, current.deal, queue[j].type, queue[j].time, queue[j].price, current.time, current.price, description);

                queue[j].volume -= current.volume;
                current.volume = 0;
                ++count;
                break;
              }
            }
          }
          
          // purge all inactive from queue
          for(int j = queue.size() - 1; j >= 0; --j)
          {
            if(!queue[j].isActive())
            {
              queue >> j;
            }
          }
        }
        
        if(current.isActive()) // is _still_ active
        {
          if(current.isIn())
          {
            queue << current;
          }
        }
      }
      
      if(!isQueueEmpty())
      {
        Print("Warning: not all deals are processed (probably, open positions left).");
      }
      
      return count;
    }

The algorithm advances through the list of all deals and places market entry deals in the queue. When an exit deal is found, an appropriate opposite deal with the matching size is searched in the queue. If no position with the exactly matching size is found, volumes of entry deals ate consistently selected by FIFO rule, to cover the exit volume. Deals with the fully covered volume are removed from the queue. A trendline (createTrend) is created for each combination of in and out volume.

The FIFO rule is utilized since it is the most reasonable one from an algorithmic point of view. However other options are also possible. A specific trading robot can close deals not only by FIFO, but also by LIFO rule or even in an arbitrary order. The same applies to manual trading. Therefore, in order to establish correspondence between the entry and exit deal in hedging mode, it is necessary to find a certain "workaround", such as an analysis of profits or comments. An example of profit calculation between two price points is available in my blog post, which however does not include accounting for exchange differences. This task is generally not quite simple and is therefore not covered in this article. 

Thus, we have reviewed how the HTML report is processed in the described classes. The implementation of the HistoryProcessor class is much simpler for CSV files. It can be easily understood from the attached source codes. Note that CSV files with the mql5.com signals history have different number of columns for MT4 and MT5. The appropriate format is selected automatically based on the double extension: ".history.csv" for MetaTrader 4 and ".positions.csv" for MetaTrader 5. The only setting for CSV files is the separator character (the default character in the mql5.com signal files is ';').

The major source code of SubChartReporter is inherited from SubChart, using which we checked the display of third-party quotes in subwindows. So let us dwell on new fragments.

Class objects are created and used in event handlers. The processor is created in OnInit and is destructed in OnDeinit:

Processor *processor = NULL;

int OnInit()
{
  if(StringFind(ReportFile, ".htm") > 0)
  {
    processor = new ReportProcessor();
  }
  else if(StringFind(ReportFile, ".csv") > 0)
  {
    processor = new HistoryProcessor();
  }
  string Symbol = SubSymbol;
  if(Symbol == "") Symbol = _Symbol;
  else SymbolSelect(Symbol, true);
  processor.apply(Symbol);
  ...
}

void OnDeinit(const int reason)
{
  if(processor != NULL) delete processor;
}

Graphical object event processing is added to the OnChartEvent handler:

void OnChartEvent(const int id,
                  const long& lparam,
                  const double& dparam,
                  const string& sparam)
{
  if(id == CHARTEVENT_CHART_CHANGE)
  {
    ... // same code
  }
  else
  {
    processor.onChartEvent(id, lparam, dparam, sparam);
  }
}

In the OnCalculate handler, we need to track the situation when the analyzed symbol has changed in response to user actions, after which the full recalculation is performed:

  string Symbol = processor._realsymbol();
  if(Symbol == NULL) Symbol = _Symbol;
  if(lastSymbol != Symbol)
  {
    _prev_calculated = 0;
    lastAvailable = 0;
    initialized = false;
    IndicatorSetInteger(INDICATOR_DIGITS, (int)SymbolInfoInteger(Symbol, SYMBOL_DIGITS));
  }

When the initial drawing is completed and the number of available bars of the working symbol becomes stable, start the timer to load the report data.

  if(lastAvailable == iBars(Symbol, _Period) && lastAvailable != 0)
  {
    if(!initialized)
    {
      Print("Updated ", Symbol, " ", iBars(Symbol, _Period), " bars");
      initialized = true;
      if(ReportFile != "") //
      {                    //
        EventSetTimer(1);  //
      }                    //
    }
    
    return rates_total;
  }

Upload the report in the timer handler (if it has not yet been uploaded) and activate the selected character (it is set from the parameters in OnInit or by a button click in OnChartEvent).

void OnTimer()
{
  EventKillTimer();
  
  if(processor.isEmpty()) // load file only once
  {
    if(processor.attach(ReportFile))
    {
      processor.apply(/*keep already selected symbol*/);
    }
    else
    {
      Print("File loading failed: ", ReportFile);
    }
  }
}

Let us test the indicator performance now. Open the EURUSD chart, attach the indicator and specify the ReportTester-example.html file (attached) in the ReportFile parameter. After initialization, the subwindow will contain the EURUSD chart (because the Symbol parameter is empty) and a number of buttons with the names of all symbol included in the report.

The SubChartReporter indicator without a selected working symbol

Since the report does not have EURUSD, all buttons are gray. Clicking on any button, for example, EURGBP, will load this currency quotes into the subwindow and will display appropriate deals. The button will turn green.

The SubChartReporter indicator with a selected working symbol

In the current implementation, the order of buttons is determined by the symbol occurrence chronology in the report. If necessary, it can be sorted in any desired order, for example, alphabetically or by the number of deals.

By switching the buttons, we can view all symbols from the report. But this is not very convenient. For some reports, it would be better to view all symbols at once, each symbol in its own subwindow. For this purpose, let us write the SubChartsBuilder script, which will create subwindows and launch in these subwindows SubChartReporter instances for different symbols.

The SubChartsBuilder script

The script has the same set of parameters as the SubChartReporter indicator. To provide a single start, parameters are added to the MqlParam array and then IndicatorCreate is called from the MQL API. This is done in one application function createIndicator.

bool createIndicator(const string symbol)
{
  MqlParam params[18] =
  {
    {TYPE_STRING, 0, 0.0, "::Indicators\\SubChartReporter.ex5"},
    
    {TYPE_INT, 0, 0.0, NULL}, // chart settings
    {TYPE_STRING, 0, 0.0, "XYZ"},
    {TYPE_BOOL, 1, 0.0, NULL},
    
    {TYPE_INT, 0, 0.0, NULL}, // common settings
    {TYPE_STRING, 0, 0.0, "HTMLCSV"},
    {TYPE_STRING, 0, 0.0, "PREFIX"},
    {TYPE_STRING, 0, 0.0, "SUFFIX"},
    {TYPE_INT, 0, 0.0, NULL}, // time shift

    {TYPE_INT, 0, 0.0, NULL}, // html settings
    {TYPE_STRING, 0, 0.0, "ROW"},
    {TYPE_STRING, 0, 0.0, "COLUMNS"},
    {TYPE_STRING, 0, 0.0, "SUBST"},
    {TYPE_BOOL, 0, 0.0, NULL},
    {TYPE_BOOL, 0, 0.0, NULL},
    {TYPE_BOOL, 0, 0.0, NULL},

    {TYPE_INT, 0, 0.0, NULL}, // csv settings
    {TYPE_STRING, 0, 0.0, ";"}
  };
  
  params[2].string_value = symbol;
  params[5].string_value = ReportFile;
  params[6].string_value = Prefix;
  params[7].string_value = Suffix;
  params[8].integer_value = TimeShift;
  params[10].string_value = RowSelector;
  params[11].string_value = ColumnSettingsFile;
  params[12].string_value = SubstitutionSettingsFile;
  params[17].string_value = CSVDelimiter;
  
  int handle = IndicatorCreate(_Symbol, _Period, IND_CUSTOM, 18, params);
  if(handle == INVALID_HANDLE)
  {
    Print("Can't create SubChartReporter for ", symbol, ": ", GetLastError());
    return false;
  }
  else
  {
    if(!ChartIndicatorAdd(0, (int)ChartGetInteger(0, CHART_WINDOWS_TOTAL), handle))
    {
      Print("Can't attach SubChartReporter for ", symbol, ": ", GetLastError());
      return false;
    }
  }
  return true;
}

Please note that the indicator is taken from the resource, for which it is pre-registered in the source code, in the following instruction:

#resource "\\Indicators\\SubChartReporter.ex5"

Thus, the script is a fully-featured independent program which does not depend on the presence of the indicator with a particular user.

There is another purpose of this approach. When indicator instances are generated for all the symbols used in the report, control buttons are no longer needed. Unfortunately, MQL does not provide an opportunity to determine whether the indicator is launched by the user or through IndicatorCreate, whether it runs by itself or is an integral (dependent) part of a larger program. When the indicator is placed in a resource, it is possible to show or hide buttons depending on the indicator path: the path in the resource (i.e. the presence of the "::Indicators\\" string) means the display of buttons should be disabled.

To call the createIndicator function for each of the report symbol we need to parse the report in the script.

int OnStart()
{
  IndexMap *data = NULL;
  int columnsCount = 0, symbolColumn = 0;
  
  if(ReportFile == "")
  {
    Print("cleanUpChart");
    return cleanUpChart();
  }
  else if(StringFind(ReportFile, ".htm") > 0)
  {
    data = HTMLConverter::convertReport2Map(ReportFile, true);
    columnsCount = COLUMNS_COUNT;
    symbolColumn = COLUMN_SYMBOL;
  }
  else if(StringFind(ReportFile, ".csv") > 0)
  {
    data = CSVConverter::ReadCSV(ReportFile);
    if(data != NULL && data.getSize() > 0)
    {
      IndexMap *row = data[0];
      columnsCount = row.getSize();
      symbolColumn = CSV_COLUMN_SYMBOL;
    }
  }
  
  if(data != NULL)
  {
    IndexMap symbols;
    
    for(int i = 0; i < data.getSize(); ++i)
    {
      IndexMap *row = data[i];
      if(CheckPointer(row) == POINTER_INVALID || row.getSize() != columnsCount) break;
      
      string s = row[symbolColumn].get<string>();
      StringTrimLeft(s);
      if(StringLen(s) > 0) symbols.set(s);
    }
    
    for(int i = 0; i < symbols.getSize(); ++i)
    {
      createIndicator(symbols.getKey(i));
    }
    delete data;
  }

  return 0;
}

If the script is run with an empty report name, it will remove all sub-windows with indicator instances from the window (if they were previously created). This is done by the cleanUpChart function.

bool cleanUpChart()
{
  bool result = true;
  int n = (int)ChartGetInteger(0, CHART_WINDOWS_TOTAL);
  for(int i = n - 1; i > 0; --i)
  {
    string name = ChartIndicatorName(0, i, 0);
    if(StringFind(name, "SubChartReporter") == 0)
    {
      Print("Deleting ", name);
      result &= ChartIndicatorDelete(0, i, name);
    }
  }
  return result;
}

This is an efficient feature for clearing the chart after completing the report analysis.

To test the script, I downloaded CSV files with signal histories. Here is what it might look like (the main chart is minimized):

Multiple SubChartReporter instances when analyzing multicurrency trading

The generated objects are provided with descriptions which contain details from the report (deal numbers, volumes, profit and comments). To show the details, enable "Show object descriptions" in chart settings.

If the number of working symbols is large, subwindow sizes are decreased. Although this provides an overall picture, studying of details might be difficult. When you need to analyze each deal, use as much space as possible, including the main window. For this purpose, let us create a new version of the SubChartReporter indicator, which will display deals on the main chart instead of using the subwindow. Let us call it MainChartReporter.

The MainChartReporter indicator

Since the indicator is displayed on the price chart, there is no need to calculate buffers and draw anything other than trend objects. In other words, this is a bufferless indicator, which will change the working symbol of the current chart and will set the analyzed symbol. From the point of view of implementation, the indicator is almost ready: the new version code is a significantly simplified SubChartReporter. Here are the code features.

The source code of the three main classes, Processor, ReportProcessor and HistoryProcessor, will be moved to the header file and included in both indicators. The differences which are specific for any of the versions will be used in preprocessor instructions for conditional compilation. The CHART_REPORTER_SUB macro will be defined for the SubChartReporter indicator code, and CHART_REPORTER_MAIN will be defined for MainChartReporter.

Two lines need to be added to MainChartReporter. We need to switch the chart to a new real symbol in the 'apply' method:

#ifdef CHART_REPORTER_MAIN
      ChartSetSymbolPeriod(0, real, _Period);
#endif

We also need to display a comment with the text, which in the first indicator version was equal to its name (INDICATOR_SHORTNAME).

#ifdef CHART_REPORTER_MAIN
      Comment(title);
#endif

In the onChartEvent handler method, we updated the current chart, because the first indicator version drew data on the symbol which differed from the main window symbol. In the new version, the main symbol quotes are used "as is" and this there is no need to update the window. Therefore the relevant line is added into conditional compilation only of the SubChartReporter indicator.

#ifdef CHART_REPORTER_SUB
            ChartSetSymbolPeriod(0, _Symbol, _Period);
#endif

From the MainChartReporter indicator source code, delete the buffers (by specifying their number equal to 0 in order to avoid compiler warning).

#property indicator_chart_window
#property indicator_buffers 0
#property indicator_plots   0

Also delete the group of subwindow settings, which is no longer used:

input GroupSettings Chart_Settings; // S U B C H A R T    S E T T I N G S
input string SubSymbol = ""; // · Symbol
input bool Exact = true; // · Exact

The OnCalculate function becomes empty, but it must be present in the indicator. The timer for receiving report from data is started in OnInit.

void OnTimer()
{
  EventKillTimer();
  
  if(processor.isEmpty()) // load file only once
  {
    if(processor.attach(ReportFile))
    {
      processor.apply();
      datetime start = processor.getStart();
      if(start != 0)
      {
        ChartSetInteger(ChartID(), CHART_AUTOSCROLL, false);
        // FIXME: this does not work as expected
        ChartNavigate(ChartID(), CHART_END, -1 * (iBarShift(_Symbol, _Period, start)));
      }
    }
  }
}

This was an attempt to scroll the chart to the first deal by calling ChartNavigate. Unfortunately, I couldn't make this code part work properly, so the chart was never shifted. The possible solution is to determine the current position and to navigate relative to it using CHART_CURRENT_POS. However this solution does not seem optimal.

This is what the MainChartReporter indicator looks like on the chart.

The MainChartReporter indicator

Attached files

• SubChart.mq5 — the SubChart indicator
• SubChartReporter.mq5 — the SubChartReporter indicator
• MainChartReporter.mq5 — the MainChartReporter indicator
• SubChartsBuilder.mq5 — a script for creating a group of SubChartReporter instances for all symbols used in the report
• ChartReporterCore.mqh — the main common classes for the indicators
• WebDataExtractor.mqh — HTML parser
• CSVReader.mqh — CSV parser
• HTMLcolumns.mqh — determining HTML report columns
• CSVcolumns.mqh — determining CSV report columns
• IndexMap.mqh — auxiliary map class
• RubbArray.mqh — auxiliary rubber array class
• StringUtils.mqh — utility functions for operations with strings
• empty_strings.h — list of empty tags for the HTML parser
• GroupSettings.mqh — group of empty parameters
• Refresh.mqh — request of symbol quotes
• ReportTester-example.html — example of the HTML tester report
• ReportHistoryDeals.cfg.csv — CSS selector settings for selecting table columns using the HTML parser

Conclusion

We have viewed several indicators which visualize quotes and trading deals of multiple symbols. Reports in the HTML format serve as the source of input data. By using a universal parser, it is possible to parse not only standard reports (settings for which are already included in the source code), but also other report types. Also we have implemented support for the CSV format, in which the trading history of mql5.com signals is usually provided. By using the open source, anyone can adjust the application to suit your needs.