MT5 Rates HTTP Provider
- Utilities
- Anton Sychov
- Version: 1.2
- Updated: 11 November 2023
- Activations: 20
MT5 Broker Rates (OHLC, candles) HTTP Provider
Description
EA turns your MT5 terminal into historical/realtime rates data provider for your application.
There are many market data providers on the internet, but in practice, the data provided is not always of good quality. Moreover, these services are often more expensive and typically require monthly subscription fees per each symbol.
With this EA, you can feed your application with exactly the same rates data that you see in the MT5 terminal, the same data on which you base your trades and market analysis.
Together with the EA that provides ticks data, a complete data set necessary for market analysis is offered.
Capabilities
- Enables the transmission of rates data to a pre-configured external HTTP URL.
- Guarantees the delivery of every rate, with the capability to resume from the last sent rate in case of disruptions.
- Offers two storage options for rate timestamp offsets:
- Local Files (default, managed automatically)
- HTTP endpoint (for integration with applications)
- Ensures reliable rates delivery, with retry mechanisms for HTTP request failures.
- Allows for data transfer with configurable batching.
Configuration
| Input | Description | Default |
|---|---|---|
| dataProvider | Specifies the name of data provider. This name can then be used in URL templates as {dataProvider}. | - |
| maxRatesPerBatch | Maximum number of rates that can be semt in a single batch. | 10000 rates |
| debugLogsEnabled | Whether to enable debug logs. | true |
| targetExportUrlTemplate | URL template for endpoint where the rates data will be sent. Supported placeholders: dataProvider. | - |
| shouldReExportHistoricalRates | Option to re-export rates starting from a specific value, as specified in the startExportFromTimestampMillis input. | false |
| startExportFromTimestampMillis | If shouldReExportHistoricalRates is set to true, rates will be sent starting from the timestamp specified in milliseconds. | 1672531200000 (2023-01-01 0:00:00) |
| exportedSymbolsSource | Source for the symbols for which rates will be exported. Options are: 'HTTP_ENDPOINT' or 'EA_INPUT'. | EA_INPUT |
| symbols | In cases where exportedSymbolsSource is set to 'EA_INPUT', this setting specifies which symbols rates will be exported. | EURUSD |
| timeframes | Specifies the timeframes for the rates to be exported when exportedSymbolsSource is set to 'EA_INPUT'. | M1,M5,M15,M30,H1,H4,D1,W1 |
| exportedSymbolsFetchUrlTemplate | URL template to fetch the rates symbols. Used in case exportedSymbolsSource='HTTP_ENDPOINT'. Supported variables: dataProvider. | - |
| lastSentRateTimestampSource | Source for the timestamp of the last sent rate. Options are: 'HTTP_ENDPOINT' or 'FILES'. | FILES |
| lastSentRateTimestampUrlTemplate | URL template to fetch the timestamp of the last sent rate. Used in case exportedSymbolsSource='HTTP_ENDPOINT'. Supported placeholders: dataProvider, symbol, timeframe. | - |
EA Workflow Description:
Upon initialization, the Expert Advisor (EA) performs the following operations:
-
Reset Offset Check: The EA checks the shouldReExportHistoricalRates variable to determine if there's a need to export historical rates data from a specific timestamp. If a re-export is required, it:
- Establishes the startExportFromTimestampMillis value as the new timestamp offset.
- Saves this offset securely, either locally in a file or transmits it to a designated HTTP endpoint, based on the configuration set by lastSentRateTimestampSource.
-
Periodic Rates Data Harvesting and Sending: The EA is programmed with a timer set to activate at intervals defined by dataSendingIntervalMilliseconds. Upon each activation, the EA:
- Requests rates data from the broker, starting from the last stored offset (the most recent rate timestamp) to the present moment.
- Converts the acquired rates data into json and sends it to the predefined URL derived from targetExportUrlTemplate input."
By executing these steps, the EA ensures a continuous and automated stream of the most recent rates data for external use, keeping your application or analysis tools supplied with up-to-the-minute market information.
Usage
Below are the minimum actions required for the EA to start exporting rates to your configured URL
-
Implement an HTTP endpoint that handles a POST request on a URL derived from the targetExportUrlTemplate. This endpoint should:
- Accept rates data with the JSON structure described below.
- Respond with a 200 status code if the reception is successful.
-
Ensure that you add the host to the list of allowed hosts in 'Tools > Options > Allow WebRequest for listed URL'.
- If you are testing locally on localhost, create a hostname in 'C:\Windows\System32\drivers\etc\hosts' on Windows or '/etc/hosts' on Linux-based systems. Use this hostname in the targetExportUrlTemplate.
-
Attach the EA to any chart in MT5 and configure the following inputs:
- startExportFromTimestampMillis: Set the timestamp from which you need to get rates.
- symbols: Configure the symbols that you need rates for.
- shouldReExportHistoricalRates: This must be set to true on the first run so that the EA creates all necessary files for tracking the last sent rate timestamp.
Offset Management
The EA maintains the timestamp (offset) of the last successfully sent rate in order to continue the export process in the event of disruptions or a terminal restart.
By default, timestamps are stored in files that are automatically created at the following path: C:\Users{userName}\AppData\Roaming\MetaQuotes\Terminal{id}\MQL5\Files. There is a separate file for each symbol and timeframe.
Additionally, there is an option to use separate HTTP endpoints for reading/storing the offset instead of files.
To use this feature, set lastSentRateTimestampSource to 'HTTP_ENDPOINT' and implement HTTP endpoints based on the URL defined in lastSentRateTimestampUrlTemplate.
As a backend solution, you can choose to store offsets in a database (such as PostgreSQL). If you need to re-export rates for a specific symbol, you would only need to update the timestamp values in the database table.
Exported Symbols Management
By default, the EA uses the input symbols to determine which symbols rates need to be exported.However, there is also an option to retrieve the list of symbols from a separate HTTP endpoint.
To utilize this feature, set exportedSymbolsSource to 'HTTP_ENDPOINT' and implement an endpoint using the URL defined in exportedSymbolsFetchUrlTemplate.
EA External Management
If you set both lastSentRateTimestampSource and exportedSymbolsSource to 'HTTP_ENDPOINT', then the EA can be fully controlled externally:
- You can initiate re-export for specific symbols without needing to perform any actions in MT5.
- You can specify which symbols to export without having to change the inputs in MT5.
Integrating application REST api specification
| HTTP Endpoint | Description | Method | Request | Response |
|---|---|---|---|---|
| targetExportUrlTemplate | URL for sending rates data. | POST | Headers: Content-Type: application/json Body [ { "symbol": "EURUSD", "timeframe": "H1", "timestampMillis": 1670001234567, "open": 1.12345, "high": 1.12500, "low": 1.12200, "close": 1.12400, "tickVolume": 123456, "spread": 0.0002, "realVolume": 1000000 }, ... ] | Response Code 200 in case rates are obtained successfully |
| exportedSymbolsFetchUrlTemplate | URL is used to retrieve the list of rates symbols that the EA will export to the URL derived from 'targetExportUrlTemplate'. | GET | Headers: Content-Type: text/plain Body (coma separated list of symbols) EURUSD,GBPUSD,AUDUSD,BTCUSD | |
| lastSentRatesTimestampUrlTemplate | URL is utilized to fetch the specific timestamp from which the export process should begin. | GET | Headers Content-Type: text/plain Body 1625135912000 | |
| lastSentRateTimestampUrlTemplate | URL is used to store the timestamp of the last successful POST request made to the URL derived from 'targetExportUrlTemplate'. | POST | Headers: Content-Type: text/plain Body 1625135912000 |
Demo version (NZDUSD only)
Tags: rates, rate, price, price aggregate, stream, streaming, export, exporting, webhook, webhooks, integration, mt5, http, rest, forex, crypto, data, historical, realtime, rest api, provider, broker, data feed, ohlc
User didn't leave any comment to the rating