MT5 Ticks HTTP Provider
- Utilities
- Anton Sychov
- Version: 1.2
- Updated: 11 November 2023
- Activations: 20
MT5 Broker Ticks HTTP Provider
Description
EA turns your MT5 terminal into historical/realtime ticks data provider for your application.
There are many market data providers on the internet, but in practice, the data provided is not always of good quality. Moreover, these services are often more expensive and typically require monthly subscription fees per each symbol.
With this EA, you can feed your application with exactly the same tick data that you see in the MT5 terminal, the same data on which you base your trades and market analysis.
Together with the EA that provides rates data (OHLC, candles), a complete data set necessary for market analysis is offered.
Capabilities
- Enables the transmission of tick data to a pre-configured external HTTP URL.
- Guarantees the delivery of every tick, with the capability to resume from the last sent tick in case of disruptions.
- Offers two storage options for tick timestamp offsets:
- Local Files (default, managed automatically)
- HTTP endpoint (for integration with applications)
- Ensures reliable ticks delivery, with retry mechanisms for HTTP request failures.
- Allows for data transfer with configurable batching.
Configuration
| Input | Description | Default |
|---|---|---|
| dataProvider | Specifies the name of data provider. This name can then be used in URL templates as {dataProvider}. | - |
| tickQueryingIntervalDurationSeconds | Interval for querying ticks from a broker, in seconds. | 10000 seconds |
| maxTicksPerBatch | Maximum number of ticks that can be sent in a single batch. | 10000 ticks |
| debugLogsEnabled | Whether to enable debug logs. | true |
| dataSendingIntervalMilliseconds | Period for querying ticks and sending them to an external URL. | 10000 milliseconds |
| targetExportUrlTemplate | URL template for endpoint where the tick data will be sent. Supported placeholders: dataProvider. | - |
| shouldReExportHistoricalTicks | Option to re-export ticks starting from a specific value, as specified in the startExportFromTimestampMillis input. | false |
| startExportFromTimestampMillis | If shouldReExportHistoricalTicks is set to true, ticks will be sent starting from the timestamp specified in milliseconds. | 1672531200000 (2023-01-01 0:00:00) |
| exportedSymbolsSource | Source for the symbols for which ticks will be exported. Options are: 'HTTP_ENDPOINT' or 'EA_INPUT'. | EA_INPUT |
| symbols | In cases where exportedSymbolsSource is set to 'EA_INPUT', this setting specifies which symbols ticks will be exported. | EURUSD |
| exportedSymbolsFetchUrlTemplate | URL template to fetch the ticks symbols. Used in case exportedSymbolsSource='HTTP_ENDPOINT'. Supported variables: dataProvider. | - |
| lastSentTickTimestampSource | Source for the timestamp of the last sent tick. Options are: 'HTTP_ENDPOINT' or 'FILES'. | FILES |
| lastSentTickTimestampUrlTemplate | URL template to fetch the timestamp of the last sent tick. Used in case exportedSymbolsSource='HTTP_ENDPOINT'. Supported placeholders: dataProvider, symbol | - |
EA Workflow Description:
Upon initialization, the Expert Advisor (EA) performs the following operations:
-
Reset Offset Check: The EA checks the shouldReExportHistoricalTicks variable to determine if there's a need to export historical ticks data from a specific timestamp. If a re-export is required, it:
- Establishes the startExportFromTimestampMillis value as the new timestamp offset.
- Saves this offset securely, either locally in a file or transmits it to a designated HTTP endpoint, based on the configuration set by lastSentTickTimestampSource.
-
Periodic Ticks Data Harvesting and Sending: The EA is programmed with a timer set to activate at intervals defined by dataSendingIntervalMilliseconds. Upon each activation, the EA:
- Requests ticks data from the broker, starting from the last stored offset (the most recent tick timestamp) to the present moment.
- Converts the acquired ticks data into json and sends it to the predefined URL derived from targetExportUrlTemplate input.
By executing these steps, the EA ensures a continuous and automated stream of the most recent tick data for external use, keeping your application or analysis tools supplied with up-to-the-minute market information.
Usage
Below are the minimum actions required for the EA to start exporting ticks to your configured URL
-
Implement an HTTP endpoint that handles a POST request on a URL derived from the targetExportUrlTemplate. This endpoint should:
- Accept tick data with the JSON structure described below.
- Respond with a 200 status code if the reception is successful.
-
Ensure that you add the host to the list of allowed hosts in 'Tools > Options > Allow WebRequest for listed URL'.
- If you are testing locally on localhost, create a hostname in 'C:\Windows\System32\drivers\etc\hosts' on Windows or '/etc/hosts' on Linux-based systems. Use this hostname in the targetExportUrlTemplate.
-
Attach the EA to any chart in MT5 and configure the following inputs:
- startExportFromTimestampMillis: Set the timestamp from which you need to get ticks.
- symbols: Configure the symbols that you need ticks for.
- shouldReExportHistoricalTicks: This must be set to true on the first run so that the EA creates all necessary files for tracking the last sent tick timestamp.
Offset Management
The EA maintains the timestamp (offset) of the last successfully sent tick in order to continue the export process in the event of disruptions or a terminal restart.
By default, timestamps are stored in files that are automatically created at the following path: C:\Users{userName}\AppData\Roaming\MetaQuotes\Terminal{id}\MQL5\Files. There is a separate file for each symbol.
Additionally, there is an option to use separate HTTP endpoints for reading/storing the offset instead of files.
To use this feature, set lastSentTickTimestampSource to 'HTTP_ENDPOINT' and implement HTTP endpoints based on the URL defined in lastSentTickTimestampUrlTemplate.
As a backend solution, you can choose to store offsets in a database (such as PostgreSQL). If you need to re-export ticks for a specific symbol, you would only need to update the timestamp values in the database table.
Exported Symbols Management
By default, the EA uses the input symbols to determine which symbols ticks need to be exported. However, there is also an option to retrieve the list of symbols from a separate HTTP endpoint.
To utilize this feature, set exportedSymbolsSource to 'HTTP_ENDPOINT' and implement an endpoint using the URL defined in exportedSymbolsFetchUrlTemplate.
EA External Management
If you set both lastSentTickTimestampSource and exportedSymbolsSource to 'HTTP_ENDPOINT', then the EA can be fully controlled externally:
- You can initiate re-export for specific symbols without needing to perform any actions in MT5.
- You can specify which symbols to export without having to change the inputs in MT5.
REST API Specification for Application Integration
| HTTP Endpoint | Description | Method | Request | Response |
|---|---|---|---|---|
| targetExportUrlTemplate | URL for sending ticks data. | POST | Headers: Content-Type: application/json Body [
{
"symbol": "EURUSD",
"bid": 1.1800,
"ask": 1.1802,
"last": 1.1801,
"volume": 1234.56,
"timestampMillis": 1625135912000
},
...
] | Response Code 200 in case ticks are obtained successfully |
| exportedSymbolsFetchUrlTemplate | URL is used to retrieve the list of ticks symbols that the EA will export to the URL derived from 'targetExportUrlTemplate'. | GET | Headers: Content-Type: text/plain Body (coma separated list of symbols) EURUSD,GBPUSD,AUDUSD,BTCUSD | |
| lastSentTickTimestampUrlTemplate | URL is utilized to fetch the specific timestamp from which the export process should begin. | GET | Headers Content-Type: text/plain Body 1625135912000 | |
| lastSentTickTimestampUrlTemplate | URL is used to store the timestamp of the last successful POST request made to the URL derived from 'targetExportUrlTemplate'. | POST | Headers: Content-Type: text/plain Body 1625135912000 |
IMPORTANT: Before purchasing this EA, please verify that your MT5 broker supports exporting historical tick data. Availability and depth of tick data vary across brokers. In our tests, using Roboforex provided access to all necessary tick data going several years back.
Demo Version (NZDUSD only)
Tags: ticks, tick, quotes, quote, stream, streaming, export, exporting, webhook, webhooks, integration, mt5, http, rest, forex, crypto, data, historical, realtime, rest api, provider, broker, data feed
User didn't leave any comment to the rating