Discussion of article "SQLite: Native handling of SQL databases in MQL5" - page 2

New comment
 

Renat Fatkhullin:

Within different MQL5 Expert Advisors launched in one terminal, you can share the same database.
.

I have tested the work of two Expert Advisors with the same database in one terminal. It works without errors - both Expert Advisors write data to one file by a timer of 2 seconds.

The files are attached, they differ only in the name of the Expert Advisor.

2020.02.19 17:56:14.624 SecondExpertDataBase (GBPUSD,H1)        16:56:14: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:16.627 SecondExpertDataBase (GBPUSD,H1)        16:56:16: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:18.616 SecondExpertDataBase (GBPUSD,H1)        16:56:18: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:20.630 SecondExpertDataBase (GBPUSD,H1)        16:56:20: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:22.078 FirstExpertDataBase (USDCHF,H1)         16:56:21: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:22.629 SecondExpertDataBase (GBPUSD,H1)        16:56:22: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:24.071 FirstExpertDataBase (USDCHF,H1)         16:56:24: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:24.619 SecondExpertDataBase (GBPUSD,H1)        16:56:24: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:26.070 FirstExpertDataBase (USDCHF,H1)         16:56:26: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:26.619 SecondExpertDataBase (GBPUSD,H1)        16:56:26: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:28.079 FirstExpertDataBase (USDCHF,H1)         16:56:27: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:28.625 SecondExpertDataBase (GBPUSD,H1)        16:56:28: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:30.079 FirstExpertDataBase (USDCHF,H1)         16:56:30: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:30.617 SecondExpertDataBase (GBPUSD,H1)        16:56:30: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:32.069 FirstExpertDataBase (USDCHF,H1)         16:56:32: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:32.627 SecondExpertDataBase (GBPUSD,H1)        16:56:32: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:34.073 FirstExpertDataBase (USDCHF,H1)         16:56:33: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:34.627 SecondExpertDataBase (GBPUSD,H1)        16:56:34: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:36.616 SecondExpertDataBase (GBPUSD,H1)        16:56:36: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:38.079 FirstExpertDataBase (USDCHF,H1)         16:56:38: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:38.615 SecondExpertDataBase (GBPUSD,H1)        16:56:38: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:40.072 FirstExpertDataBase (USDCHF,H1)         16:56:40: added 1 record on USDCHF to timer.sqlite
2020.02.19 17:56:40.618 SecondExpertDataBase (GBPUSD,H1)        16:56:40: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:42.630 SecondExpertDataBase (GBPUSD,H1)        16:56:42: added 1 record on GBPUSD to timer.sqlite
2020.02.19 17:56:44.617 SecondExpertDataBase (GBPUSD,H1)        16:56:44: added 1 record on GBPUSD to timer.sqlite


Here is how the database looks like in MetaEditor


 
Chiripaha:

Is what is described for µl5 possible to use in µl4?

In MQL4 you can work with native Sqlite3_32.dll through the wrapper for MQL

/*
 * SQLite interface for MT4
 */

#import "sqlite3_wrapper.dll"
int sqlite_exec (string db_fname, string sql);
int sqlite_table_exists (string db_fname, string table);
int sqlite_query (string db_fname, string sql, int& cols[]);
int sqlite_next_row (int handle);
string sqlite_get_col (int handle, int col);
int sqlite_free_query (int handle);
#import

This is a wrapper for MQL, do not confuse it with SQLite3Wrapper.dll

But whether it is still online, I don't know, I used to use it before.

UPD: https: //github.com/Shmuma/sqlite3-mt4-wrapper
Shmuma/sqlite3-mt4-wrapper
Shmuma/sqlite3-mt4-wrapper
  • Shmuma
  • github.com
Database file is by default stored to . If you specify a full path as database filename, it's used. Terminal data path TERMINAL_DATA_PATH can be known by the following instruction. Open MT4 Open [File] menu Click "Open Data Folder" Sample Many sample scripts in under . Precautions Argument mess MT4 build 610 has a weird bug when dll function...
 
fxsaber:

Who has figured it out, please show the implementation of such a task.

  1. There are two Terminals.
  2. It is necessary to transfer real-time quotes of one symbol from Terminal1 to the corresponding custom symbol of Terminal2.

There is another nuance - saving in the database consistent data with growing value, not going scattered, such as quotes (with a growing date) - a VERY expensive operation. The tree is not balanced, one branch grows indefinitely, and the engine rebalances the tree regularly (this is worse than rubbish collection). Maybe this only applies to the indexed table, I don't remember anymore. Or maybe in the few years I haven't used it, it's already been solved. I'll have to try it out. Maybe even the developers will analyse it, they should be interested in it.

 

DatabaseImport()

simple script

void OnStart()
  {
  string filename="DOMdb.db";
  int database=DatabaseOpen(filename, DATABASE_OPEN_READWRITE  );
  DatabaseImport(database,             // database handle received in DatabaseOpen
                     "DOMtb",             // table name for data insertion
                     "AllDOM.csv",          // file name for importing data
                       0,             // flag combination
                      ";",         // data separator 
                        0,         // how many first lines to skip 
                      "//"     // a string of characters that define comments
   );
   
  }

2020.02.20 15:09:27.171 MQL5 'DOMcopy.ex5' has newer unsupported version, please update your client terminal

version 5 build 2323 latest meta-editor is the same and debugging does not work.

is something unfinished?

 

TheDatabaseImport function has not been released to the public yet.

Please wait for the release.

 
Maybe not in the topic, but: I'm wondering why the developers haven't implemented the connection to MS SQL Server yet?
 
Rashid Umarov:

The DatabaseImport function has not yet been released to the public.

Please wait for the release.

version 5 build 2326 same thing, so wait for a stable version when?

 
IuriiPrugov:

version 5 build 2326 the same thing that is to wait for a stable version and when?

Forum on trading, automated trading systems and testing trading strategies.

New version of MetaTrader 5 build 2340: Managing account settings in the tester and expanding integration with Python

MetaQuotes, 2020.02.20 14:55

On Friday 21 February 2020, an updated version of the MetaTrader 5 platform will be released. The update contains the following changes:

  1. MetaEditor: Added the ability to conveniently work with SQLite databases.

    In the previous platform update, we added support for working with SQLite databases directly from MQL5. Now the main functions are available through the MetaEditor user interface:

    .
    • Creating and connecting to databases
    • Viewing tables and quick data query
    • Making and executing SQL queries, rolling back changes

    How it works
    To quickly create databases, use the "MQL5 Wizard". Here you can immediately create the first table and define a list of its fields.




    Having created a database, you will go to a new section of the "Navigator". All the work with data is performed from this section.

    In the left part, the database tables are displayed. To quickly query the first 1,000 records, double-click on the table name. Here you can also create and open other databases and work with tables.

    The main part of the editor is where you work with the database: fill the table with data, make searches and selections, enter SQL queries, etc.
    .



    For more details on working with databases in MetaTrader 5, please read the article"SQLite: Native Work with SQL Databases in MQL5".

    .
  2. MetaEditor: We continue to add support for multi-language projects. In this version we have extended the functions for working with Python scripts:

    .
    • Now you can conveniently create them through the "MQL5 Wizard", immediately including dependencies on necessary libraries in the code.
    • Special icons for them have been added in the navigator and syntax highlighting in the editor.
    • When running the script through MetaEditor, messages from the Python console (stdout, stderr) are displayed in the "Errors" section.




    To run the script in the editor, click "Compile":
    .





    To work with Python, don't forget to specify the path to it in the "Settings / Compilers" section in MetaEditor. And to work with MetaTrader 5 library, install it with the command:
    .
    pip install MetaTrader 5
    Read more about integration with Python in the documentation.

    .
  3. MQL5: Completely redesigned integration with Python. Many new features have been added and command naming has been changed.

    New command naming
    Existing commands have been renamed:
    MT5Initialize       -> initialize
    MT5Shutdown         -> shutdown
    MT5TerminalInfo     -> terminal_info
    MT5Version          -> version
    MT5CopyRatesFrom    -> copy_rates_from
    MT5CopyRatesFromPos -> copy_rates_from_pos
    MT5CopyRatesRange   -> copy_rates_range
    MT5CopyTicksFrom    -> copy_ticks_from
    MT5CopyTicksRange   -> copy_tick_range

    New commands
    The list of supported commands has been greatly expanded. Added functions of trading and working with trading history, getting information about financial instruments and current account.

    • account_info() - getting information about the current account. An analogue of AccountInfoInteger, AcountInfoDouble and AccountInfoString.
    • positions_total() - receiving the number of open positions. An analogue of PositionsTotal.
    • positions_get(symbol, ticket) - get open positions by symbol or ticket.
    • orders_total() - getting the number of orders. An analogue of OrdersTotal.
    • orders_get(symbol, ticket) - getting open orders by symbol or ticket.
    • history_orders_total(from, to) - getting the number of orders in the history in a specified time interval.
    • history_orders_get(from, to, position, ticket) - getting orders from history in a specified time interval by ticket or with filtering by position.
    • history_deals_total() - getting the number of deals in history. An analogue of HistoryDealsTotal.
    • history_deals_get(from, to, position, ticket) - getting deals from history in a specified interval by ticket or with filtering by position.
    • symbol_info(symbol) - getting information about a financial instrument. An analogue of SymbolInfoInteger, SymbolInfoDouble, SymbolInfoString.
    • symbol_info_tick(symbol) - getting the last tick by symbol. An analogue of SymbolInfoTick.
    • symbol_select(symbol, enable) - enable/disable symbol in the "Market Watch". Analogue of SymbolSelect.
    • order_check(request) - check margin for an order. An analogue of OrderCheck.
    • order_send(request) - sending an order to the server. An analogue of OrderSend.
    • order_calc_margin(action, symbol, volume, price) - margin calculation for the order. An analogue of OrderCalcMargin.
    • order_calc_profit(action, symbol, volume, price_open, price_close) - profit calculation. An analogue of OrderCalcProfit.
      .

    Running Python scripts on charts
    Now Python scripts can be run directly on charts in the platform, similar to regular MQL5 programs. They will be displayed with special icons in the "Navigator".
    .



    Script messages will be displayed in the "Tools / Experts" section. If the script uses MetaTrader 5 library, it will be able to receive information about the instrument, account and trade.

    Python scripts can be run on the same chart in parallel with other MQL5 scripts and Expert Advisors. To stop a script if its execution is looped, simply remove it from the chart.

    Additional protection
    To further protect your accounts when using third-party Python libraries, the "Disable automatic trading via external Python API" option has been added to the terminal settings.
    .




    Python scripts will be allowed to trade only if this option is explicitly disabled.

    .
  4. MQL5: The re-launch of MQL5 programs and re-creation of custom indicators from MQL5 programs has been significantly accelerated. In some cases, the speed has increased hundreds of times.
    .
  5. MQL5: Added functions for working with databases:

    DatabaseImport
    Imports data from a file into a table.
    long  DatabaseImport(
   int           database,          // database handle received in DatabaseOpen
   const string  table,             // table name for data insertion
   const string  filename,          // file name for importing data
   uint          flags,             // flag combination
   const string  separator,         // data separator 
   ulong         skip_rows,         // how many first lines to skip 
   const string  skip_comments      // a string of characters that define comments
   );

    DatabaseExport
    Exports a table or the result of SQL query execution to a CSV file. The file is created in UTF-8 encoding.
    long  DatabaseExport( 
   int           database,           // database handle received in DatabaseOpen 
   const string  table_or_sql,       // table name or SQL query 
   const string  filename,           // CSV file name for data export 
   uint          flags,              // flag combination 
   const string  separator           // data separator in CSV file 
   );

    DatabasePrint
    Prints a table or the result of an SQL query execution to the Expert Advisor log. 
    long  DatabasePrint(
   int           database,          // database handle received in DatabaseOpen
   const string  table_or_sql,      // table or SQL query
   uint          flags              // flag combination
   );

  6. MQL5: Added FileSelectDialog function that calls the system dialogue for creating/opening a file or folder. 
    int  FileSelectDialog(
   string   caption,              // window title
   string   initial_dir,          // start folder
   string   filter,               // extension filter
   uint     flags,                // flag combination
   string&  filenames[],          // array with file names
   string   default_filename      // default filename
   );
    The new function will allow to implement convenient user interaction with MQL5 program.

    .
  7. MQL5: DEAL_FEE - payment for conducting a transaction - has been added to the ENUM_DEAL_PROPERTY_DOUBLE enumeration. In fact, this is a separate type of commission that can be charged by a broker.

    .
  8. Tester: Added the ability to set your own trading account settings when testing strategies - trading limits, margin settings and commissions. Thus, you get even more possibilities to simulate different trading conditions of brokers.
    .




    General settings
    In this section you can set the maximum number of open orders and positions you can have on your account at the same time. Also here you can set up the sessions when the tested program will not be allowed to trade.


    Margin
    Here you can fully control how the margin will be reserved and what position accounting system will be used during testing:

    .
    • Risk Management - risk management model: OTC and exchange-traded, with netting and hedging. Detailed information about them is available in the Help.
      .
    • "Margin call" level - the level of funds on the account, when reached, it switches to the Margin call state.
    • Stop out" level - the level of funds at which the account is forced to withdraw orders and close trading positions. Both levels can be specified in per cent and in money. In the first case, the levels are defined as the value of the "Funds" indicator on the account. If the "Percentage" option is selected, the levels are defined as the value of the "Margin Level" indicator on the account (Funds/Margin*100).
    • Unrealised - this field specifies how the current unrealised profit/loss will be accounted for in the free margin:
      .
      • Do not use unrealised profit/loss - do not take open positions into account in the calculation.
      • Use unrealised profit/loss - use loss and profit on open positions in the calculation.
      • Use unrealised profit - use only profit.
      • Use unrealised loss - use only loss.
    • Daily fixed - this field specifies how the profit/loss fixed by the trader during the trading day will be taken into account in the free margin:
      .
      • Use daily fixed profit/loss - take into account the profit and loss fixed during the trading day in the free margin.
      • Use daily fixed loss - consider only the loss fixed during the trading day in free margin. During the day the accumulated profit is fixed in a separate account field ("Locked"). At the end of the trading day, the accumulated profit is released (zeroed) and reflected on the account balance (accounted in free margin).
    • Release accumulated profit at the end of the day - this option is available only if the "Use daily fixed loss" option is enabled. If it is enabled, then at the end of the trading day the profit accumulated during the day will be released and recorded on the balance (and accordingly accounted in the free margin). Otherwise - it will not be.


    Commission
    In this section you have full control over how commission is charged on all trades.

    .
    • Commissions can be single or multi-level, i.e. they can be charged in the same amount regardless of the transaction volume/turnover or vary depending on their size. The relevant information is displayed in the specification.
    • Commissions may be charged immediately upon execution of a transaction or at the end of a trading day/month.
    • Commissions can be charged depending on the direction of the transaction: for entry, for exit or for both types of transactions.
    • Commissions may be charged per lot or per trade.
    • Commissions can be charged in different amounts: in money, per cent or pips.



  9. Tester: Optimised and significantly accelerated work in the "Market Scan" mode, when several test passes are made on all symbols from the "Market Watch".
    .
  10. Tester: When calculating profit in pips, the size of a trade or position is now taken into account. Previously, the calculation was made without taking into account the volume - as for one lot.
  11. Tester: Optimisation results chart management has been improved. When zooming in on a regular optimisation chart, it is now possible to scroll it. A double click on a point of the chart now highlights the corresponding result in the table of passes.
    .
  12. MetaEditor: Added display of SQLite database files (*.db;*.sdb;*.sqlite;*.db3;*.s3db;*.sqlite3) in the "Navigator".
  13. MetaEditor: Fixed errors when saving project properties.
    .
  14. Updated documentation.

The update will be available through LiveUpdate system.


 

stable version 5 build 2340 the same DatabaseImport shows the same DatabaseImport on execution

2020.02.22 14:01:42.338 MQL5 'DOMcopy.ex5' has newer unsupported version, please update your client terminal

 

Hello!

I tried to create a connection to a database in the indicator, but for some reason it didn't work.

Why is it not written anywhere, in which programmes can I use databases?

123456789...12
New comment