ATcl - Tcl interpreter for MT4

Maxim Kuznetsov 2018.01.15 05:44

The holidays were fruitful, and I am pleased to present ATcl - the built-in Tcl interpreter for MT4.

The idea of the project was (and still is) to make the most convenient interface to the interpreter without a direct porting of Tcl C Api functions.

This is what I got (working example - a script that saves a quote to a SQLite database):

//+------------------------------------------------------------------+
//|                                                  atcl_sqlite.mq4 |
//|                                                Maxim A.Kuznetsov |
//|                                                      luxtrade.tk |
//+------------------------------------------------------------------+
#property copyright "Maxim A.Kuznetsov"
#property link      "luxtrade.tk"
#property version   "1.00"
#property strict
#property script_show_inputs
#include "ATcl.mqh"

const string Shema = 
"CREATE TABLE IF NOT EXISTS Bar ("
"   symbol TEXT,"
"   period TEXT,"
"   time INTEGER NOT NULL,"
"   open REAL NOT NULL, "
"   high REAL NOT NULL, "
"   low REAL NOT NULL,  "
"   close REAL NOT NULL,"
"   volume INTEGER NOT NULL, "
"   PRIMARY KEY(symbol,period,time)"
" )";
void OnStart()
{
   ATcl_OnInit(); // включить ATcl
   
   ATcl *tcl=new ATcl; // создать интерпретатор
   if (tcl==NULL || !tcl.Ready()) {
      Alert("Ошибка при создании интерпретатора");
      ATcl_OnDeinit();
      return;
   }
   int ok=false;
   do { 
      Print("Берём встроенный SQLite");
      if (tcl.Eval("package require tdbc::sqlite3")!=TCL_OK) break;
      Print("Считаем путь к базе");
      tcl.Set("dbname",tcl.Obj("bar.db3"));
      tcl.Set("datadir",tcl.Obj(TerminalInfoString(TERMINAL_DATA_PATH)));
      if (tcl.Eval("set fullPath [ file join $datadir MQL4 Files $dbname ]")!=TCL_OK) break;
      PrintFormat("Открываем базу %s",tcl.String(tcl.Get("fullPath")));
      if (tcl.Eval("tdbc::sqlite3::connection create db $fullPath")!=TCL_OK) break;
      Print("Задаём схему");
      if (tcl.Eval(StringFormat("db allrows {%s}",Shema))!=TCL_OK) break;
      Print("Готовим стейтмент");
      if (tcl.Eval("set stmt [ db prepare {REPLACE INTO Bar (symbol,period,time,open,high,low,close,volume) VALUES (:symbol,:period,:t,:o,:h,:l,:c,:v)} ]")!=TCL_OK) break;
      Print("Делаем переменные");
      tcl.Set("symbol",tcl.Obj(Symbol()));
      tcl.Set("period",tcl.Obj(EnumToString((ENUM_TIMEFRAMES)Period())) );
      tcl.Set("time",tcl.Obj(Time));
      tcl.Set("open",tcl.Obj(Open));
      tcl.Set("high",tcl.Obj(High));
      tcl.Set("low",tcl.Obj(Low));
      tcl.Set("close",tcl.Obj(Close));
      tcl.Set("volume",tcl.Obj(Volume));
      tcl.Set("n",tcl.Obj((long)0));
      Print("Запускаем стейтмент по массивам");
      // скрипт как объект, чтобы скомпилялся
      Tcl_Obj f=tcl.Obj("foreach t $time o $open h $high l $low c $close v $volume { $stmt execute ; incr n ; if {$n==100} break }");
      tcl.Ref(f);
      if (tcl.Eval(f)!=TCL_OK) break;
      tcl.Unref(f);
      Print("Удаляем стейтмент");
      tcl.Eval("$stmt close");
      Print("Закрываем базу");
      tcl.Eval("$db close");
      ok=true;
   }while(false);
   if (!ok) {
      PrintFormat("Что-то пошло не так: %s",tcl.StringResult());
   }
   delete tcl;          // удалить интерпретатор
   ATcl_OnDeinit();  // выключить ATcl
}

At this stage it is a dormant alpha or API Preview - evaluation of usability of API. But as you can see the interfaces to the DBMS work, with some accuracy also EventLoop works, that is, you can use asynchronous commands, I/O and timers.

I have attached the archive with the current version, you can try it...

The documentation, which I'm updating as much as I can and the latest versions can be found on the project page http://luxtrade.tk/atcl.

In this thread I'm ready to answer questions about Tcl/Tk itself and the mentioned ATcl library. Ideas, comments, suggestions, critics will be appreciated.

UPDATE: attached source code of script in this topic
UPDATE: updated atcl.zip

Files:

atcl_sqlite.mq4 6 kb

atcl.zip 9 kb

СанСаныч Фоменко 2018.01.15 07:59 #1

Interesting stuff.

I am, as always, noshing on the idea of R. It seems to me that you are the right person to at least discuss the idea with.

The idea is as follows.

Today it is possible to drag an exesh file, which was generated by mcl4/5 compiler, to the terminal chart.

Is it possible to make a development to be able to chain an R script to the chart?

MetaEditor Experts: HFT EA (LIQUIDEX_V1) Is it possible to...(Script/Indicator

Maxim Kuznetsov 2018.01.15 08:14 #2

СанСаныч Фоменко:

Interesting stuff.

I am, as always, noshing on the idea of R. It seems to me that you are the right person to at least discuss the idea with.

The idea is as follows.

Today it is possible to drag an exesh file, which was generated by mcl4/5 compiler, to the terminal chart.

And is it possible to make a development to be able to chain an R script to the chart?

By the way, about R :-) Did you pay your attention to R distribution kit ?

Well, there is a directory library/tcltk, which contains built-in R tcl. When a language/platform lacks facilities, they add tcl, aka "Tool Common Language", to it. R lacked GUI and put tcl/tk in it (just like in Python and Ruby).

I lacked many features in MQL, except for graphs, so I got ATcl.

Making a crowdsourced project Programming Machine learning in trading:

Maxim Kuznetsov 2018.01.15 13:07 #3

ATcl can also do statistics :-) Another small test case - calculating basic statistical values for a dataset. The script takes Close, how many will be given in the input parameter

#property copyright "Maxim A.Kuznetsov"
#property link      "luxtrade.tk"
#property version   "1.00"
#property strict
#property script_show_inputs
//--- input parameters
input int      DEPTH=200;

#include "ATcl.mqh"
const string NAMES[]={
   "mean", "minimum", "maximum", "number of data", "sample standard deviation", "sample variance", "population standard deviation","population variance"
};
void OnStart()
{
   ATcl_OnInit();
   ATcl *tcl=new ATcl;
   if (tcl==NULL || !tcl.Ready()) {
      ATcl_OnDeinit();
      Alert("Ошибка при создании интерпретатора");
   }
   bool ok=false;
   do {
      if (tcl.Eval("package require math::statistics")!=TCL_OK) break;
      tcl.Set("data",tcl.Obj(Close,0,DEPTH));
      if (tcl.Eval("math::statistics::basic-stats $data")!=TCL_OK) break;
      Tcl_Obj stats=tcl.Result();
      tcl.Ref(stats);
      int total=tcl.Count(stats);
      for(int i=0;i<total;i++) {
         PrintFormat("stats %d \"%s\" = %s",i,(i<ArraySize(NAMES)?NAMES[i]:"??"),tcl.String(stats,i));
      }
      tcl.Unref(stats);
      ok=true;
   } while(false);
   if (!ok) {
      PrintFormat("Что-то пошло не так : %s",tcl.StringResult());
   }
   delete tcl;
   ATcl_OnDeinit();
}

Attached is the script. There is only one line of calculation :-)

Files:

atcl_basicstat.mq4 4 kb

Machine learning in trading: How do you algorithmise Any rookie question, so

Maxim Kuznetsov 2018.01.15 15:10 #4

And one more example :-)

Let's take a zip and calculate checksums of files stored in it. We calculate MD5 and SHA256

#property copyright "Maxim A.Kuznetsov"
#property link      "luxtrade.tk"
#property version   "1.00"
#property strict
#property script_show_inputs
//--- input parameters
input string   ZIP="MQL4\\Scripts\\atcl.zip";
#include "ATcl.mqh"
void OnStart()
{
   ATcl_OnInit();
   ATcl *tcl=new ATcl;
   if (tcl==NULL || !tcl.Ready()) {
      ATcl_OnDeinit();
      Alert("Ошибка при создании интерпретатора");
   }
   bool ok=false;
   do {
      if (tcl.Eval("package require vfs::zip")!=TCL_OK) break;
      if (tcl.Eval("package require md5")!=TCL_OK) break;
      if (tcl.Eval("package require sha256")!=TCL_OK) break;
      tcl.Set("archive",tcl.Obj(ZIP));
      if (tcl.Eval("set z [ vfs::zip::Mount $archive $archive ]")!=TCL_OK) break;
      if (tcl.Eval("set list [ glob -directory $archive -nocomplain *.* ]")!=TCL_OK) break;
      Tcl_Obj list=tcl.Result();
      tcl.Ref(list);
      int total=tcl.Count(list);
      // создадим новую процедуру 
      if (tcl.Eval("proc content { name } { set f [ open $name rb ] ; set ret [ read $f ] ; close $f ; set ret }")!=TCL_OK) break;
      for(int i=0;i<total;i++) {
         string fileName=tcl.String(list,i);
         tcl.Set("fileName",tcl.Obj(fileName));    
         PrintFormat("%s",fileName);
         if (tcl.Eval("set content [ content $fileName ]")!=TCL_OK) {
            PrintFormat("Не удалось прочесть файл %s:%s",fileName,tcl.StringResult());
            continue;
         }
         string sha256=tcl.StringEval("sha2::sha256 -hex -- $content");
         PrintFormat("  sha256: %s",sha256);
         string md5=tcl.StringEval("md5::md5 -hex -- $content");
         PrintFormat("  md5: %s",md5);
      }
      tcl.Unref(list);
      tcl.Eval("vfs::zip::Unmount $z $archive");
      ok=true;
   } while(false);
   if (!ok) {
      PrintFormat("Что-то пошло не так : %s",tcl.StringResult());
   }
   delete tcl;
   ATcl_OnDeinit();   
}

Files:

atcl_zipcrc.mq4 5 kb

Maxim Kuznetsov 2018.01.16 20:26 #5

Just for starters, a screenshot of course :-) This is an EA tcp-server in 60 lines of mql + 60 lines of tcl

And MQ4 of the Expert Advisor itself:

#property copyright "Maxim A.Kuznetsov"
#property link      "luxtrade.tk"
#property version   "1.00"
#property strict

//--- input parameters
input ushort   PORT=8000;

#include "ATcl.mqh"
ATcl *tcl=NULL;   // интерпретатор
Tcl_Obj StartServer=0,StopServer=0,SendMsg=0,SymName=0,PortNum=0; // объекты Tcl - имена команд и имя символа
int OnInit()
{
   ATcl_OnInit(); // включить ATcl
   tcl=new ATcl;  // создать интерпретатор
   if (tcl==NULL || !tcl.Ready()) {
      return INIT_FAILED;
   }
   // прочтём исходник c командами
   if (tcl.Eval("source ./MQL4/Experts/atcl_tcpserv.tcl")!=TCL_OK) {
      PrintFormat("Error in Source:%s",tcl.StringResult());
      return INIT_FAILED;
   }
   StartServer=tcl.Ref(tcl.Obj("StartServer")); // команда запуска сервера
   StopServer=tcl.Ref(tcl.Obj("StopServer"));   // остановка сервера
   SendMsg=tcl.Ref(tcl.Obj("SendMsg"));         // рассылка сообщения всем клиентам
   SymName=tcl.Ref(tcl.Obj(Symbol()));          // запомним имя текущего символа
   PortNum=tcl.Ref(tcl.Obj((long)PORT));        // и номер порта
   /// запускаем сервер
   if (tcl.Call(StartServer,PortNum)!=TCL_OK) {
      PrintFormat("Error on StartServer:%s",tcl.StringResult());
      return INIT_FAILED;
   }
   EventSetTimer(2);
   Print("Server started");
   return(INIT_SUCCEEDED);
}
void OnDeinit(const int reason) 
{
   if (tcl!=NULL) {
      if (tcl.Ready()) {
         // остановить сервер
         tcl.Call(StopServer);
      }
      // освободить все объекты
      tcl.Unref(StartServer);
      tcl.Unref(StopServer);
      tcl.Unref(SendMsg);
      tcl.Unref(SymName);
      tcl.Unref(PortNum);
      // удалить интерпретатор
      delete tcl;
   }
   ATcl_OnDeinit(reason);
}
void OnTick()
{
   MqlTick tick;
   tcl.Update();
   if (SymbolInfoTick(Symbol(),tick)!=0) {
      Tcl_Obj message=tcl.Obj();
      tcl.Ref(message);
      tcl.Append(message,SymName);
      tcl.Append(message,tcl.Obj((long)tick.time));
      tcl.Append(message,tcl.Obj((long)tick.time_msc));
      tcl.Append(message,tcl.Obj((double)tick.bid));
      tcl.Append(message,tcl.Obj((double)tick.ask));
      tcl.Append(message,tcl.Obj((long)tick.volume));
      if (tcl.Call(SendMsg,message)!=TCL_OK) {
         PrintFormat("Error in SendMsg:%s",tcl.StringResult());
      }
      tcl.Unref(message);
   }
}
void OnTimer() {
   tcl.Update();
}
void OnChartEvent(const int id,         // идентификатор события   
                  const long& lparam,   // параметр события типа long 
                  const double& dparam, // параметр события типа double 
                  const string& sparam  // параметр события типа string
   )
{
   tcl.Update();
}

So far everything (library and interpreter) works in EAs or scripts. For indicators it is not ready yet :-( There is some kind of weirdness with OnInit/OnDeinit sequences, DLL loading and timers.

Files:

atcl.zip 9 kb

atcl_tcpserv.zip 3 kb

[WARNING CLOSED!] Any newbie Emulation of ticks from Error that the library

Maxim Kuznetsov 2018.01.18 11:32 #6

Everything is becoming more and more stable, EventLoop works confidently in EAs and scripts (and still not in indicators)

New, useful demo - Asynchronous HTTP-client and parser. The source code (mq4 and tcl) is about 60 lines.
Query is executed parallel to Expert Advisor and html parsing is performed correctly, via DOM parser.

input int      EVERY_MINUTS=15;

#include "ATcl.mqh"
ATcl *tcl=NULL;
Tcl_Obj StartClient=0,StopClient=0,report=0,minutes=0;
int OnInit()
{
   ATcl_OnInit(); // включить ATcl
   tcl=new ATcl;  // создать интерпретатор
   if (tcl==NULL || !tcl.Ready()) {
      return INIT_FAILED;
   }
   // прочтём исходник c командами
   if (tcl.Eval("source ./MQL4/Experts/atcl_httpclient.tcl")!=TCL_OK) {
      PrintFormat("Error in Source:%s",tcl.StringResult());
      return INIT_FAILED;
   }
   StartClient=tcl.Ref(tcl.Obj("StartClient"));
   StopClient=tcl.Ref(tcl.Obj("StopClient"));
   report=tcl.Ref(tcl.Obj("report"));
   minutes=tcl.Ref(tcl.Obj((long)EVERY_MINUTS));
   if (tcl.Call(StartClient,minutes)!=TCL_OK) {
      Print("call failed");
      return INIT_FAILED;
   }
   EventSetTimer(1);
   return(INIT_SUCCEEDED);
}
void OnDeinit(const int reason) 
{
   if (tcl!=NULL) {
      if (tcl.Ready()) {
         tcl.Call(StopClient);
      }
      tcl.Unref(StartClient);
      tcl.Unref(StopClient);
      tcl.Unref(report);
      tcl.Unref(minutes);
      delete tcl;
   }
   ATcl_OnDeinit(reason);
}
void OnTimer()
{
   tcl.Update();
   if (tcl.IsSet(report)) {
      string comment=tcl.String(tcl.Get(report));
      tcl.Unset(report);
      Comment(comment);
   }   
}

void OnTick() {
   tcl.Update();
}

I am almost satisfied. It won't take long before I will be able to remove from my MQL fakes everything that is not directly related to trading and add normal communications with the outside world (databases, emails, reports and even GUI).

If only not for the current problem with indicators and timers :-(.

Files:

atcl_httpclient.zip 20 kb

Asynchronous and multi-threaded programming Questions from Beginners MQL5 Any questions from newcomers

Maxim Kuznetsov 2018.01.18 11:35 #7

You can't attach more than one zip file here ?
or is there a problem with the files of the same name...

The current atcl version is still NOT attached :-) The site is glitchy or something else... for now you can get it from the project page.
When the glitches pass, I will attach it here

Indicator Forex Trading Signal Errors, bugs, questions Mashki and I. Captured

Alexey Oreshkin 2018.01.18 12:12 #8

Maxim Kuznetsov:
In a bit longer, you could remove everything from your MQL creations that is not directly related to trading and add normal communications with the outside world (databases, emails, reports and even a GUI).

And what prevents you from going to the outside world via piping?
The result is the same: you can practically forget about mql - write a server once and that's it.

Maxim Kuznetsov 2018.01.18 12:20 #9

Alexey Oreshkin:

What prevents you from going to the outside world via pipelines?
the result is the same: you can practically forget about µl - write a server once and that's it.

but who will control the pips, implement the protocol and parse the results? it's long, slow and only by polling.

and there's a completely different level of integration - data can easily be transferred between tcl and mql - they sit right in one memory, in a single process. In theory (didn't check the work, but it should work) and with some accuracy you can link variables.
It turns out that all the features and libraries that are in tcl but lacking in mql are easily available.

Questions on OOP in The market is a [WARNING CLOSED!] Any newbie

Maxim Kuznetsov 2018.01.18 17:37 #10

INDICATORS are defeated (almost) :-) That is, ATcl now works in indicators, including the ill-fated EventLoop.

On screenshot - the same program as last time (asynchronous http client), but now in indicator:

Of course, the algorithm of initialization of DLL is not the same :-)

We can say that the API-Preview is almost completed, just clean up the source code and documentation, and then we can announce the beta-version

Attach turkey and library. And I'm going to celebrate with beer :-)

Files:

atcl_httpclient_ind.zip 19 kb

atcl.zip 10 kb

Any questions from newcomers Developers! Do you even Indicators

1 2 3 4 5 6 7 8 ...10

New comment