What should be added for additional support of universal mathematical calculations in MQL5 and MQL5 Cloud Network?

 

Not everyone knows that the MetaTrader 5 tester can test not only trading strategies but also solve a whole class of universal problems. For this purpose there is a special mode "Mathematical Calculations", which disables data delivery of the market environment, which greatly saves the amount of transferred data.

Currently we have implemented a number of important functions for the construction of a universal calculation network:

  • Mathematical Calculation" mode of operation to disable market environment.
  • Transmission of initial data through attachment of #property tester_file "my_file.dat", which allows us to process initial data of any size and return the results.
  • Transmission of source data in the form of #resource attachments.
  • Calculation server mode on the terminal and transfer data arrays from agents in the form of frames. Agents can return any data to the terminal via FrameAdd, while the server part of the same Expert Advisor is launched on the terminal side in a separate window to receive results via OnTesterPass entry points with data extraction via FrameXXX functions. The obtained data can be recorded on the terminal side, visualized in a chart, etc. You can learn more about it in the topic"Real-time control of optimization processes and transfer of massive data from the agents in MetaTrader 5".
  • Operation of local and remote agents (without MQL5 Cloud Network) with a separate server using the named pipes.
  • Renting a huge network of agents to speed up calculations.
  • Build your own calculation farm in your network on remote agents.

Of course, this is accompanied by: the security of MQL5, its speed, genetic enumerators, etc.

Hence the question - what other features should be included to improve the capabilities of the computational network?

 
Renat:

Hence the question - what other features should be enabled to improve the calculation network capabilities?

Allow the use of OpenCL in "mathematical computing" mode in the cloud.
OpenCL: Мост в параллельные миры
OpenCL: Мост в параллельные миры
  • 2012.05.16
  • Sceptic Philozoff
  • www.mql5.com
В конце января 2012 года компания-разработчик терминала MetaTrader 5 анонсировала нативную поддержку OpenCL в MQL5. В статье на конкретном примере изложены основы программирования на OpenCL в среде MQL5 и приведены несколько примеров "наивной" оптимизации программы по быстродействию.
 
TheXpert:
Allow OpenCL to be used in "math computing" mode in the cloud.

This has been discussed many times - it is architecturally impossible because of the way Windows drivers work. As long as the calculation functions are a protected part of the video drivers, there is no solution. Right now, even if you connect via Remote Desktop to Windows, you will find that there is no access to OpenCL.

As soon as the video card manufacturers move the calculation functions to a separate interface independent of the video drivers, there would be some happiness. So far, only Nvidia Tesla is going in that direction.

 
In theOnTesterInit() function

Add the transfer of raw data to the agent, not as an attached resource, but as an array of data generated in the course of optimisation.

Approx:

void OnTesterInit()
  {
   // здесь формируем массив данных (например ArrayPass[])
   // и передаем сформированный массив агенту для расчетов
   return(ArrayPass);
  }

.

...

Please.

 
And who has to transmit this data and how exactly?

It is the technical details of who, what and how that is of interest.
 
Renat:
1. who and how exactly should transfer this data?

2. it is the technical details of who, what and how that are of interest.

1. It should naturally be transmitted by the user, as I suggested above -"Approximately like this:", you know best.

2. Who is who and how, I can't describe )

What are the agents for? - To solve problems.

What tasks do we solve? - I don't know, the main thing is that they solve them.

We need input and output for solving these tasks.

The output is clear, but the input is unclear.

Maybe I missed something? How do I pass the input data (calculated based on previous output data) to the agent?

 
her.human:

1. The transmission should naturally be made by the user, as I suggested above -"Approximately like this:", you know best.

2. Who is who and how, I can't describe )

The point is that we need exactly technically elaborated proposals in at least 2 steps. "Don't know what and how" is not good enough.

 
her.human:

How to pass the input data (calculated from previous output data) to the agent

This has already been asked, controlling the progress of optimization would sometimes be very helpful.

But a lot of questions arise on a closer dive (unless the optimization is done on a single local agent).

 
Renat:

The point is that we need technically sound proposals in at least two steps. "Don't know what and how" is not good enough.

You should say so - "proposals are accepted in the form of terms of reference", otherwise it is perceived as "we will listen to ideas".
 
Komposter, not so categorical, but still a bit more technical.

For our part, we are thinking about independent communication between agents without a terminal. For example, one of the agents could generate the initial data and send it to the others by fordcast.
 
Renat:
We, for our part, are thinking about independent communication between agents without the involvement of the terminal. For example, one of the agents can generate initial data and send it to the others by fordcast.

It's a little bit different. I wanted to control the course of optimisation.

That is, to generate parameter sets as the optimization progresses and send them to the agents for execution.

Naturally, there will be a lot of questions. I'm not ready to answer them.