All (not yet) about Strategy Tester, Optimization and Cloud - page 7

 
Sergey Golubev:

Good article was published -

----------------

Continuous Walk-Forward Optimization (Part 1): Working with Optimization Reports 

...

Continuing with Part 2

----------------

Continuous Walk-Through Optimization (Part 2): Mechanism for creating an optimization report for any robot

This is the next article within a series devoted to the creation of an automated optimizer, which can perform walk-through optimization of trading strategies. The previous article described the creation of a DLL to be used in our auto optimizer and in Expert Advisors. This new part is entirely devoted to the MQL5 language. We will consider optimization report generation methods and the application of this functionality within your algorithms. 

The strategy tester does not allow access to its data from an Expert Advisor while the provided results lack details, therefor,e we will use the optimization report downloading functionality implemented in my previous articles. Since separate parts of this functionality have been modified, while others were not fully covered in earlier articles, let's consider these features once again as they constitute the key parts of our program. Let's start with one of the new features: addition of custom commission. All classes and functions described in this article are located under the Include/History manager directory.

Continuous Walk-Through Optimization (Part 2): Mechanism for creating an optimization report for any robot
Continuous Walk-Through Optimization (Part 2): Mechanism for creating an optimization report for any robot
  • www.mql5.com
This is the next article within a series devoted to the creation of an automated optimizer, which can perform walk-through optimization of trading strategies. The previous article described the creation of a DLL to be used in our auto optimizer and in Expert Advisors. This new part is entirely devoted to the MQL5 language. We will consider...
 

today I tried tests on my local farm and my metatrader 5 used on linux dissapeared my agents, I tried installing metatester alone but still not work.

and journal states "2020.04.18 17:15:22.124    Tester    Cloud servers switched off"





 
herrcrowley:

today I tried tests on my local farm and my metatrader 5 used on linux dissapeared my agents, I tried installing metatester alone but still not work.

and journal states "2020.04.18 17:15:22.124    Tester    Cloud servers switched off"


...

It may be some limitation ...
I know that cloud does not work in VPS and in 32-bit Metatrader (but I am not sure about Linux ... it may be same limitation):

  • Tester agents/Remote agents/MQL5 Cloud Network agents‌/MetaTester 5 Agents Manager are no longer available for use in the 32-bit terminals: they can work only in the 64-bit systems: read more here
  • Prohibition for using testing agents working on virtual OS in MQL5.Cloud - read more here
List of changes in MetaTrader 5 Client Terminal builds
List of changes in MetaTrader 5 Client Terminal builds
  • 2014.06.26
  • www.mql5.com
List of changes in the Help for MQL5:Corrected description of the GlobalVariablesFlush() function , input parameters are not required, Length the c...
 

I want to use different subset of PC in like IP 180.214.90.6 ,not use local network 192.168.1.5 

192.168.1.5 is pass ok connect and work good.

180.214.90.6 log always show connect ....  (no password issue or task going on..) 

If this is possible to do? 


I make more test case .. (And there is no debug message to make sure what happen ?! It's sock ><)

Test Env MT5 Build 2410(08 May 2020) / Win10 x64 base /All PR >120  All software use same version.

NB 192.168.18.3

PC1 192.168.18.7

PC2 180.214.90.6 --->(192.168.18.5)

PC3 192.168.18.8 (Ubuntu) 

Case A NB can see PC1 (But speed is limit by lowest one, it's look like load balance is not working? )

Case B PC1 can't see NB

Case C NB,PC1 can't see PC2

Case D PC1 can see PC1 in local net.

Case E NB can see PC3 (ubuntu after add  winbind)

I try different way to use. And PC1  get several agents inside , I don't know if it will get side effect ?

And I try to check firewall ,and remove agents and add it again.

It's not work ><  

 
Continuous Walk-Forward Optimization (Part 1): Working with Optimization Reports
Continuous Walk-Forward Optimization (Part 1): Working with Optimization Reports
  • www.mql5.com
In the previous articles (Optimization Management (Part I) and Optimization Management (Part 2)) we considered a mechanism for launching the optimization in the terminal through a third-party process. This allows creating a certain Optimization Manager which can implement the process similarly to a trading algorithm implementing a specific...
 
Sergey Golubev:

"Due to the apparent lack of memory with an excessive number of agents and a decrease in the speed of calculations on hyper-threading cores, we decided to limit ourselves to only physical cores when working in cloud.

..

We have long been evaluating the approximate resource sufficiency of agents before issuing tasks to them, and one of the most effective is to work only on physical cores in cloud.

Locally, you can use all the cores as you can easily control their shutdown."

On my new hardware (AMD Ryzen 9300, 32GB DDR4) I am observing a number of agent results - that were (presumably) running on hyper-threaded cores, produce erroneous results in the strategy tester.

So, as this appears to me, it's not possible to use all the cores locally - or can anyone confirm that testing works on his/her hyper-threading cores?

 

New MetaTrader 5 platform build 2530: Sorting in Market Watch and advanced operations with optimization results

Tester: Optimized work of the MQL5 Cloud Network. Now, when an optimization is launched for the same Expert Advisor, the network tries to use the same testing agents which were used during the previous optimization pass. This is because the required market environment, including the trading history, is already available for such agents.

New MetaTrader 5 platform build 2530: Sorting in Market Watch and advanced operations with optimization results
New MetaTrader 5 platform build 2530: Sorting in Market Watch and advanced operations with optimization results
  • 2020.07.09
  • www.mql5.com
The MetaTrader 5 platform update will be released on Friday, July the 10th, 2020...
 

We will have the big improvement soon related to the cloud.
This is the reply from MetaQuotes (machine translation to the English):

Forum on trading, automated trading systems and testing trading strategies

Cores were disappeared after updating the tester.

Renat Fatkhullin , 2020.07.25 18:20

In cloud, we only allow using the physical cores.

Statistics showed that many tasks in the cloud fail due to lack of resources. Very expensive experts went.

Over the past month, we have dramatically improved the MQL5 Cloud Network in terms of speed and responsiveness.


When we move to a new tester that will use one process for all cores instead of separate processes for each core, we will drastically reduce the need for resources due to consolidated caches and will be able to flexibly manage the load on all cores.

Here I described the plans in more detail: https://www.mql5.com/ru/forum/340770/page7#comment_16401357

Forum on trading, automated trading systems and testing trading strategies

New version of the MetaTrader 5 platform build 2450: "Subscriptions" service, interface improvements and convenient functions in MetaEditor

Renat Fatkhullin , 05/20/2016 12:35 PM

Yes, the tester will be rewritten/reprogrammed, as the current one is choked with data and an unsuccessful scaling scheme.

Each agent will work as a microcloud with N cores to keep one copy of the data for N threads. Since the time has come for the NUMA architecture among the masses, each agent will clearly live in its own NUMA node to avoid the penalties for accessing memory outside of its node.

This will significantly reduce memory requirements, reduce disk activity and increase the speed of optimization.

In this case, the main role will always be played by the volume of RAM, which allows you to keep more caches and quickly issue them.

The goal is to screen thousands of stocks in bulk, where we are heading with a new initiative to launch detailed stock data subscriptions.