Multicore/thread tester

[Jeremy Roach]

Hi all,

 Just wondering if there is an option somewhere that affects the use of multicore/thread CPUs in the MT5 tester?

I ask, as previously - up until yesterday my MT5 tester would stream through testing using all 8 cores, now it only does 1, then it might start using another core 3/4 through the whole test.

Does anyone know why this is going on and how to fix it?

My PC isn't doing much else other then testing.

Thanks 

[jcollin.be]

Hi,

 I also have an issue with the tester on my 8 core I7.

The Tester uses all the cores but it is very very slow and even much slower than on another machine 4 cores I5.
For the same EA, it takes 6 minutes to perform 100 passes on the 8 core and only 22 seconds on the 4 core!!! And I clean the cache before each execution.

 Is there something wrong with MT5 when run on 8 core machines?

Of course, solutions are welcome :)

 Ah and I cannot use the cloud or the remote agent as I use external dlls.

 Thank you for your answers? 

[Lars Rompe]

Hi both,

 

1. first of all you may want to check if all cores are really active

Therefore enter the msconfig (I assume you know how that works - keep in mind to do that changes as ADMIN) and click:

• Win-R - msconfig
• Boot tab 
• Advanced Options
• Number of Processors

You may check the processors that are actively used by Windows

 

 2. You may limit the cores used by certain programs

Enter:

• Press Ctrl+Alt+Del 
• Start Task Manager 
• Processes Tab 
• right click desired program 
• Set Affinity  

Adjust according to your expectations (e.g. to free some CPU for your regular work ;-))

3. Disable Hyperthreading Cores

In case you own Intel dual/multi-core - it has 4 "logical" cores and a quad-core has 8 "logical" cores which is meant by "Hyperthreading". 

I think Hyperthreading isn't working as it should in the marketing material - CPU is getting hot and the real benchmark results in the performance section of Windows do not improve.

Also subjective feeling is that it does not work.

You can disable these logical cores in the mainboard BIOS. 

You will have to press F2 during boot (or F8 or F12 in certain models) to enter BIOS. 

The option you have to look for is named "Hyperthreading [Enabled/Disabled]".

Enjoy!

[Stephen Njuki]

Also make sure you have enough virtual memory per core/thread. Typically 2GB per thread is a minimum.

[Ruy Bentes]

What a great explanation, Lars! Thanks!!

[TradeNzt]

Hi! i have the same problem... i also modified my EA to work with classes and functions as OOP model require, but still backtest on a single core :-(   .. i urgently need help... 

[Stanislav Korotky]

TradeNzt:

Hi! i have the same problem... i also modified my EA to work with classes and functions as OOP model require, but still backtest on a single core :-(   .. i urgently need help... 

You have already asked this and got the answer here. No matter which paradigm you use - OOP or functional programming - single backtest runs on a single core/thread.

[TradeNzt]

Stanislav Korotky:

You have already asked this and got the answer here. No matter which paradigm you use - OOP or functional programming - single backtest runs on a single core/thread.

have you ever tried using dll? they are executed in separated threads.

[Stanislav Korotky]

TradeNzt:

have you ever tried using dll? they are executed in separated threads.

What exactly do you want to calculate in parallel? Probably there exist a better way to do what you're about to feed into threads according to your current design.

Bear in mind that trade execution in the tester is consecutive in any case.

Did you profile your code? Which parts induce the main time losses?

[David John Thompson]

Hi

I am setting up a 'local farm' to run optimisations.

My first MT5 terminal machine is a dual core PC, with HT - so effectively 4x logical cores
When running optimisations locally I see / use 4x cores.

I then started to use a more powerful dual Socket Xeon Server - each CPU is 8x core HT - so a total of 32 logical cores
When I started running optimisations locally I see / use 32x cores

The MT5 documentation indicates that when a 2nd/3rd etc server is available on the LAN, and the logical CPU cores are made available I can use all them - ie the above case 32+4
Manual : https://www.metatrader5.com/en/terminal/help/algotrading/metatester says:

"Agents — number of agents you want to install. It is recommended to install as many agents as there are logical processor cores."

I have found that when running the METATESTER only the physical, not logical cores are listed - hence on the 'slave' servers I'm only able to use 50% of the logical cores

Is this correct (and the manual is wrong), or is there a specific setting I need to look for?

Any help greatly appreciated

[Alain Verleyen]

That's a known issue with current Build 2280.

Forum on trading, automated trading systems and testing trading strategies

Metatester 5 agent manager strategy tester does not see processor threads.

Renat Fatkhullin , 2019.11.27 06:12

Due to the apparent lack of memory with an excessive number of agents and a decrease in the calculation speed on hyper-trading kernels, we decided to limit ourselves to physical kernels when working in cloud.

Now in Cloud they are running very expensive tasks, and the vast majority of Cloud participants have a banal lack of memory with many agents. For example, 2 GB per agent and 16 agents already require at least 32 GB of memory, although 8-16 GB is usually available.

As a result, many passes fail, wasting the network and agents. But most importantly, they are slowing down the calculation due to expectations of results and restarting tasks.

We have long been assessing the approximate resource sufficiency of agents before issuing tasks to them, and one of the most effective is to work only on physical cores in cloud.

Locally, you can use all the kernels as you can easily manage their shutdown.

Metaquotes CEO also said they will work on it.