Metatrader5 Strategy Tester Limitations on the Cloud Network: how to fix? - page 3

[siolibros]

It is since many years we are already doing that.
On MT4 we already splitted backtest optimization on several instances on several desktop computers (about 10 instances per single desktop)

The big job was to collect all optimization results from all desktop computers, copying data from txt into database and working on it.
We already do it since about 5 years...very very annoying and frustrating job.

This because MT4 is not multithread and cannot allow distributed calculation like MT5 does.

But: If Cloud Network allows to use poor quality data history ONLY, It's a non-sense.
Why should I use the Cloud if cannot use high quality data history?

Professionals knows that backtest optimization have to be done with high quality data history.
Say tick, say 1M, or 1H timeframe. Original data history cannot contain holes.

HCS files (inside MT5 tester folder) use a great compression, why not to let sharing price data with high quality from MT5 to Agents in the Cloud?

[Fernando Carreiro]

siolibros:

It is since many years we are already doing that.
On MT4 we already splitted backtest optimization on several instances on several desktop computers (about 10 instances per single desktop)

The big job was to collect all optimization results from all desktop computers, copying data from txt into database and working on it.
We already do it since about 5 years...very very annoying and frustrating job.

This because MT4 is not multithread and cannot allow distributed calculation like MT5 does.

But: If Cloud Network allows to use poor quality data history ONLY, It's a non-sense.
Why should I use the Cloud if cannot use high quality data history?

Professionals knows that backtest optimization have to be done with high quality data history.
Say tick, say 1M, or 1H timeframe. Original data history cannot contain holes.

HCS files (inside MT5 tester folder) use a great compression, why not to let sharing price data with high quality from MT5 to Agents in the Cloud?

You did not read my post properly. You should have followed the link I provided, which would allow you to automate the compilation of the optimisation results.

Forum on trading, automated trading systems and testing trading strategies

Metatrader5 Strategy Tester Limitations on the Cloud Network: how to fix?

Fernando Carreiro, 2017.09.23 18:52

Complaining about the problem is not going to resolve it. Instead, focus on finding a solution.

So, consider coding your own solution by dividing your optimisations among the various PC's you have and use the MQL5 functions for working with optimisation results so as to consolidate your statistics.

[siolibros]

Fernando, maybe you didn't fully underdestood our question.

The question is not about how to automate the compilation of the optimization results.
We don't need that, because we are already doing that.

We need the great improvement of distributed calculation offered by the Cloud Network, using external high quality price data (i.e. Dukascopy).
Using price data from brokers is unuseful, it's not professional enough.

Actually, with our little server room (40 desktop computer), it takes about 1month of calculation to fully elaborate just one single pair on all timeframes (using high quality data history from dukascopy).
Results work great in live, so the way we found is correct!

We simply need to dramatically reduce backtest optimization time.

We expect to do that JOB (i.e one day) using the Cloud Network, with several thousand of Cloud Agents.

Hope this time it's more clear.

cheers

[Fernando Carreiro]

siolibros:

Fernando, maybe you didn't fully underdestood our question.

The question is not about how to automate the compilation of the optimization results.
We don't need that, because we are already doing that.

We need the great improvement of distributed calculation offered by the Cloud Network, using external high quality price data (i.e. Dukascopy).
Using price data from brokers is unuseful, it's not professional enough.

Actually, with our little server room (40 desktop computer), it takes about 1month of calculation to fully elaborate just one single pair on all timeframes (using high quality data history from dukascopy).
Results work great in live, so the way we found is correct!

We simply need to dramatically reduce backtest optimization time.

We expect to do that JOB (i.e one day) using the Cloud Network, with several thousand of Cloud Agents.

Hope this time it's more clear.

cheers

I understood the first time and as I said, you don't have the power to change MetaQuotes development based on a singular use. Even with a large number of people pushing for a change, it is still difficult to sway MetaQuotes' opinion and development.

So, instead of focusing on what is NOT available, focus on creating your own solution - your own "Cloud", by using the functions I referenced!

[siolibros]

thanks for your quick reply.

But that article does not explain / refers to find a way to increase power computational.

Anyway, I will test some personal ideas using a workaround, and if will work, will describe here.

regards

[Fernando Carreiro]

siolibros:

thanks for your quick reply.

But that article does not explain / refers to find a way to increase power computational.

Anyway, I will test some personal ideas using a workaround, and if will work, will describe here.

regards

I did not refer to any article or link in order to increase computational power! That was never your question in the first place!

[Alain Verleyen]

siolibros:

thanks for your quick reply.

But that article does not explain / refers to find a way to increase power computational.

Anyway, I will test some personal ideas using a workaround, and if will work, will describe here.

regards

Buy an MT5 Server licence, use your Dukascopy data as price feed. And test using your own "broker", it will not be a custom symbol but a normal one. The Cloud will be able to use it.

You need a serious budget though, but if you have a 40 computers room, maybe you have it.

[Stanislav Korotky]

siolibros:

Actually, with our little server room (40 desktop computer), it takes about 1month of calculation to fully elaborate just one single pair on all timeframes (using high quality data history from dukascopy).

This sounds like a flaw in your own custom design. You should probably refactor your code.

[siolibros]

Alain Verleyen:

Buy an MT5 Server licence, use your Dukascopy data as price feed. And test using your own "broker", it will not be a custom symbol but a normal one. The Cloud will be able to use it.

You need a serious budget though, but if you have a 40 computers room, maybe you have it.

Thanks Alain, this is a positive material possible solution.
I will consider it.

I also find this interesting article about using OPEN CL https://www.mql5.com/en/articles/690

I know you Alain already replied to that thread..

ty

[Alain Verleyen]

siolibros:

Thanks Alain, this is a positive material possible solution.
I will consider it.

I also find this interesting article about using OPEN CL https://www.mql5.com/en/articles/690

I know you Alain already replied to that thread..

ty

Yes could eventually be used, depends of your strategy. This is solution is more in same sense as re-factoring your code, sometimes it could be seriously improved.