Interesting topic for many: what's new in MetaTrader 4 and MQL4 - big changes on the way - page 57

 
Avals:

well)) I.e. if I want to test with a bigger lot, or another entry/exit method, I need to manually generate a new custom story))

Exactly! Note that we are not talking about indicative history, which is seen by the battle robot, but about a custom history for Execution in the tester. Obviously, the change of volumes in TS leads to a change in the Execution-history. Or, for example, a change in ping - similarly.

Generating the Execution-history during testing is absurd. For this reason, we need to correctly estimate the result with reinvest-TC. Because as the volume increases on the fly the Execution-history changes (deteriorates). And since it is impossible to predict in advance the dynamics of changes in volumes it operates with, it is also problematic to collect the Execution history in advance. But because of this, in particular, nuance, no one in their right mind would generate the Execution-history in a real-time tester.

Again, the golden mean is important.

You, of course, do not need such a dance with tambourine, as everyone practically. But there are TCs where you can't do without dancing.

To squeeze the maximum out of their TC (any), you need to try very hard. Almost no one does this, being content with what they get.

 
hrenfx:

Exactly! Note that we are not talking about the indicative history seen by the battle robot, but about the custom history for Execution in the tester. Obviously, changing volumes in the TS leads to a change in the Execution history. Or, for example, a change in ping - similarly.

Generating the Execution-history during testing is absurd. For this reason, we need to correctly estimate the result with Reinvest-TC. Because as the volume increases on the fly the Execution-history changes (deteriorates). And since it is impossible to predict in advance the dynamics of changes in volumes it operates with, it is also problematic to collect the Execution history in advance. But because of this, in particular, nuance, no one in their right mind would generate the Execution-history in a real-time tester.

Again, the golden mean is important.

You, of course, do not need such a dance with tambourine, as everyone practically. But there are TCs where you can't do without dancing.

To squeeze the maximum out of their TC (any), you need to try very hard. Almost no one does this, being content with what they get.

And there's no need to test with reinvestment)) such things only with Level2 without any pre-processing (either manually or automatically). In other cases, the execution volume can be taken into account accurately enough. Of course, this can only be estimated in real trading without influence on the market of your own orders.

There are 2 global ways to use real ticks:

1. the terminal stores ticks and preprocesses them before testing to save resources.

2. the user stores the ticks, compresses them himself (but according to the tester's instructions) and gives them to the tester. For example, as now in mt5, but tf<1min. Then you can build with your requirements in mind.

I think most will be satisfied with any of these options.

P.S. But MQ earns not from those who test their strategies in MT5, but from brokers and DCs. The latter ones care about trading through them and it does not matter for them who and how they test. That is, they care about the trading terminal, not the tester. But, the popularity of the platform depends on the tester, and hence the choice of broker) The dilemma - custom history and more accurate testing will somewhat detach users from the DC/broker (which will not please the latter), but it will somewhat increase the popularity of the platform. Which is more profitable is up to MQ)

 

You are simply not familiar with the portrait of the average terminal user, which is why you have such a distorted view of what affects the popularity of the platform.

Renat is doing almost everything right to popularize the mass product among traders. Yes, there are a lot of restrictions, but the average user does not even feel them.

The average broker is also a dumb creature. And it targets the average user at best.

The big money in FOREX is behind platform-independent platforms with advanced approaches to aggregation. We need to improve trading conditions to build volume.

Compare simply which scheme has potentially more money:

  1. Collect a subscription fee from brokers for using their platform.
  2. Collect a commission (from turnover) from any brokers + institutions, for the use of the platform.

Not on a level playing field, the same LMAX is abandoning retail clients and moving to the second scheme - to be the best LP, competing with EBS and others. Moving retail to their IBs with MT4/5 and other platforms.

Whoever is serious about algotrading does not depend on trading platforms.

 

Forum on trading, automated trading systems and testing trading strategies

Who Supports MT5

MetaDriver, 2013.08.13 01:36

If the increase in clients is greater than the loss of implementation, it is clearly a rent. If the client growth is greater than the implementation loss - it is already clearly profitable.

On the contrary.
 
hrenfx:

You are simply not familiar with the portrait of the average terminal user, which is why you have such a distorted view of what affects the popularity of the platform.

Renat is doing almost everything right to popularize the mass product among traders. Yes, there are a lot of restrictions, but the average user does not even feel them.

The average broker is also a dumb creature. And it targets the average user at best.

The big money in FOREX is behind platform-independent platforms with advanced approaches to aggregation. We need to improve trading conditions to build volume.

Compare simply which scheme has potentially more money:

  1. Collect a subscription fee from brokers for using their platform.
  2. Collect a commission (from turnover) from any brokers + institutions, for the use of the platform.

Not on a level playing field, the same LMAX is abandoning retail clients and moving to the second scheme - to be the best LP, competing with EBS and others. Moving retail to their IBs with MT4/5 and other platforms.

Who is serious about algotrading does not depend on trading platforms.

The two options are well illustrated by two worldviews.

One says to activate the environment, the other says to freeze everything as is. Democrats and Republicans, Liberals and Conservatives.

The first option (with a subscription fee) forces the broker to update its clients and encourage them to trade (the subscription fee must be paid back), otherwise the broker suffers losses and whether to remain part of the marketplace is questionable.

The second option doesn't matter, even if there is not a single trade in a month, the broker will still be a user of the platform (no transactions, no expenses).

What is better is a perennial question, there are times when one is better and times when the other is better.

 

I needed to improve my tester for a new trade. To understand the code of the old one was more time wasted (I killed several weeks - occasional approaches), than to write a new one.

So, it took me 5 hours (with debugging) to write a new tester from scratch. Its characteristics (satisfies me as a start):

  • Works with M1 HighBid + LowAsk betting model (results are more accurate than MT5 tester).
  • One symbol.
  • Speed (i7 2700K) about 100,000,000 bars per second (277 years of FOREX per second) on the test TS - not empty, lots of things calculated. Constantly in the market.
  • No MM - no lots.
  • Profit only in pips.
  • Trader has possibility to control the slippage and commission.
  • Optimizer with its own optimization criteria - for each of them it opens its own file (can take many GB) with sorted strings of trades.
  • There are no error checks in the tester - I know all the nuances of my tester, so I don't allow it.
  • TS is written in a slightly abridged MQL4 - all unnecessary things have been removed. But you can use all the power of C++.
  • The tester is written without OOP - I don't know how to do it competently. So it's practically a pure C language.
  • Source code in free C++~ 20Kb.
  • Stops and marketplaces were thrown out (OrderClose was left) - not necessary.
  • Every run (by choice - specify run number) can be visualized and studied in the math package - Equity and Balance changes file is written simply.
  • No genetics.
  • No OpenCL - can't do it.
  • Single-threaded. Loads all cores stupid - multiple optimizers manually run.
  • Console application.
  • TC input parameters, tester settings and optimization ranges are set in file.
  • TC is compiled together with the tester into one EXE file.
  • Before the start there are only three files: EXE, history, settings.
  • The optimizer has an interruption of the run, if the current state of the run does not satisfy the conditions (for example, subsidence is too high).
  • MQL4 script prepares the history for the tester, it was written long ago.
  • No dynamic allocation of memory for the order table - once allocated memory and forgotten.
  • No logs - I do not look at them.
  • No order history - in the same way.
  • There is no idea about indicators and everything to do with them - we do not need them for TS.
  • Prices are integer (long int).

If the same nonsense is implemented in MQL5, we can use Cloud in the matrix optimization mode. Only history has to be sent every time - the built-in compression of such information is required.

Theoretically, the speed of ~ 100 Bb/s (at the test TS) can theoretically be achieved. I wonder what is the performance of MT5-tester on the whole cloud in the same parrots?

100 billion bars per second is a good speed for all sorts of research. If converted to other units, that speed says that a year of one minute of FOREX history is run ~300,000 times per second on one symbol.

Most of the time was spent learning the syntax of the language - just googling it. Not a programmer.

I'll tell you right away, writing a universal framework for your own modest needs is a huge waste of time. Better to explore. If something needs to be taken into account - just add it.

This is only a core, now you need a cunning toolkit for the optimizer. It will take considerably more time here - you have to think.

The Dev-C++ Resource Site
  • Hongli Lai
  • www.bloodshed.net
Dev-C++ is Free Software distributed under the GNU General Public License. This means you are free to distribute and modify Dev-C++, unlike most Windows software! Be sure the read the license. Donations Please support Dev-C++ by making a donation ! The money will be shared between the active developers and the support manager in order to...
 
hrenfx:
  • Speed (i7 2700K) around 100,000,000 bars per second (277 years of FOREX per second) on the test TS - not nothing, lots of things are being calculated. Constantly in the market.

Most importantly, don't forget to add reading from the disk to the time.

The number of 100,000,000 bars is very pleasing, especially with the clarification that there is HighBid + LowAsk (i.e. no). Or is there only 100,000 bars?

Analyse the test results in Excel? Calculate in a second and look at Excel for an hour?


For comparison, the MetaTrader 5 tester on M1 on Openbar, without indicators, but with all baggage and services (which you have not bothered with), including GUI animation and lifting the entire base from disk:

  • EURUSD,M1: 19,538,553 ticks (5,180,083 bars) generated within 7906 ms

which gives about 2.4 million ticks per second on a similar i7.

 
Renat:

The main thing is not to forget to add reading from disk to the time.

This is useless time, as it has no effect on optimisation. Now you have SSD, you can arrange RAM-Drive as well. In short, this time is of no use.

The number of 100 000 000 bars is very pleasant, especially when specifying that there is HighBid + LowAsk(i.e. no). Or is there only 100 000 bars?

Why do you think there isn't? I only used a few hundred thousand bars for the test TS. I.e. 100 million bars per second is the optimization speed. So the duration of initial history is not important for this parameter.

For example, the initial history of 100 000 bars is run 1000 times per second in the optimizer - it means the speed of 100 million bars per second.

Analyse the test results in Excel? Calculate in a second and look at Excel for an hour?

Excel is crap (don't really know it simply). In matrix packages it's elementary to twist and turn the result of a run. You don't need to change the matrix document itself. Almost any characteristics of TC gives out on the spot. This is much more advanced and efficient than analysis in an MT5-tester or in any other tester.

By the way, the MT5-tester has no filters of optimization results. I integrated them immediately (without GUI, but 100% justified) . For example, I sorted runs by profit. I've set filter by drawdown, and everything is clear: row below is always a run with less drawdown than the one above it. Very effective method - I recommend you to implement it in your tester.

For comparison, MetaTrader 5 tester on M1 on Openbar, without indicators, but with all baggage and services (which you didn't bother with), including GUI animation and lifting of the entire base from the disk:

  • EURUSD,M1: 19,538,553 ticks (5,180,083 bars) generated within 7906 ms

which gives approximately 2.4 mln ticks per second on a similar i7.

Better quote the optimization speed (wrote above) - in theory it should be higher for you. Not quite sure why the ticks were generated in openbar mode?

By the way, does MT4 still beat (and if I take into account MQL4 code acceleration?) the speed of MT5 in openbar mode in optimization or not?

 

I see, so it's exclusively about the cycle for(i=0;i<100000;i++) { microdevice(); }, costs for everything else are discarded.

In this case there are no speed effects to speak of. Because you can't even launch a task of optimization on such a bare-metal engine. You cannot launch any research. You immediately face the problem of "you need indicators, convenient analysis mechanisms, mathematical functions and so on" for a sensible task.

About matpackages, where you spin everything you want - also from the same thread, as "I don't notice overheads". Nothing in matpackages (how much do they cost, by the way?) is spun, everything has to be done hand-to-hand, worse than excel. Not to mention getting all the trades correctly displayed on the price chart with one button.

We're still checking/overlaying key points on the openbars. And yes, we do simulate the development of history from ticks for the sake of accuracy and not being able to look ahead. We don't have a messy "go through the bars as is" mode, you just turn a blind eye to the fact that in your case the full bar lies whole, you just honestly don't look at anything but the Open price, thereby protecting yourself from looking into the future. Can you imagine what they would call us if we rolled out a mode that allows you to look to the future? Exactly.

MT4 is architecturally faster than MT5 because of the tester integrated into the terminal, the lack of overhead costs on transferring the whole market environment to a third-party process via TCP/IP, unpacking the well-compressed history on the agent, running the test, relaying logs and statuses to the terminal and then returning the results.

What is amazing, for so many years you have been telling about arbitrage, scalping, vortexing of micropips in the tick-flow, you criticize our testers, and now you show us a for-bar cycle with two bid/ask prices.

 
Renat:

I see, so it's only a loop for(i=0;i<100000;i++) { microdeal(); }, everything else is discarded.

Roughly speaking, the optimizer is a stupid enumeration of parameters for each run. So, the for loop will do just fine too. There is no expense incurred for anything else but testing.

We cannot speak about any speed effects in this case. You cannot even run an optimization task on such a bare-metal engine. You cannot launch any research whatsoever. You immediately face the problem of "we need indicators, convenient analysis mechanisms, mathematical functions and so on" for a sensible task.

The optimisation task is not only run, but also solved. Otherwise I wouldn't be able to measure the average speed of optimization of my test TS (millions of passes with different values of TS input parameters). The tester/optimizer is the toolkit for studying the market for various patterns. I've written only a few indicators, and only for CodeBase. Indicators are not really needed for building TS. I never use them. I've never used them. The mechanism of market pattern analysis lies in the ability to decompose the resultant optimization matrix by kernels. This does not exist in matrix packages or in any testers. Something homemade for myself in this case is better than nothing. The reasonableness of the task depends on what bell tower to look from. For example, from my vantage point almost all research tasks which are solved here are unreasonable. In the end, time will show who is right.

About mate-packages, where you twist everything as you like - also from the same point as "I don't notice the overhead". Nothing in mate-packages is twisted, you have to do everything hand-to-hand, worse than excel. Not to mention, on a price chart, you have to display all the trades correctly with one button.

Well you have different characteristics of the run being shown. The Matrix package can do this automatically too - you don't have to do anything manually every time. I have never needed to overlay trades on a chart. But I do understand that the Matrix will not have any problems with that either. Today, mathematical packages can do a lot of things and all of them have great visualization and programming capabilities. Once a template is created, the only thing to do next is to input the tester results. It is possible to write a tester in a matrix package, but it will be strongly inferior in speed. In general, you are poorly aware of the capabilities of matrix packages.

We're still checking/overlaying the key points on the openbars. And yes, we do simulate story development from ticks for the sake of accuracy and not being able to look ahead. We don't have a messy "go through the bars as is" mode, you just turn a blind eye to the fact that in your case the full bar lies whole, you just honestly don't look at anything but the Open price, thereby protecting yourself from looking into the future. Can you imagine what they would call us if we rolled out a mode that allows you to look to the future? Exactly.

Oddly enough, I wouldn't get the chance to look into the future if I wanted to. Honestly, I don't know why your tester even once had that option. Apparently something was not taken into account architecturally somewhere. I don't really have such a problem with it - I checked it, otherwise I wouldn't talk about it.

MT4 is faster than MT5 because of the tester integrated into the terminal, the lack of overhead costs of passing the whole market environment to an external process via TCP/IP, unpacking the well-compressed history on the agent, launching a test, relaying logs and statuses to the terminal and then returning the results.

That's all cool. But I was asking about the net speed of the MT5 agent vs MT4 tester. Can you provide such data?

You've solved a very complex programming problem, which is nothing short of respectable. However, clearly overdone as far as the needs of algotrading are concerned.

The amazing thing is that for so many years you have been telling about arbitrage, scalping, wriggling of micropips in the tick flow, you have criticized our testers, and now you show us a forwards bar cycle with two bid/ask prices.

For arbitrage, of course, this tester is not suitable. For scalping and thickskating it's a super thing. I just need to run a trade in the near future, which will be public (with lag, of course). So I need to implement uncomplicated trading ideas. And to explore them thoroughly requires a very fast thresher and with good accuracy. Unfortunately, none of the available testers (not only yours) are able to solve this problem.

You, like many, have a very wrong idea of how a profitable TS is created. For this reason, your excellent programming toolkit is very poorly suited for research.

Still don't understand what you resent about arbitrage and scalping? Probably, I will show you scalping if you have a desire. Arbitrage - not yet.

P.S. If you do not know something - do not hesitate to do as I did - ask. This is normal and professional reputation will not lower, if it is important to you.