Errors, bugs, questions - page 2963

[Nikolai Semko]

fxsaber:

Indicator1 and Indicator2 must then be synchronised in some way so that both numbers can be written on the same common line.

This is a very complex solution.

Why would they need to be synchronized? user32.dll will do it for them.
The main thing is to number them correctly, so that each number has a unique number and this number will be an index of the data array.
This can, as a last resort, be done manually. Or you can do it automatically. By the way, I recently published something similar in KB(Indicator Cases).
What's not to like about resources? I think it is optimal within the confines of one terminal. The variant I suggest, using user32.dll (I implemented this variant about 10 years ago, when I was young and dabbled in arbitrage), access time and data parsing are about 50 microseconds (I think it can be sped up in 1.5-2 times). Is it any slower with resources?

[fxsaber]

Nikolai Semko:

Why would they need to synchronise? user32.dll will do that for them.

Try writing it. Perhaps you don't fully understand the task.

Why don't you like resources? I think it's optimal within the confines of a single terminal. The variant I suggested using user32.dll (I implemented this variant about 10 years ago, when I was dabbling in arbitrage when I was young) has access and parsing time of about 50 microseconds (I think it can be sped up 1.5-2 times). Is it slower with resources?

It takes 100 microseconds to read on a home machine under ideal conditions. An Expert Advisor on a single tick can cause reading a hundred times. It's slow.

In ideal conditions, GlobalVariableGet will be executed in 10 microseconds. But that's not an indicator, as under new to me combat conditions it's a horrible brake.

[Andrey Khatimlianskii]

fxsaber:

This is HistoryTicks - catching all ticks for EAs. Therefore, EventChartCustom is not suitable, it has its own queue. It is the same with the buffer.

I have it working on EventChartCustom. 99.8% of ticks are received within 0.15 ms.

Forum on trading, automated trading systems and strategy testing

EventChartCustom => indicator is too slow

Andrey Khatimlianskii, 2019.12.12 09:27

Here are the statistics for 9 hours of working with 5 symbols:

Windows 8.1 (build 9600) x64, IE 11, UAC, Intel Core i5-3570  @ 3.40 GHz, Memory: 6979 / 16346 Mb, Disk: 341 / 499 Gb, GMT+2

[USDCHF]: 22784 of 22833 (99.8 %) ticks were processed (0.14 ms delay in average), 49 (0.2 %) ticks were skipped (103.4 ms delay in average)
[EURUSD]: 22944 of 22974 (99.9 %) ticks were processed (0.16 ms delay in average), 30 (0.1 %) ticks were skipped (115.6 ms delay in average)
[USDCAD]: 15331 of 15347 (99.9 %) ticks were processed (0.13 ms delay in average), 16 (0.1 %) ticks were skipped (104.6 ms delay in average)
[EURCHF]: 22516 of 22571 (99.8 %) ticks were processed (0.13 ms delay in average), 55 (0.2 %) ticks were skipped (127.8 ms delay in average)
[EURAUD]: 66842 of 66924 (99.9 %) ticks were processed (0.13 ms delay in average), 82 (0.1 %) ticks were skipped (117.8 ms delay in average)
[GBPUSD]: 41393 of 41393 (100.0 %) ticks were processed (0.00 ms delay in average)
Total trade requests time: 4.280 sec

The Expert Advisor was on GBPUSD, so native OnTick worked for it.

No "indicator is too slow" error.

In contrast the percentage of missed ticks and the average latency is much higher on MQ's VPS (I'll post statistics later).
And there are a lot of "indicator is too slow" errors.

I don't understand the nature of queue overflow, because the Expert Advisor processes the accumulated events immediately (it just returns).
Does anyone else process it?

[fxsaber]

Andrey Khatimlianskii:

I have it working onEventChartCustom. 99.8% of ticks are received within 0.15 ms.

I send ticks from the indicator via this: sparam contains MqlTick, lparam - tick number.

The Expert Advisor in OnChartEvent catches these ticks. And it needs to understand if the current tick is the most actual one or not? I.e., is there a queue of ticks or is it empty?

For this, it reads the number (the task is to read this number) of the latest tick sent by the indicator. If the tick has the same number - the queue is empty, and it's possible to start working with ticks.

And during the operation of OnTick, after OrderSend it's necessary to check if the indicator has sent more ticks. For this, we again need to read the number from the indicator. And there can be more than a hundred of such checks during one OnTick. That's why we need to read quickly.

[traveller00]

Within 1 terminal, the fastest winapi will be memory allocation (global within a process) and interlocked functions like https://docs.microsoft.com/ru-ru/windows/win32/api/winnt/nf-winnt-interlockedexchange.

These are non-blocking, atomic and essentially executed in a few asm instructions.

[fxsaber]

traveller00:

Within 1 terminal, the fastest winapi will be memory allocation (global within a process) and interlocked functions like https://docs.microsoft.com/ru-ru/windows/win32/api/winnt/nf-winnt-interlockedexchange.

These are non-blocking, atomic and essentially executed in a few asm instructions.

Not without an example.

[traveller00]

fxsaber:

To do this, it reads the number (the task is to read this number) of the most recently sent tick by the indicator. If the received tick has the same number - the queue is empty, and you can start working with a pack of ticks.

And during the operation of OnTick, after OrderSend it's necessary to check if the indicator has sent more ticks. For this, we again need to read the number from the indicator. And there can be more than a hundred of such checks during one OnTick. Therefore, we need to read it quickly.

Initialization.

1. In the first thread (most likely, the writing thread), allocate memory in any way for the variable of the required size.

2. In the required threads (reading threads), send the address of this memory.

Basic work.

3. The writing thread initiates InterlockedExchange or InterlockedExchange64 depending on the variable size for writing to it.

4. Reading thread pulls e.g. InterlockedCompareExchange to read.

Completion.

5. Release allocated memory, preferably in the same thread that allocated it.

If necessary it can be repeated for creation of several counters. On the downside, you will need the WinAPI connection. Of the features of the allocated memory address should be aligned, but by default it usually is.

Working will be within 1 process, memory is shared for the threads of 1 process. If needed, there are other interlocked functions like InterlockedDecrement, InterlockedAdd, etc.

Functions are non-blocking, do not wait for anything, automata, they are executed in a few asm instructions.

P.S. As far as I remember, regular read and write operations via mov in assembler are atomic anyway. And if the compiler doesn't make a mess (it shouldn't in theory), we can try to read and write to the variable in the allocated memory and it will be atomic.

If it suits the task, it is unlikely that it will be faster within 1 process. For interprocess the fastest will be similar, but with shared memory, in this case you can't do without WinAPI.

[Roman]

fxsaber:

This scheme does not seem to work. Show an elementary example, please.

Why not workable? Just the way I see it. The getter setter principle.
Getting the value of a hidden variable through a function.
If you pass the MqlTick structure, define your structure with the fields set like MqlTick, and add your counter field to the structure.
And return this structure from export function.
An elementary example for the indicator. Do not pay attention to the example in the script.

struct myMqlTick 
{ 
   datetime     time;          // Время последнего обновления цен 
   double       bid;           // Текущая цена Bid 
   double       ask;           // Текущая цена Ask 
   double       last;          // Текущая цена последней сделки (Last) 
   ulong        volume;        // Объем для текущей цены Last 
   ulong        count;  //Свой счётчик
   
}myStruct;

//--------------------------------------------------
myMqlTick GetTickStruct() export
{
   
   return(myStruct);
}

//+------------------------------------------------------------------+
//| Script program start function                                    |
//+------------------------------------------------------------------+
void OnStart()
{
   myStruct.time   = 0.0;
   myStruct.bid    = 0.0;
   myStruct.ask    = 0.0;
   myStruct.last   = 0.0;
   myStruct.volume = 0;
   myStruct.count  = 1;  //свой счётчик  
  
}
//+------------------------------------------------------------------+

In Expert Advisor, you call GetTickStruct and get the entire structure with your counter.

[fxsaber]

Roman:

In Expert Advisor you call GetTickStruct, you get the whole structure, with its counter.

Please write an elementary number transfer from indicator to EA.

[Roman]

fxsaber:

An elementary number transfer from indicator to EA, please write.

Replace the structure with one variable ))