Errors, bugs, questions - page 706

[Renat Fatkhullin]

MetaDriver:

1.

To create arrays of pointers to structures (arrays) and then initialize them for(i){ S[i] = GetPointer(StaticStruct[i]); }

2. to keep solid (packed) arrays of meaningful data.

Important when dealing with data output to OpenCL raw buffers (or sending to DLL, writing to files, etc.)

At the same time, it is possible to reorder data accesses (for example, when sorting pointers) without rewriting the data.

A language with secure execution should not expose/give direct access.

Classes have more protections and that is why they have a handle.

Only class objects have pointers. Instances of structures and variables of simple types do not have pointers. A class object that is not created using the new() operator, but, for example, automatically created in an object array, still has a pointer. Only this pointer will have the automatic type POINTER_AUTOMATIC, and you can't apply the delete() operator to it. In other respects, a pointer of type does not differ from a dynamic pointer of type POINTER_AUTOMATIC.

Since variables of the structure type and simple types do not have pointers, you cannot use GetPointer() on them. It is also prohibited to pass a pointer as a function argument. In all the above cases the compiler will report an error.

There will be no handles for other objects, as security is more important.

In OpenCL, any data types can be used for passing data, including structures. The void* is used there. Just prepare uniform data in the required format and get to work. Anticipating the question "I don't want to do it that way, I want to do it my way", I will answer that you cannot do that - security is more important.

[Vladimir Gomonov]

Renat:

1. any data type can be used for data transfer in OpenCL, including structures. The void* is used there. Just prepare uniform data in the required format and get to work. Anticipating the question "I don't want to do it that way, I want to do it my way", I will answer that you cannot do it - security is more important.

2. a language with secure execution should not expose/give direct access.

The point is that for all the classes, including the most primitive, mql5 compiler creates a VMT, with corresponding hidden field in the objects (pointer to VMT). And this pointer in the raw buffer (file) is too much for me. Not only it is trash taking up space, it also breaks the packet alignment inappropriately (OpenCL buffers have 128-bit alignment). That's the point. Classes are tempting to use: they are convenient to work with in mql. But... see above.

In Delphi there is a good alternative example of implementation. VMT and, accordingly, the pointer to VMT appears in the classes only after the first virtual method appears in the class hierarchy. If it were the same in mql5, the situation would be much more manageable. One could use classes without virtual methods instead of structures and there would be no structure-damaging "add-ons".

Now I've encountered a situation where the implementation of an abstract (intended for inheritance) class encapsulating an array of structures does not lend itself to inherited services (such as sorting). Replacing an array of structures with an array of classes solves this problem, but creates another.... (see above).

And I'm not asking for any "direct access" and I'm not interested in any address arithmetic. Why are you scaring children for nothing? :) mql5 handle is not even close to a "raw" pointer. And if it (surreptitiously) is - so the real security hole is here, not in the implementation of handles (pseudo-pointers) to any user data.

---

Right now your structures are actually classes without virtual functions (and VMT, respectively). So fine, just add some class features (handles) to them. Then you won't need pointers to arrays so badly either (you can wrap them into structures if necessary).

[Renat Fatkhullin]

MetaDriver:

The point is not that I want to do it my way; the point is that for all classes, including the most primitive ones, mql5 compiler creates VMT with corresponding hidden field in objects (pointer to VMT). And this pointer in raw buffer (file) is too much for me. Not only it is trash taking up space, it also breaks the packet alignment in a completely inappropriate way (OpenCL buffers have 128-bit alignment). That's the point. Using classes is tempting: they are convenient to work with in mql. But... see above.

In Delphi there is a good alternative example of implementation. VMT and, accordingly, the pointer to VMT appears in the classes only after the first virtual method appears in the class hierarchy. If it were the same in mql5, the situation would be much more manageable. One could use classes without virtual methods instead of structures and there would be no structure-damaging "add-ons".

Now I've encountered a situation where the implementation of an abstract (intended for inheritance) class encapsulating an array of structures does not lend itself to inherited services (such as sorting). Replacing an array of structures with an array of classes solves this problem, but creates another.... (see above).

And I'm not asking for any "direct access" and I'm not interested in any address arithmetic. Why are you scaring children for nothing? :) mql5 handle is not even close to a "raw" pointer. And if it (surreptitiously) is - so the real security hole is here, but not in the implementation of handles (pseudo-pointers) to any user data.

You want to build a complex system with abstract data (which already means a lot of internal metadata and relationships) and then hand it over to an uncontrolled OpenCL environment where it can be changed in a tricky way. That's why we allow only simple objects and arrays to operate freely without the ability to overwrite virtual tables.

You are actually indirectly asking for direct access via "freedom of abstract constructions". This is a topic we've explored well and covered architecturally for the sake of security. Classes in MQL5 live by their own rules with an emphasis on security. Other types won't have handles. Instead of handles for simple types and structures, you can use indexes in arrays.

The handles themselves in MQL5 are correct (growing from one), you can check.

[Vladimir Gomonov]

Renat:

1. you want to build a complex system with abstract data (which already means a lot of internal metadata and relationships), and then hand it over to an uncontrolled OpenCL environment where it can be changed in clever ways. this is why we allow only simple objects and arrays to operate freely without the ability to overwrite virtual tables.

2. you are actually indirectly asking for direct access via "freedom of abstract constructions". This topic is well researched by us and covered architecturally for the sake of security. Classes in MQL5 live by their own rules with an emphasis on security.

Handles in MQL5 are correct (growing from one), you can check it yourself.

I want to pass strictly clean data into the buffer without any metadata (handles). I don't need these metadata in the buffer. They get in the way there. And I won't be able to put them there either - CLBufferWrite() won't let them in. And this is correct.

2. I'm not actually asking for any direct access, I can use DLL for direct access (if I need it).

aPointer = memcpy(a,a);

will solve the problem of getting a raw pointer. Renat, try to get into the real problem. Don't make up anything that doesn't " actually" exist. All that actually exists - I have described it directly and without any subtleties in the request. As constructively as possible and with a full understanding of safety issues.

3. that's great.

--

You should not do dynamic creation and deletion of structures (new, delete) at all. Not even in any way. Then the security issues won't arise either. I understand what the problem is "actually" (to put it in your language). There is a problem of defining the real type for dynamic objects. For classes it is most likely solved by analyzing a pointer to VMT. However : no dynamic structures, no problem.

[Renat Fatkhullin]

Think about it, what is a "handle" in relation to an entity of any scale?

You can provide handles to objects that are reasonable in number (classes, files, etc.). But if you go into an array area of any size, any handle only has a chance of being a direct reference to a piece of memory. Otherwise the "handle -> memory" mapping table will eat up even more memory than the entity being referenced.

And based on the security provision, you can't have handles that are direct pointers to memory. That's why there are no handles for "any unbounded entity".

[Vladimir Gomonov]

Renat:

1. you can provide handles to objects that are reasonable in number (classes, files, etc.). But if you go into an array area of any size, any handle only has a chance of being a direct reference to a piece of memory. Otherwise the handle -> memory mapping table will eat up even more memory than the entity being referenced.

2. And based on the security clause, you can't have handles that are direct pointers to memory. That's why there are no handles for "any unbounded entity".

1. Constructive is a good thing. I've been thinking. The problem was raised exactly in connection with massive structures. For small structures copying time is short anyway. I think problems may arise here only because of unreasonableness of the user, but you can "foolishly" fill up memory anyway ( with indicator buffers, for example). I do not suppose that anyone would prefer working with handles on static structures without any particular need. And if you overflow memory, it's your own fault. Don't worry so much about folk nonsense, there's no way to prevent (and even predict) everything anyway. :)

2. No, no. No need for direct pointers, but it wouldn't hurt to have handles, even "for any unrestricted entity". The main thing is that there's no obligation to use them. Then there'll be enough memory for everyone. :)

[Mykola Demko]

MetaDriver:

I don't understand what you're barking about here. If you want handles, declare your structures as classes, that's all.

If you want direct access to memory, write a dll.

[Vladimir Gomonov]

Urain:

I don't understand what you're breaking your neck here.

1. you want handles, declare your structures as classes, that's all.

2. if you want to have direct access to memory, write a dll.

1. That's the point: it's problematic. I don't need to write an array of classes into the buffer (and it's impossible). I need to write structures there. And I want it quickly (without actually rewriting from classes into structures). And handles are needed for reordered access (for sorting, moreover, with different keys).

A replacement could be my own index table - but then I can't make a class encapsulating the work with an array of structures with the ability to inherit it, along with once-prescribed services (sorting and binary search).

2. I don't want it! !! !!! Stop making a case for me on the spot! :))

[Renat Fatkhullin]

No, we will not do such hendles. This is outright evil, for which we will be held accountable to the end.

[Vladimir Gomonov]

The ideal solution would be parameterizable classes

class MyClass<T>
{
  T MyArray[];
....
};

But I'm not even talking about that anymore, maybe I should?