Dynamically fill objects with data

New comment
 
Hey! I need to dynamically fill objects of complex types (class or struct) with data.

The class/struct may look like this:

class FooClass
{
   public:
      int Int;
      string Str;
      double Dbl;
      .
      .
      .
};
struct FooStruct
{
   int Int;
   string Str;
   double Dbl;
   .
   .
   .
};
The catch is that there is no constructor or method to fill them with data! They have just publicly accessible properties, which I have info about in form of strings (some rudimental reflection).
Of course I could set all properties individually like myFoo.Int = 123 etc. or instantiate with FooStruct myFoo = { 1, "string", 2.2 } (in case of a struct).
But that would mean a lot of hard coded stuff and a LOT of work.

So I hope there are smarter solutions to fill objects with data the dynamic way.

Maybe there is some special design pattern to accomplish this I haven't heard of?

I thought of working with memory functions like memcpy/HeapXXX to set/copy/manipulate memory directly.

My Idea: Get the address of the object of complex type, which would act as starting address. 
Then use the size of the properties' types from reflection as offset and put value in memory at starting address + offset.

Repeat this for all properties from reflection. And we would have our filled object.

With #import imported functions I can do memcpy(myFoo, myFoo, 0) to get the starting address, at least for structures.
For classes we need something like C++'s address-of operator (&) or alternatively reinterpret_cast operator. Unfortunately, both aren't supported in MQL5 (yet).
Maybe we could build our own reinterpret_cast with some bitwise operation...?

I also tried GetProcAddress() like this:

typedef void* (*Func)(FooClass*, FooClass*, int);
Func func_ptr = (Func)GetProcAddress(GetModuleHandleW(dll32), "memcpy");
Handle is ok, but GetProcAddress always returns NULL, even with int instead of void* or char& instead of FooClass*

However, if it is only possible with structs what I am trying to accomplish, that would be fine...

Moving on, I read about dynamic classloading, but I don't know exactly how to do it and if it solves my problem at all.

The steps could be: 
  1. Write classloader logic in C++.
  2. Write all classes/structs in MQL5 from Metatrader to header file at runtime with strings from reflection.
    We can "build" them as needed with constructor or some Fill(Data).
    The main file includes this header file.
  3. Compile files to dll with ShellExecute at runtime.
  4. Import dll into Metatrader and call function with GetProcAddress to get pointer to object(s) when needed. Function should return void*.
Only a few thoughts about this that come to my mind right now:
- we need to use GetProcAddress which hasn't worked for me yet. So we need to make it work
- we need to make a typecast from void* back into our custom type. Can MQL5 use void* returned by GetProcAddress?
- if MQL5 can't use void* from GetProcAddress, do we need to typecast at all or can we "just" do some magic mambo jambo in the memory?
- is it compatible with MQL's pointer which is not a pointer to memory, but a descriptor?
- do we need to utilize polymorphism?

I am pretty sure that the solution lies in one or in a combination of MQL5's rich features:
  • Import functions from DLLs
  • Polymorphism
  • template functions
  • basic typecasting
  • dynamic_cast operator
Only I haven't found it yet :)
I really need your help please! Any thoughts/tips/ideas are greatly appreciated, ANY really :)
 

Typical OOP auto created issue.

Add an Init() method and use it, what is "hardcoded and a lot of work" with that ?

If you need something more dynamic use a class and pointers instead of a struct.

 
Alain Verleyen:

Typical OOP auto created issue.

Add an Init() method and use it, what is "hardcoded and a lot of work" with that ?

If you need something more dynamic use a class and pointers instead of a struct.

Hey Alain! Thanks for your response!

You mean add Init() to each type?

Lets assume that I can't add any constructor, Init() or Fill() to them.

What I am actually trying to do is something like Newtonsoft's JsonConvert.DeserializeObject<T>(string) where you pass a complex Type T and a json string.
The function auto fills an instance of Type T and returns it. 

Of course it is C#, which has a powerful reflection library, properties have setter and getter and Newtonsoft can easily call PropertyInfo.SetValue().

I was just curious if something like this is possible in MQL w/ C++.

I would say yes, but it's def not easy (at least for me).

Lets see what we got:

template functions: check
info about properties type and name (reflection): check
logic to set values to properties (like setter in C# + PropertyInfo.SetValue()): Not yet :(

Ideas?

 
Domp:

Hey Alain! Thanks for your response!

You mean add Init() to each type?

Lets assume that I can't add any constructor, Init() or Fill() to them.

What I am actually trying to do is something like Newtonsoft's JsonConvert.DeserializeObject<T>(string) where you pass a complex Type T and a json string.
The function auto fills an instance of Type T and returns it. 

Of course it is C#, which has a powerful reflection library, properties have setter and getter and Newtonsoft can easily call PropertyInfo.SetValue().

I was just curious if something like this is possible in MQL w/ C++.

I would say yes, but it's def not easy (at least for me).

Lets see what we got:

template functions: check
info about properties type and name (reflection): check
logic to set values to properties (like setter in C# + PropertyInfo.SetValue()): Not yet :(

Ideas?

C++ and MQL don't have reflection. In order to implement something like you want you'd need to be able to iterate over the member variables. I'm just thinking outside of the box...but you could declare a subclass with a macro and pass the JSON string thru the constructors to be parsed by each class. This is obviously a crude implementation, but it works as an example. 

include:

class JsonItem
{
protected:
   string m_key;
   string m_value;
   string m_json;
public:
   void parse(string key,string to_parse)
   {
      m_json = to_parse;
      StringToLower(to_parse);
      StringToLower(key);
      m_key = key;
      int begin = StringFind(to_parse,key) + StringLen(key) + 1;
      int end = StringFind(to_parse,",",begin);
      if(end < 0)
         end = StringFind(to_parse,"}",begin);
      m_value = StringSubstr(m_json,begin,end-begin);
   }
};

#define __DEC(TYPE,CLASSNAME,OBJNAME) class CLASSNAME:public JsonItem{public:CLASSNAME(string json){this.parse(typename(this),json);}TYPE get(){return(TYPE)m_value;}}OBJNAME;

implemented:

#include "json_lib.mqh"
class MyJSON
{
public:
   __DEC(string,Name,name)
   __DEC(int,Age,age)
   __DEC(string,Email,email)
   __DEC(datetime,Updated,updated)
   MyJSON(string json):name(json),email(json),age(json),updated(json){}
};

void OnStart()
{
   string json = "{name:Emma,email:emma@mql.com,age:27,updated:2018.05.17}";
   MyJSON j(json);
   
   printf(
      "name:%s, email:%s, age:%d, updated:%s", 
      j.name.get(),
      j.email.get(),
      j.age.get(),
      TimeToString(j.updated.get(),TIME_DATE)   
   ); 
}

output:

name:Emma, email:emma@mql.com, age:27, updated:2018.05.17
New comment