Discussion of article "Cross-Platform Expert Advisor: The CExpertAdvisor and CExpertAdvisors Classes" - Expert Advisor - Articles, Library comments

Shephard Mukachi 2017.10.25 00:50 #1

Hi Enrico. Looks like you have completed your project. I was just looking at your last article and noticed what I believe might be an error:

bool CExpertAdvisorBase::Init(string symbol,int period,int magic,bool every_tick=true,bool one_trade_per_candle=true,bool position_reverse=true)
  {
   m_symbol_name=symbol;
   CSymbolInfo *instrument;
   if((instrument=new CSymbolInfo)==NULL)
      return false;
   if(symbol==NULL) symbol=Symbol();
   if(!instrument.Name(symbol))
      return false;
   instrument.Refresh();
   m_symbol_man.Add(instrument);
   m_symbol_man.SetPrimary(m_symbol_name);
   m_period=(ENUM_TIMEFRAMES)period;
   m_every_tick=every_tick;
   m_order_man.Magic(magic);
   m_position_reverse=position_reverse;
   m_one_trade_per_candle=one_trade_per_candle;
   CCandle *candle=new CCandle();
   candle.Init(instrument,m_period);
   m_candle_man.Add(candle);
   Magic(magic);
   return false;
  }

I have highlighted in orange above. I think the return statement should be a true instead of false. I am resetting my PC, picked up a really bad virus which ruined my day, so decided to do some reading. Your work is one of the best reads on the forum. You have outdone yourself with this project, well done.

My idea is similar to yours, and I have picked up some really great ideas from your work, thanks. When I am done, I will take some time to share it here.

Conditions based on the PipMaker v1 - Price Tukul Thanks To Mr.D,Gardito

Dewiie PaRt II PaRt 2017.10.25 09:09 #2

Shephard Mukachi:

Hi Enrico. Looks like you have completed your project. I was just looking at your last article and noticed what I believe might be an error:

I have highlighted in orange above. I think the return statement should be a true instead of false. I am resetting my PC, picked up a really bad virus which ruined my day, so decided to do some reading. Your work is one of the best reads on the forum. You have outdone yourself with this project, well done.

My idea is similar to yours, and I have picked up some really great ideas from your work, thanks. When I am done, I will take some time to share it here.

Enrico Lambino 2017.10.25 09:24 #3

Hi Shep,

Shephard Mukachi:

Hi Enrico. Looks like you have completed your project. I was just looking at your last article and noticed what I believe might be an error:

I have highlighted in orange above. I think the return statement should be a true instead of false. I am resetting my PC, picked up a really bad virus which ruined my day, so decided to do some reading. Your work is one of the best reads on the forum. You have outdone yourself with this project, well done.

My idea is similar to yours, and I have picked up some really great ideas from your work, thanks. When I am done, I will take some time to share it here.

Sorry to hear about what happened, but thank you for your feedback. Yes, you are right. I have not noticed it until now. I will correct it. The call to the Init() method on the examples is not checked, but part of my future plans to the library is to make the function and method calls more formal (just like in the standard library), as well as an EA template for easier coding. Later, I will upload the latest version of the library on the codebase, and make the repository public. Your further feedback and pull requests are welcome.

You're welcome and I look forward to seeing your library.

adventures of a newbie MQL5 Standard Library Panels Is possible add extern

Shephard Mukachi 2017.10.25 20:37 #4

Enrico Lambino:

Hi Shep,

Sorry to hear about what happened, but thank you for your feedback. Yes, you are right. I have not noticed it until now. I will correct it. The call to the Init() method on the examples is not checked, but part of my future plans to the library is to make the function and method calls more formal (just like in the standard library), as well as an EA template for easier coding. Later, I will upload the latest version of the library on the codebase, and make the repository public. Your further feedback and pull requests are welcome.

You're welcome and I look forward to seeing your library.

Hi Enrico. I am finally on the verge completing a new system installation. I struggled for hours last night as I could not get control of my system at all, but it's nearly done now. Using my tablet to browse to do some reading. Your library is really great work. Now that it's complete, I can see the full logic - wow!

So my idea is this.

- The EA creates a list of Robots (as in AI entity) based on a Magic, Symbol and Timeframe combination (using Watch Window)

- The AI entity then goes into a bag of tricks (the signals - MA, PSAR, etc) and determines the best signal or a combination of signals. Best here is based on an internal back-testing engine that allows it to compare between the signals.

- The AI then adds the signal/s into a list, and trades based on that, and relearns if it needs to.

- Each AI will have Stop, Order and Position objects which they manage individually, and access to an optimiser (GA) object for learning. I thought of using the Utility Theory for AI decision making (similar to the weighting used in MQL5 EA System), but I did not like the arbitrary nature applied, so I chose learning through virtual trading instead, giving the AI an almost real person like behaviour.

I had a tough time understanding how to implement some of the ideas, and reading articles like yours and a lot of other articles on the forum and books has lead me to a point where I'm about to complete a really cool project - so I think anyway.

One of the major issues that you mentioned is serialisation. Due to that a live order or position does not have all the attributes of the object that opened it, a system crash will put the AI's into an invalid state,leaving a lot of orphaned trades. I thought of using the order comment. So on sending an order, create a comment with Timeframe, SignalName,SignalSettings concatenated. Soon I realised it becomes unwieldy very quickly and just prone to implementation issues. So writing to file seems to be the only thing that makes sense.

Well, I let you know of my progress if you are interested. But your work is really great, thanks for your time into the project.

Shep

expert advisor - miscellaneous How do you trade Anyone here would like

Enrico Lambino 2017.10.26 01:12 #5

Hi Shep,

Shephard Mukachi:

Hi Enrico. I am finally on the verge completing a new system installation. I struggled for hours last night as I could not get control of my system at all, but it's nearly done now. Using my tablet to browse to do some reading. Your library is really great work. Now that it's complete, I can see the full logic - wow!

So my idea is this.

- The EA creates a list of Robots (as in AI entity) based on a Magic, Symbol and Timeframe combination (using Watch Window)

- The AI entity then goes into a bag of tricks (the signals - MA, PSAR, etc) and determines the best signal or a combination of signals. Best here is based on an internal back-testing engine that allows it to compare between the signals.

- The AI then adds the signal/s into a list, and trades based on that, and relearns if it needs to.

- Each AI will have Stop, Order and Position objects which they manage individually, and access to an optimiser (GA) object for learning. I thought of using the Utility Theory for AI decision making (similar to the weighting used in MQL5 EA System), but I did not like the arbitrary nature applied, so I chose learning through virtual trading instead, giving the AI an almost real person like behaviour.

I had a tough time understanding how to implement some of the ideas, and reading articles like yours and a lot of other articles on the forum and books has lead me to a point where I'm about to complete a really cool project - so I think anyway.

One of the major issues that you mentioned is serialisation. Due to that a live order or position does not have all the attributes of the object that opened it, a system crash will put the AI's into an invalid state,leaving a lot of orphaned trades. I thought of using the order comment. So on sending an order, create a comment with Timeframe, SignalName,SignalSettings concatenated. Soon I realised it becomes unwieldy very quickly and just prone to implementation issues. So writing to file seems to be the only thing that makes sense.

Well, I let you know of my progress if you are interested. But your work is really great, thanks for your time into the project.

Shep

That sounds like an interesting project. However, that looks like a very large project (probably larger and more challenging than this one). Allow me to share some of my thoughts which I believe might help you:

I once wrote a backtester script before. I managed to make it work as I wanted with virtual trades, but its performance would be slow if it were to be used in an actual EA (not to mention, a self-optimizing one!). You might come to better success in your work if you can somehow make the other CPU cores (or even GPU) do calculations as well (i.e. DLL or OpenCL).

The list of robots works, but it has a major flaw: they are not executed in parallel. The lag is fine if the number is just few to several, but if you have more, those at the end of the list would get processed last. And these would also have to compete with the resources the EA needs to perform its actual work.

Virtual trading can make nice models, but it does not capture the actual noise in the market (i.e. news). This I learned the hard way.

I share the same thoughts with you regarding the weighing system used in the CSignal class of the standard library. In my opinion, the system is very prone to overfitting if put on an optimizer. I have a market maker bot that uses a market scoring model (LMSR, but still working on improving inventory risk). I believe this to be a very good alternative, since the weight of the buy signals and sell signals (and even no signal) can be collectively taken into account, and the outputs are in terms of probabilities. And it's far less resource-intensive than deep learning.

Saving to file is probably the best option right now to save volatile data. Using GVs is another, but it can get even messier than using order comments. Also, some brokers insert other codes on order comments (e.g. binary options), so it is also not a very good method in my opinion. Another option is to save all the data on a dictionary object, so you will avoid having to write lines of code implementing a number of Load and Save methods, but you need to save to disk as well. But there is a major problem when the user wants to start a new session with an EA. This has to be addressed manually (either deleting the save file, or assigning a new save file name).

I broke a few rules in OOP while implementing the file classes in this library. I experienced some problem with the file handle losing its value (becoming INVALID_HANDLE) when passed across methods and functions (even with const qualifier). I am not sure if it is still the case now, but I am quite satisfied with the current solution. To be honest, the use of a CFile descendant on the library was not every elegant, but it works, and results to much less code.

All the best,

Enrico

What is the best Need Programming/General Idea Help: EA for close all

Shephard Mukachi 2017.10.26 03:48 #6

Thank you very much for your input. I welcome your ideas any time.

Yes you are right that it will be a large project, and hopefully not anymore bigger than yours, it's a lot of work and stressful at that too. However I love programming, and to think I am actually a Financial Analyst. I started programming as a necessity because I needed to automate some of the more complex financial models. I just have a weird ability to come up with crazy, bizarre ideas. If only I could code as we as you!

I am looking at Dictionaries, and Lists<T> at the moment. List<T> specifically as an easier means for sharing the lists of objects, but also for binding. And Dictionary for storage and serialisation. I have been experimenting with a few ideas so that I can make my code base lighter, but also have a system that is fast, light on maintenance and that is literally hands free once dropped on a Chart. Your point on the lists is really valuable so this idea my not be so ideal.

You are spot with regards to processing slag if I do not employ parallel executions - and even then its slow. This is one of the issues I discovered in my tests. The last objects in the list are way late, especially if the market is really volatile, the OnTick, it can be called several before even the first objects in the list are called. As you suggested, I looked at OpenCL as possible solution. I will have to invest in a custom made system.

Indeed virtual trading makes great models that can completely wipe your account in noisy markets. I watched in panic one day as my EA struggled to place and let alone close the trades during news. This is the cold harsh reality about trading, but we must try.

Yes I agree with you that the scoring model approach is better. The LMSR is definitely superior approach, especially considering that bullish and bearish moves are very different for each instrument. A probability weighted approach most definitely improves accuracy.

With the self-learning, yes it will be fairly slow. It will only be used at the beginning to set up the AI agents, to arm them with the appropriate signals, and then at given intervals, say end of day for 5Min agents and on a Friday for the 1Hr agents. Indeed this brings the point you made about virtual training that makes billions only to lose all your money in live trading. So it's quest for me to find the signals that are very simple and fast so that I do not lose too much speed. I want to the system to scalp, say for a few pips after spread and commission. This means that the trades have to be really high probability setups, like the hammer that is at lowest of last 20 bars or something like that.

Because I want to focus on scalping, I might not need to monitor the trades so much. The agents will just check the setup, then order with Stop and Takeprofit which will be stopped out or closed in profit. Again, still experimenting with this.

I still have a lot of work, experimenting with ideas around the signals. When I find something with really great results I'll share with you.

You did break a few rules, but isn't that the beauty of programming! It's amazing how you can stick to rules and get completely stuck and feel like you could climb the wall. And at times breaking the rules is what you need to progress. Your work is great, and for that I have a lot of respect for you. Enrico programming is hard work. It's nice when smooth, and a real pain when it's just not working according to plan.

Thanks very much for your input. I am extremely grateful, and will definitely keep you in the loop.

Thanks once again,

Shep

MT4 || Getting data Raw Ideas Taking Neural Networks to

Enrico Lambino 2017.10.26 11:11 #7