Discussion of article "Experiments with neural networks (Part 3): Practical application" - page 2

 
CapeCoddah #:

Roman,

One possible reason the EA doesn't work in the Strategy tester is that the "File Sandbox" location changes when optimizing into an Agernt.  I just ran into that issue and am trying to fix it.  I haven't looked at your latest code yet. My suggestion is to us WinFile.mql and see the discussion on

how to read an csv-file into a string with the Windows-Functions?

which offers several upgrades to WinFile.  If you are trying to read a CSV file and obtain every line, I suggest using the "ReadWholeFile' function and doing the parsing internally instead of relying on "ReadLineFromFile" which cause a system crash. 

Another problem could be the revised Perceptrons are only selecting one trade at the beginning of the run and closing it when the optimizing run ends.  In looking at my Optimization report, I was surprised to find that there were several hundred passes that resuled in only one trade but had a very good profit for the optimization period e.g. the trade was put on at the start and closed when the optimization run ended.

Thanks for the info. I will check everything myself.

 
CapeCoddah #:

Roman,

One possible reason the EA doesn't work in the Strategy tester is that the "File Sandbox" location changes when optimizing into an Agernt.  I just ran into that issue and am trying to fix it.  I haven't looked at your latest code yet. My suggestion is to us WinFile.mql and see the discussion on

how to read an csv-file into a string with the Windows-Functions?

which offers several upgrades to WinFile.  If you are trying to read a CSV file and obtain every line, I suggest using the "ReadWholeFile' function and doing the parsing internally instead of relying on "ReadLineFromFile" which cause a system crash. 

Another problem could be the revised Perceptrons are only selecting one trade at the beginning of the run and closing it when the optimizing run ends.  In looking at my Optimization report, I was surprised to find that there were several hundred passes that resuled in only one trade but had a very good profit for the optimization period e.g. the trade was put on at the start and closed when the optimization run ended.

Hi.

Thanks for the info. I will check everything myself.

 
Roman Poshtar #:

Thanks for the feedback. Which EA does not work? If it works in the strategy tester, then it should work in real life.

EA 1 perceptron 4 angle SL TP
 

Hi Roman,

I just finished reading your last article thoroughly.  I am not surprised by the 443 DNN Angle results; they mirror mine.  I suspect that the problem arises in the close out decision processing although I have not examined it in depth.  I will be examining your new EAs in detail shortly. 

In the meantime, here is my completed CSV Reformatter with the associated Winfile for you. You may be able to use its concepts to automate part of your EA by allowing you to read in the weight values directly without performing the file insert process.  It is designed to read the saved CSV version of an Optimization Report while removing passes that have a low frequency of trades or produce losing result.  The reformatted CSV file will be read directly into my EA to either set the DNN weights or to select the most effective values in a optimization test. 

It is important to remember that the Excel file of the Optimization run must reformat the Equity and Profit columns to NUMBERS without a comma 1000's separator prior to running the reformatter   Path and file names that contain spaces MUST be enclosed with a triple double quotes, """, not just a single quotes to pass the spaces correctly to Windows.  Also directory separators \ must be entered as \\ to eliminate the escape processing in the compiler.

Hope this helps,

Cheers

 
CapeCoddah #:

Hi Roman,

I just finished reading your last article thoroughly.  I am not surprised by the 443 DNN Angle results; they mirror mine.  I suspect that the problem arises in the close out decision processing although I have not examined it in depth.  I will be examining your new EAs in detail shortly. 

In the meantime, here is my completed CSV Reformatter with the associated Winfile for you. You may be able to use its concepts to automate part of your EA by allowing you to read in the weight values directly without performing the file insert process.  It is designed to read the saved CSV version of an Optimization Report while removing passes that have a low frequency of trades or produce losing result.  The reformatted CSV file will be read directly into my EA to either set the DNN weights or to select the most effective values in a optimization test. 

It is important to remember that the Excel file of the Optimization run must reformat the Equity and Profit columns to NUMBERS without a comma 1000's separator prior to running the reformatter   Path and file names that contain spaces MUST be enclosed with a triple double quotes, """, not just a single quotes to pass the spaces correctly to Windows.  Also directory separators \ must be entered as \\ to eliminate the escape processing in the compiler.

Hope this helps,

Cheers

Thank you. I will definitely look.

 

Roman,

I am just beginning to evaluate your current work in detail.  Here is a chart of comparisons that I have run.

Time Frame Original DNN 1 - 1N SL 1-1SL 2-2 No SL 2-2 SL 3-3No SL 3-3SL 2-2 No SL 2-2 SL
Optimized From 1/1/21 - 1/1/23 12/9/21 - 12/9/22 1/1/23 - 3/10/23 12/9/21 - 12/9/22 1/1/23 - 3/10/23
h1 -1070 2762 7700 3870 -874 4320 638 -627
H1 Modified TP to 120:   5381 39%
h4 2,735 1394.00 237.00 -992 -1120 -993


It demonstrates clearly that 8 Preceptron  EAs are vastly superior to a 4443 NDD Original model.  In running these tests I noticed a small oversight in MQ5 BackTest tab.  It shows results in thousands with a space between the third and fourth digits which was their attempt to eliminate commas.  However, the space misled me to think it was the number of trades.

I am interested in modifying the Preceptrons to include fewer or more than 8 nodes.  Can you explain the schema used to generate 5, 6, 7, 9 etc node Preceptrons?  Alternatively, can you cite any references  that explain its structure?  In looking at your 2 Preceptron EAs, it seems that making a set of Preceptron Classes and parameterize the input might be beneficial.  As then you could instantiate multiple versions of identical nodes to be used for different purposes.  I think I will be trying this approach although I'm sure it will be slower than your code.

Stay Safe,

CapeCoddah

 
CapeCoddah #:

Roman,

I am just beginning to evaluate your current work in detail.  Here is a chart of comparisons that I have run.

Time Frame Original DNN 1 - 1N SL 1-1SL 2-2 No SL 2-2 SL 3-3No SL 3-3SL 2-2 No SL 2-2 SL
Optimized From 1/1/21 - 1/1/23 12/9/21 - 12/9/22 1/1/23 - 3/10/23 12/9/21 - 12/9/22 1/1/23 - 3/10/23
h1 -1070 2762 7700 3870 -874 4320 638 -627
H1 Modified TP to 120:   5381 39%
h4 2,735 1394.00 237.00 -992 -1120 -993


It demonstrates clearly that 8 Preceptron  EAs are vastly superior to a 4443 NDD Original model.  In running these tests I noticed a small oversight in MQ5 BackTest tab.  It shows results in thousands with a space between the third and fourth digits which was their attempt to eliminate commas.  However, the space misled me to think it was the number of trades.

I am interested in modifying the Preceptrons to include fewer or more than 8 nodes.  Can you explain the schema used to generate 5, 6, 7, 9 etc node Preceptrons?  Alternatively, can you cite any references  that explain its structure?  In looking at your 2 Preceptron EAs, it seems that making a set of Preceptron Classes and parameterize the input might be beneficial.  As then you could instantiate multiple versions of identical nodes to be used for different purposes.  I think I will be trying this approach although I'm sure it will be slower than your code.

Stay Safe,

CapeCoddah

Hi. Send me a private message. I am now recruiting a development team. If you are ready to work hard join. Participation is paid.

 
Roman Poshtar #:

Hi. Send me a private message. I am now recruiting a development team. If you are ready to work hard join. Participation is paid.

Not really interested, retired 20 years ago and programing is now a part time hobby.  Thanks for the offer & I don't know how tos send a private message.

 
CapeCoddah #:

Not really interested, retired 20 years ago and programing is now a part time hobby.  Thanks for the offer & I don't know how tos send a private message.

Please, if you change your mind, you are welcome.

 

Hi Roman,

I am focusing on your 4 Perceptron TP/SL models.  In doing Visualize runs in the tester, I am noticing some significant order processing problems that are causing large draw downs, most notably around 2022 07/05 where there is a $1,350 draw down see Bad Trades attachment

This appears to be caused by order 3534 which lacks both a TP & SL and is highlighted in Light green.  In some cases the the highlight is in the Rose color indicating that the Price identified is outside the trading range.   The comments identify it as "tp104740" instead of "Perceptron EN_xx and the volume is ).62/0.62.   This seems to indicate that there was incomplete processing of the order setup.

This problem repeats itself each time the Perceptron row for loop resets,  for(int i=0; i<=(ArraySize(EURUSD)/6)-2; i++){ .  I tested a version by decreasing the upper limit by one and the errors persist. 
BTW you should change the ArraySize to ArrayRange(EURUSD,0) ad drop the calculations.

The problem also manifests itself each time the signal switches from buy to sell or vice versa.

The problem could be be caused by an initialization problem caused either at the start of the loop or at the end or is this a Netting problem and the buy/sell functions should be moved outside of the for loop?

In reviewing all of the trades with a zero SL, I noticed that nearly all have a date time which   is seconds off of 00.  Thinking that your IsNewBar was off, I substituted my NewBar and obtained identical results. Consequentially it seems that anytime there is no trade activity in the first second of a new bar, the error may occur.  This does not bode well for using this concept for other currency pairs that do not trade as frequently as EURUSD.

Thus, I have a lot of potential problems but no good concept on how to proceed as I am in the beginning of converting from MT4 to MT5 and do not thoroughly understand the order processing details of MT5.  Can you identify and correct the problem??

Thanks CapeCoddah

BTW Your concept of using 10 of the first 100 Perceptron rows from the optimization run is brilliant.  I certainly increases the efficiency of the EA.

Files:
Bad_Trades.png  84 kb