Walk Forward Analysis - is there an automated approach?
I have been all over the forum and read all the articles in Tester, but I can't seem to find much about how others may be achieving Walk Forward Analysis for their EA's with MT4.
I know there is a tool by Easy expert forex, but this tool unfortunately has limitations. Essentially it only seems to select the top (most profitable) pass and then move forward with that. As we know from optimization and curve fitting, this pass may not be the most appropriate to use. It may appear to be the most profitable, but run in Out of sample data it may not be, and in actual fact on of the less profitable pass results may be more stable in out of sample data. So you would need to rerun the process several times, making it a very manual effort.
From what I can see of other trading platforms, in their inbuilt WFA, they have the ability to run several result passes of the optimization through the WFA process (Out of sample data).
Have others been able to achieve this in an automated fashion?
I am hoping to get to the point where i can run WFA and then hopefully see how often I need to optimise my EA. (I read somewhere that using WFA may give an indication of this time period, though I am not sure how this is achieved -anyone?)
I think the closest example I have seen is via this article https://www.mql5.com/en/articles/1347
Is this how others are achieving this? Is there something that I have missed.
thanks
Simon
Yes that article is the closest thing you've got. You might look into post for easier ways of auto-mating the optimizer ... but thats about it. Everything else from there, you'll have to get creative.
hmmm with my coding skills :(
i am surprised that there isn't a solution - does that mean others are doing this manually? or not bothering at all?
I have been all over the forum and read all the articles in Tester, but I can't seem to find much about how others may be achieving Walk Forward Analysis for their EA's with MT4.
Hi Simon
I realise a few years have passed since you posted your question, so you probably have an alternative solution by now. However, you might still (and others might) find my reply useful.
I shared your frustrations with the tools that were available to perform full walk forward analysis on MT4. The very limited number of options all seemed to have the issues that you talk of. Because I am a software developer I decided to write my own application to meet the need I had.
I have been using this application now myself very successfully for over a year. I have made the tool available to students I have been coaching and it was one of them that suggested I should sell the product.
After some consideration I decided to do that. This has meant a huge amount of additional work on my part to make the product into a commercially viable offering but I feel it will really be worth it. I think that this is a tool suitable for any serious algorithmic trader who wants to step up to the next level of back testing and optimization.
In terms of specifics, the tool I have developed addresses the issues you raised about the existing product on the market. It gives you the option to use ‘Intelligent’ selection of parameter combinations, which effectively looks for high plateaus in the parameter space, rather than the highest spike, which means the resulting walk forward tests on out-of-sample data are likely to produce better results and increases the robustness of the system overall.
I am also in the process of adding the ability for the application’s ‘Intelligent Advisor’ module. This will give the user advice on what the optimal periods for each stage of the WFA should be and what the ideal ‘optimization to backtest ratio’ should be, taking into account a balance between statistical significance (having enough data to be statistically significant) and recency effect (having more of a bias to recent data to tune the trading system into recent market conditions).
My aim is to make the product (which is called ‘Walk Forward Pro’ by the way) deliver the very best walk forward analysis capability available on the market (not just the best for MetaTrader, but the best there is out there for any product). For this reason I really welcome suggestions from anyone about product enhancements. Anything that makes the application better, is a good thing in my view and I will put in the work to continually improve it. The development of this product, in conjunction with my own trading has become my passion.
Anyway, take a look see what you think <external link removed by moderator>
By the way, the product works with MT5 as well as MT4.
Martyn
in 2014 i do WFA,
tons of learning also in asirikuy about how robust is WFA method, trade on open bar only, etc2
but its an effort too much, i always WFA my EA for weeks or month. And an Open Bar EA surely had long stagnant Drawdown period.
As at the end , im coming to my own conclusion, continous WFA is also curve fitting it self. In a different way.
In WFA The Best Out of sample we pick, is the best that we choose to live trade.
So whats different than curve fitting??
The results in Live trade, the market always change beyond our expectation.
Even with continous development.
The last market behaviour is the least we could adapt with our optimization.
Even i know a friend works in a real Financial Institution, a 20 years trading experience in a hedge fund.
From what i see, the way he do to adapt market change is no different than curve fitting.
Im not saying WFA is a wrong method, i also still had WFA tools, but rarely using it now.
So, as a simple statistic approach, why dont we build a most robust system that all in sample data we could had?
Even it tooks hundreds hour for a single curency. (With TickData also now!)
The more sample we optimized is the more statisticly robust proven. No matter what the market behavior change, historicly some of it repeated itself.
Market Trade only know 4 behaviour : Trend, Reversal, Breakout, and Ranging. Only the volume volatility that changing.
Then i optimized all the historical data as robust at we can with as much data as i had.
And build porftolio with those 4 character inside to minimize stagnant period. And took every chance in every condition change.
Then picking up the best KPI from it optimization results.
Such as Profit factor, CAGR, etc. But Not get too shiny on the score itself, since those Profit Factor, CAGR, Payoff, etc always had flaws statisticly
So i pick with the most trades as posible first, that mean the more data, more sample and most market condition as we could had.
Run montecarlo, stagnancy period, drawdown % and period analysis with the risk that we could stomach.
And dont change the rules and system before that max risk happened. Wich mean market still in our portfolio prediction.
(So actually i do WFA also now, but my out of sample is a live trade. LOL. Kidding)
However this is my personal conclusion. Dont pick it if you dont walk the path yourself.
Im not bashing out Asirikuy with WFA approach, but i felt like im wasting times backthen sticking to others personal rules to build a systems.
But however, he build a great platform and different insight.
No matter what system we build, tick scalping, swing, to long trend trading, or even a Grid.
Whatever approach we choose, Even now im doing curve fitting to "latest market condition" approach. From the base parameters that already robust entire period as much i can have. And back using TDS Variable spreads ON.
As It IS the "TRUE real market condition". Not open bar.
At the end of the day, we trade to made money.
Not to be a smart academicly.
So for me the hints is as i mention above "Run the system with the max risk we could stomach with".
No matter we use WFA or Curve fitting the whole period method.
Managing Risk first, then profits follow.
Cheers,
My two cents
Once again : this is my personal conclusion. Dont pick it if you dont walk the path yourself.
Hi Rachmat,
it came across to your post while looking for an approach that could combine long-term in-sample test with the Walk Forward (indeed with Walk Forward Matrix) results.
Have you got new ideas or perspective about that?
thank you in advance for your time.
Hi Simon
I realise a few years have passed since you posted your question, so you probably have an alternative solution by now. However, you might still (and others might) find my reply useful.
I shared your frustrations with the tools that were available to perform full walk forward analysis on MT4. The very limited number of options all seemed to have the issues that you talk of. Because I am a software developer I decided to write my own application to meet the need I had.
I have been using this application now myself very successfully for over a year. I have made the tool available to students I have been coaching and it was one of them that suggested I should sell the product.
After some consideration I decided to do that. This has meant a huge amount of additional work on my part to make the product into a commercially viable offering but I feel it will really be worth it. I think that this is a tool suitable for any serious algorithmic trader who wants to step up to the next level of back testing and optimization.
In terms of specifics, the tool I have developed addresses the issues you raised about the existing product on the market. It gives you the option to use ‘Intelligent’ selection of parameter combinations, which effectively looks for high plateaus in the parameter space, rather than the highest spike, which means the resulting walk forward tests on out-of-sample data are likely to produce better results and increases the robustness of the system overall.
I am also in the process of adding the ability for the application’s ‘Intelligent Advisor’ module. This will give the user advice on what the optimal periods for each stage of the WFA should be and what the ideal ‘optimization to backtest ratio’ should be, taking into account a balance between statistical significance (having enough data to be statistically significant) and recency effect (having more of a bias to recent data to tune the trading system into recent market conditions).
My aim is to make the product (which is called ‘Walk Forward Pro’ by the way) deliver the very best walk forward analysis capability available on the market (not just the best for MetaTrader, but the best there is out there for any product). For this reason I really welcome suggestions from anyone about product enhancements. Anything that makes the application better, is a good thing in my view and I will put in the work to continually improve it. The development of this product, in conjunction with my own trading has become my passion.
Anyway, take a look see what you think <external link removed by moderator>
By the way, the product works with MT5 as well as MT4.
Martyn
Martyn, can I buy your walk forward matrix solution?
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
I have been all over the forum and read all the articles in Tester, but I can't seem to find much about how others may be achieving Walk Forward Analysis for their EA's with MT4.
I know there is a tool by Easy expert forex, but this tool unfortunately has limitations. Essentially it only seems to select the top (most profitable) pass and then move forward with that. As we know from optimization and curve fitting, this pass may not be the most appropriate to use. It may appear to be the most profitable, but run in Out of sample data it may not be, and in actual fact on of the less profitable pass results may be more stable in out of sample data. So you would need to rerun the process several times, making it a very manual effort.
From what I can see of other trading platforms, in their inbuilt WFA, they have the ability to run several result passes of the optimization through the WFA process (Out of sample data).
Have others been able to achieve this in an automated fashion?
I am hoping to get to the point where i can run WFA and then hopefully see how often I need to optimise my EA. (I read somewhere that using WFA may give an indication of this time period, though I am not sure how this is achieved -anyone?)
I think the closest example I have seen is via this article https://www.mql5.com/en/articles/1347
Is this how others are achieving this? Is there something that I have missed.
thanks
Simon