Different Optimization date ranges gives different results: but which to use?
For each external that has variations, run the optimizer on just that parameter for each period.
if the profit curve looks like ___/^^^^\___ then the parameter is stable in the zone.
if the profit curve looks like ___/\__ then it is unstable and any choice will be an over optimized result.
What is backtesting? | Invest Press - Forex Trading Research and News You Can Use
How about choosing the one which gives you more profit. Or the one which gives you less drawdown?
Hi Ubzen,
I've been doing that: striking the best balance between Profit and DD of the optimization results. I do a VERY course optimization and then a finer one in the best looking zone. Also I would think that picking a result where their is a group of decent results with some variations is a better bet that if it is only one EXACT result that is profitable, but are losers if it is the smallest bit off of it, it's not going to be a winner.
But as I said, the date range that drastically skews the results considerably. I kinda figured using a VERY long time period of ~ 5 years would give the best results, but it doesn't neccisarily pan out in forward testing.
Some are of the opinion to use only recent data as it is the way the market is currently, which makes some sense. But even their, the results are quite different between using 1, 2 or 3 months of data to optimize. Again the results can vary widely within these ranges.
Also I've just noticed that the historical data that I've been downloading from the broker that I am using: FXDD Malta, has gaps all through the data )< 8( Which of course basically makes it worse than useless: misleading = GIGO! I'm most definitely NOT impressed to say the least. I could use data from MetaQuotes, but how close is it to my broker? I'm currently running tests to find out.
So this basically leaves me at where I currently am: forward testing. Which is the most accurate but VERY slow. To the point of making this whole thing almost not worth doing.
Also I've got to get a handle on the 'Modeling Qulity' bar graph at the top of the results and the 'Mismatched Chart Errors' and what the implications are and how to improve on these. If it is only 25% accurate, how good is it? Worthless I would think or as I said even worse than that: misleading.
Thanks for the response.
For each external that has variations, run the optimizer on just that parameter for each period.
if the profit curve looks like ___/^^^^\___ then the parameter is stable in the zone.
if the profit curve looks like ___/\__ then it is unstable and any choice will be an over optimized result.
What is backtesting? | Invest Press - Forex Trading Research and News You Can Use
Hi WH,
I've been utilizing a dual optimization approach: Do a VERY broad optimization on a number of relevant variables and then pick the best looking zone and then do a finer, narrower and more specific optimization on that area.
If I'm interpreting you right, some DD is acceptable, so long as the overall trend is rising. But of course the DD has to be resonable, especially if the account has numerious pairs, or it wil get stoped out. This is the way I've been aproaching it. Nice to keep the DD below 10%, but I don't mind it running hiigher IF the return is signifcantly higher in a higher DD scenario.
I have some that look promising, but have a higher DD than I would like to use. I think that this is an acceptable solution if it is only for myself and one is aware of it and compensates for it and only have a limited number of pairs on the account or only even just the one chart. I don't mind having numerious accounts IF the net outcome is more profitable. I'm not sure how brokers would like it, but as long as they don't put the kibosh on it, so what? Their are LOTS of retail brokers. Also their is the option of dealing with numerious brokers for such (multiple) EAs/charts. I think that this is a wise approach anyway if one is doing very well, which alas isn't me yet by a long shot. The down side is it can be rather cumbersome and also overload the computer. But I've got a 50Mb/sec 'net connection and I can get lots of used laptops very cheap and run numerious ones if it comes to that.
Please let me know if we are on the same wave length here or it is something else that you have in mind.
Thanks for responding (< 8)
I just noticed your link. Thanks. I'm currently searching for such info. It can only help!
How is it going with you two these days? I hope that someone is making some money from this. Gotta say I'm getting discouraged. Shoulda jumped right into MQL4 programming right from the start. But that wasn't my initial game plan. I've bought so many commercial EAs and haven't found even one that is consistantly profitable. From my experiences with these, the only people that are likely making any money on them are those that make &/or market and sell them. But you know that they say about hind sight. Ah well, live and learn. Tis what life is all about anyway: MORE school (< 8)
Have either of you ventured forth into MQL5 yet ? If so, what do you think of it?
Some brokers are offering both versions now, but I don't see any stampede and hoards of brokers droping MQL4.
I don't know if you realize it but. The question you're asking is what Criterion during back-testing would guarantee forward testing results. I hate to be the bearer of bad news but the answer to that question is none. I've asked this question about 10 times and even to a mathematician on this forum. My question went something like "whats the statistically valid # of trades.....where one could depend on the # provided by the back-tester?" Numbers like profit factor, drawdown etc. To date, I haven't gotten a satisfactory answer ... other than it depends.
You as the trader have to set you own Criterion or choose a Criterion to accept on faith; on common sense, or if you're math person on formulas. Trading is not like other math based things which have removal effects or guarantee effects or static effects. I can make all the analogy on historical / technical / fundamental data before I enter a trade. The moment I enter the trade, there nothing in the world to stop the position from going against me.
One of the Criterion I look for today is something like Risk-Adjustment-On-Return. In lay-man terms it's the Profit-to-Drawdown ratios between systems or tests. Something like Profit/Drawdown but thats the at a glance view. The true Risk-Adjustment-On-Return consider looking at the Standard-Deviation from System-A or Test-A to Test-B. For example, 2 systems/tests might have the same Profit/Drawdown Ratios. But one Equity-Curve could swing up-and-down (Wildly) like Zig-Zan. While the other could be more (Smooth) like a Zipper. The smoother characteristic is more important.
Other things to consider is even tho two systems might have the same drawdown. The wilder system could be experiencing 50% Relative-Drawdown every day (Very Risky). While the Smoother system, while it may also have a 50% Relative-Drawdown at one-point or another, it's daily drawdowns are usually in the range of 10% (Safer). This dynamic is what I consider important. Tho neither system have any guarantee against losing 50% of the account one shows less Risk-Of-Ruin historically. However if you're bad lucky to hit the 50% drawdown on the calmer system in forward test then "you just hit 50% drawdown period". No one said anything about guarantees.
As far as time-frame, my Criterion is to use as much data as I can get my hands on. If the system isn't taking advantage of something unique within "EURUSD" for example; I'll borrow data from other currencies to see how it performs against other price-movements. Hell, give me the chance, I'll generate random prices and perform back-test on it.
Added: About making money, Yes I'm doing well in forex. I just have to keep trying to keep my over-nite billionaire greed in check.
About mql5: I really like it. I managed to back-test a multi-currency system without even using the Object Oriented Programming stuff. You can use your Linear Logics you learned in Mql4. However I found their forum boring. But that's more than likely where the party is moving to next :)..... Happy Trading.
Using CIL = 'Comments In Line' for my responses.
I don't know if you realize it but. The question you're asking is what Criterion during back-testing would guarantee forward testing results. I hate to be the bearer of bad news but the answer to that question is none. I've asked this question about 10 times and even to a mathematician on this forum. My question went something like "whats the statistically valid # of trades.....where one could depend on the # provided by the back-tester?" Numbers like profit factor, drawdown etc. To date, I haven't gotten a satisfactory answer ... other than it depends.
I understand this and tend to agree. So your not bursting my bubble as I'm aware of this and understand it. But actually my question is very specific and is not about an actual 'external variables' per say. I'm trying to get a handle on what time span is optimal to use for back testing to give the best and most relevant results. Though this still has the same elements and provisos that you speak of and I am aware of that. As with almost everything about this their are widely varying opinions. What I'm hoping to get is some insight and understanding that is based on those that have far more experience than I have by people such as yourself: 'Experience come before Expertise, EVEN in the dictionary'. It would be nice if it wasn't that way, but it's the nature of the beast and is the double edged sword that makes for the opportunities that draw us into this.
You as the trader have to set you own Criterion or choose a Criterion to accept on faith; on common sense, or if you're math person on formulas. Trading is not like other math based things which have removal effects or guarantee effects or static effects. I can make all the analogy on historical / technical / fundamental data before I enter a trade. The moment I enter the trade, there nothing in the world to stop the position from going against me.
One of the Criterion I look for today is something like Risk-Adjustment-On-Return. In lay-man terms it's the Profit-to-Drawdown ratios between systems or tests. Something like Profit/Drawdown but thats the at a glance view. The true Risk-Adjustment-On-Return consider looking at the Standard-Deviation from System-A or Test-A to Test-B. For example, 2 systems/tests might have the same Profit/Drawdown Ratios. But one Equity-Curve could swing up-and-down (Wildly) like Zig-Zan. While the other could be more (Smooth) like a Zipper. The smoother characteristic is more important.
Good tips and criteria that I will look into more thanks.
Other things to consider is even tho two systems might have the same drawdown. The wilder system could be experiencing 50% Relative-Drawdown every day (Very Risky). While the Smoother system, while it may also have a 50% Relative-Drawdown at one-point or another, it's daily drawdowns are usually in the range of 10% (Safer). This dynamic is what I consider important. Tho neither system have any guarantee against losing 50% of the account one shows less Risk-Of-Ruin historically. However if you're bad lucky to hit the 50% drawdown on the calmer system in forward test then "you just hit 50% drawdown period". No one said anything about guarantees.
Perhaps you saw my response about accepting a higher DD for a signifcantly higher net profit? I got a very good taste and example of this type of thing that you are speaking of here very recently. The top 2 most profitable 'very high 'risk/reward'' results of an optimization run. Both had about the same profit, # of trades, Profit Factor and almost exactly the same very high % DD. But what was VERY different is that the actual cash DD on one was about 10 TIMES higher than the other! This isn't something that I haven't give much priority to in deciding on what results to try out and run with. But in this case it was so dramatic and obviously was a VERY relevant factor that it was a 'no brainer'. A good lesson learned and something that I will watch out for more in the future. I'm also starting to use spread sheets more for more in depth and detailed analysis and different scenarios. Nothing like checking on and finding out what is really working and what isn't rather than just keep blindly stumbling along with yet another approach cluelessly. I've seen articles in here and other places that address this and talk about incorporating it into the EA itself so that it is constantly correcting, refining and improving itself and 'learning' with feed back loops. I've bought a number of commercial EAs that supposedly had this in them. But I could see no evidence of it: they were losers from the start and just kept on losing at about the same rate or even got worse. Ah for a Quantum computer (< 8) Though it would likely destroy such markets as this. But then again, maybe not. It would very likely shift the ratio of winners to losers based quite a bit based on those that have one and those that don't. (How about those 'faster than light' Neutrinos eh? They are quite the 'particle/wave' to start with! Then their is 'Quantum Entanglement' that (apparently) IS instantaneous!) Their are some memory products out now that utilize Quantum spin orientation. Each 'gate' could potentially have dozens of different states. Now that WOULD be a 'Quantum leap' in computing power! 8GB of RAM with each one having 32 states instead of 2. Do the math. WOW! I know Op's, I'm off topic here: but no one is forced to read all of this. A little variety adds to the Spice of Life (< 8)
As far as time-frame, my Criterion is to use as much data as I can get my hands on. If the system isn't taking advantage of something unique within "EURUSD" for example; I'll borrow data from other currencies to see how it performs against other price-movements. Hell, give me the chance, I'll generate random prices and perform back-test on it.
LoL. That has been my take on it as well: the more data over longer periods of time the better. But finding out that my brokers back data is full of gaps in the data totally negates this: GIGO )< 8( Also hearing from others that say they tend to use only the most recent data as it better reflects current market conditions. I'm not sure how valid this really is, so I'm now testing these premises and comparing them. Though alas and not surprisingly, when doing short term optimizations, the variance between 1, 2 or 3 months back testing can be very dramatic. As first glance an argument against this approach. But really, I don't know, so more testing on different date ranges with much better and more complete, higher quality and more accurate back data. This is already challenging enough without starting out with garbage and ending up with the same; NO more GIGO for me thanks! LoL (< 8) Also I want to start paying more attention to and learning about both positive and negative 'complimentary pairs'. Not the proper term for them, but I think you will know what I mean.
Added: About making money, Yes I'm doing well in forex.
Glad to hear it Ubzen: I'm glad somebody is! But I'm also happy to say that you are not getting any of my money as currently I'm almost exclusively dong 'paper trading' and testing. Can't make any money this way, but then can't lose any either. I've already proved that I am quiet adept at losing it so I don't and won't be doing any live trading until I can see that I can be consistent and managing my trading well enough that I can be profitable at it for a reasonable amount of time. I did take a VERY expensive and extensive one on one tutoring regime with a proven, successful trader half my age with a profitable system. But in the end it was a system that I could not utilize for health reasons. Something that I think that they should of made clear in the sales pitch )< 8( Still, the knowledge is relevant.
I just have to keep trying to keep my over-nite billionaire greed in check.
If you're a consistent winner Ubzen and want to 'leverage' it extensively, start selling your signals. Their are a number of well established ones with extensive clientèle and you can multiply your profits quite quickly with no further risk and very little added time input. I've even seen software for sale to set up your own server and service. These are all things I'm very interested in and will be doing once I'm in a position to do so. Though obiously a Live account is going to have more credibility, you can even use Demo accounts and document your verified results on sites like That way if you have more systems or pairs than you can currently carry Live, you can still have them bringing in profits for you. The types of 'problems' that businesses like to have (< 8)
One thing that I can say that I have learned and realize is that traders that are successful over the long haul are VERY conservative! With all the super hype of 'get rich quick' EAs on the wWw, it's very easy to get sucked into it. I used to trade at 400 to 1. Now I'm using 50 or even 25 to 1 and I would go even lower if I could get it on a demo account. One most definitely need self discipline in this. I say that if one has the gambling bug at all like Martingale did, to turn around and run away from this as fast as you can and DON't LOOK BACK! If one is going to use a Martingale strategy, they better have VERY deep pockets and only have a very small portion of their capital in play at any one time. The latter holds true about this market anyway, regardless of what strategy one uses.
Mind if I ask what approaches and methods that you tend to use? Currently I tend to day trading, but I'm more interested in and want to utilize swing trading. it's also much more realistic for me with my health limitations. but need to build up a decent account balance to allow for larger SL than in day trading.
Can't say I've had much success with scalping. Regardless of what algorithm it is.
Have you ever checked into News trading? Their is much to be said for it.
Recently I've been into Gold. It's an excellent pair as it is very active and generally has lots of movement almost constantly. Again: a double edged sword. But one can't make much profit on stagnant pairs. Add Volumes to a Gold chart and you will be amazed at how regular and cyclic it is.
Slip a query in here: most pairs are pegged to the USD. But when they say that the USD is up or down in the news and give the figures on it, what is that relative to ? Gold perhaps? As unlike Canada, I believe that the USD is still (somewhat?) pegged to Gold.
About mql5: I really like it. I managed to back-test a multi-currency system without even using the Object Oriented Programming stuff. You can use your Linear Logics you learned in Mql4.
I've heard that they have an Auto EA generator which attracts me. I bought a relatively mature one for MQL4 but it wouldn't run on my Vista 32 back up computer. I've finally got my 64 bit, 6 core Windows 7 up and running and I'm hoping that it will run on it. What programming skills I've learned can't be anything but an asset and I imagine that I'll end up doing manual fine tuning and some detailed programming anyway. I'm very attracted to the idea of 'modular programming' where I have numerous specialized (sub) routines/ modules for specific tasks that I use over and over again and expand on and modify where needs be. But I want to focus on the algorithms and approaches more than the mechanics of programming. Got my 'programming bug' out of my system decades ago. Though I did quite extensive studies no that long ago in programming for computerized machining which I aced: 98%, 99%, 99% and 100%. I found errors in the very mature, in depth courses that none of the previous students or the developer who is a 'walking encyclopaedia' of it hadn't found. But alas I was unable to make a decent living at it.
However I found their forum boring. But that's more than likely where the party is moving to next :)..... Happy Trading.
It's still fairly new yet and as we know MQL4 is still dominant by a wide measure. But it will get more and more of the market share as time goes on and the forum will grow with it. I'm glad to see a 'market place' being incorporated into it. An excellent idea that is a 'win - win' situation for EVERYONE. It would be good to see some of the very experienced and successful programmers and traders such as yourselves that freely contribute and help others out so much get paid for at least some of your substantial contributions.
I look forward to (hopefully) SOMEDAY being a 'Happy (successful) Trader' thanks. (< 8)
I REALLY hope that in the MQL5 optimizer that they incorporate the ability to pause/stop and save it and then return to it and resume it later. Back testing can run into weeks or even months of computing time. Which makes it almost certain for most of us that they will never get completed. I'm aware of the measures to reduce these times. But their are times where such comprehensive Optimization runs are valid and valuable.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Understandably, utilizing different Optimization date ranges gives different results. But how to figure out which results to use? Each different Optimization set gives different back test results. Again: understandable.
So one takes one set, lets say from very long range ~ 5+ years and runs it over past 3 ~ 6 months; which is reasonable, then back tests gives very different results.
Optimize for 3 ~ 6 months and back testing over sames range or even one month gives yet again different results.
Obviously one wants the settings that are going to work best in forward testing & live trading. So other than forward testing, which I do extensively but is VERY slow, how does one know what period to optimize and back test for to get relevant and accurate settings going forward?
Thanks for any and all assistance, thoughts, ideas and input.