Serious Flaw in Backtester - No recourse
First of all I should state that this problem doesn't exist as long as your broker has permanently fixed spreads. If for any reason your spreads ever change, then this problem applies to you.
I was running the backtester and optimizer over some data using a strategy that uses no indicators. I would run it once, and it would report a profit ($55k on 1 fixed lot) over the timeframe April 5 to April 18. I would then run the same backtest again, and show a 33K loss. The only difference was *when* I pressed the start button. I checked for a large number of issues, tried it again and again. Sometimes the results were consistent and sometimes they were... completely different. The optimizer matched the backtester several times, and then they had nothing in common but I'd get the same results by repeatedly pressing the backtester.
Based on my initial disclaimer readers may have guessed the issue. The data in Market Watch is extrapolated to figure out what the spread should be for the entire history file. If you current Ask - Bid is 2 pips, the backtester will use a 2 pip spread going back to 1999. If the current Ask-Bid is 10 pips, that is what the backtester will use. So if you have a fixed spread, you're fine. If you don't, well you are not fine. I attempted to 'fix' this issue to go with the median spread my ECN has (1 pip +commission) but the data is stored in the ticks.raw and symbols.sel files. My weekend spread is 4 pips, a drastic difference for a small pip trading algorithm. I might be able to save copies of these files and use them in a disconnected mode in the future, but right now I don't even have a copy of a file I could use.
My preference for fixing this problem would be to put the spread in the history files to reflect changing spreads during news, off hours trading, etc. Many brokers change their spreads Wednesday afternoon just before NY close to prevent swap trade scalpers. This is also important for people who trade on ECN. Of course, this is nearly impossible in MT4.
My second proposed solution is to allow editing of the spread in the backtester. Some initialization variable might be set (MODE_SPREAD?) that we could override a 'normally safe' value. I thought about making this change myself, but memory is very protected and that would also violate MT4's license.
So much data is simply ignored by MT4, this very serious issue is yet another reason to build my own FIX+Python based trading platform (I have it logging in currently to ECNs). Some of the data I want to see but can't: ticks, spread, buy side vs sell side volume (which side of the market is paying the spread?), pending order sizes (ECN Level 2 data), market instead of broker volume, etc. I have a litany of aggravations with MT4, this just feels like beating a dead horse into glue and dog food. I estimate 2 man months to get the FIX+Python+MySQL trader running. I can do any part of it, but I can't do every part of it.
First of all I should state that this problem doesn't exist as long as your broker has permanently fixed spreads. If for any reason your spreads ever change, then this problem applies to you.
I was running the backtester and optimizer over some data using a strategy that uses no indicators. I would run it once, and it would report a profit ($55k on 1 fixed lot) over the timeframe April 5 to April 18. I would then run the same backtest again, and show a 33K loss. The only difference was *when* I pressed the start button. I checked for a large number of issues, tried it again and again. Sometimes the results were consistent and sometimes they were... completely different. The optimizer matched the backtester several times, and then they had nothing in common but I'd get the same results by repeatedly pressing the backtester.
Based on my initial disclaimer readers may have guessed the issue. The data in Market Watch is extrapolated to figure out what the spread should be for the entire history file. If you current Ask - Bid is 2 pips, the backtester will use a 2 pip spread going back to 1999. If the current Ask-Bid is 10 pips, that is what the backtester will use. So if you have a fixed spread, you're fine. If you don't, well you are not fine. I attempted to 'fix' this issue to go with the median spread my ECN has (1 pip +commission) but the data is stored in the ticks.raw and symbols.sel files. My weekend spread is 4 pips, a drastic difference for a small pip trading algorithm. I might be able to save copies of these files and use them in a disconnected mode in the future, but right now I don't even have a copy of a file I could use.
My preference for fixing this problem would be to put the spread in the history files to reflect changing spreads during news, off hours trading, etc. Many brokers change their spreads Wednesday afternoon just before NY close to prevent swap trade scalpers. This is also important for people who trade on ECN. Of course, this is nearly impossible in MT4.
My second proposed solution is to allow editing of the spread in the backtester. Some initialization variable might be set (MODE_SPREAD?) that we could override a 'normally safe' value. I thought about making this change myself, but memory is very protected and that would also violate MT4's license.
So much data is simply ignored by MT4, this very serious issue is yet another reason to build my own FIX+Python based trading platform (I have it logging in currently to ECNs). Some of the data I want to see but can't: ticks, spread, buy side vs sell side volume (which side of the market is paying the spread?), pending order sizes (ECN Level 2 data), market instead of broker volume, etc. I have a litany of aggravations with MT4, this just feels like beating a dead horse into glue and dog food. I estimate 2 man months to get the FIX+Python+MySQL trader running. I can do any part of it, but I can't do every part of it.
Sorry for double post, I don't think historical data includes spread data because every broker has different spreads. That is something that makes backtest not as accurate as it could be.
Sorry for double post, I don't think historical data includes spread data because every broker has different spreads. That is something that makes backtest not as accurate as it could be.
The History data is also broker specific. If you compare Alpari vs other brokers you'll see tons of spikes in different data feeds. The prices are by no means centralized. Each broker also has a different smoothing algorithm they apply to the data feed to reduce the number of ticks sent out.
First of all I should state that this problem doesn't exist as long as your broker has permanently fixed spreads. If for any reason your spreads ever change, then this problem applies to you.
I was running the backtester and optimizer over some data using a strategy that uses no indicators. I would run it once, and it would report a profit ($55k on 1 fixed lot) over the timeframe April 5 to April 18. I would then run the same backtest again, and show a 33K loss. The only difference was *when* I pressed the start button. I checked for a large number of issues, tried it again and again. Sometimes the results were consistent and sometimes they were... completely different. The optimizer matched the backtester several times, and then they had nothing in common but I'd get the same results by repeatedly pressing the backtester.
Based on my initial disclaimer readers may have guessed the issue. The data in Market Watch is extrapolated to figure out what the spread should be for the entire history file. If you current Ask - Bid is 2 pips, the backtester will use a 2 pip spread going back to 1999. If the current Ask-Bid is 10 pips, that is what the backtester will use. So if you have a fixed spread, you're fine. If you don't, well you are not fine. I attempted to 'fix' this issue to go with the median spread my ECN has (1 pip +commission) but the data is stored in the ticks.raw and symbols.sel files. My weekend spread is 4 pips, a drastic difference for a small pip trading algorithm. I might be able to save copies of these files and use them in a disconnected mode in the future, but right now I don't even have a copy of a file I could use.
My preference for fixing this problem would be to put the spread in the history files to reflect changing spreads during news, off hours trading, etc. Many brokers change their spreads Wednesday afternoon just before NY close to prevent swap trade scalpers. This is also important for people who trade on ECN. Of course, this is nearly impossible in MT4.
My second proposed solution is to allow editing of the spread in the backtester. Some initialization variable might be set (MODE_SPREAD?) that we could override a 'normally safe' value. I thought about making this change myself, but memory is very protected and that would also violate MT4's license.
So much data is simply ignored by MT4, this very serious issue is yet another reason to build my own FIX+Python based trading platform (I have it logging in currently to ECNs). Some of the data I want to see but can't: ticks, spread, buy side vs sell side volume (which side of the market is paying the spread?), pending order sizes (ECN Level 2 data), market instead of broker volume, etc. I have a litany of aggravations with MT4, this just feels like beating a dead horse into glue and dog food. I estimate 2 man months to get the FIX+Python+MySQL trader running. I can do any part of it, but I can't do every part of it.
Daraknor, I agree that not being able to change or fix the spread to what you want is very troublesome.
You can manually adjust the spread cost everytime you run the backtest, but if you are optimizing or running a large number of tests,
its VERY time consuming to have to recalculate spread cost for each scenario.
Metaquotes: If you would allow an override value for the spread, say, in the Currency Property window in the Tester, that would solve this problem. The person testing could enter their average spread cost, and so we dont have to manually recalculated spread, AND, so that spread doesn't keep changing as I am doing different backtesting.
I would dearly appreciate this feature in the next minor release of MT4.
I am not so knowledgeable about this matter. So i have to learn it. Thanks for the post.
Sorry for double post, I don't think historical data includes spread data because every broker has different spreads. That is something that makes backtest not as accurate as it could be.
Sorry for what? Just go back in and delete your posts.
And don't quote the whole thing so you can just make a one-liner reply. It is obvious your are replying to post 1 without quoting the whole <bleeping> thing again, and again.
First of all I should state that this problem doesn't exist as long as your broker has permanently fixed spreads. If for any reason your spreads ever change, then this problem applies to you.
I was running the backtester and optimizer over some data using a strategy that uses no indicators....
Hi there daraknor... I know you posted this a while ago, and I don't know if you are interested, but I've found a document online that relates to this issue and used it to make my own little "work around" to this problem. Now I can select any spread that I need to for MT backtesting and do it weekends, weekdays, whatever. It's a great little work around that allows consistency in back testing. I'll subscribe to this topic and see if there is any interest and if so I'll sell the solution to you!
(Just kidding - if there's interest I'll post my work around...) ;-)
Hi there daraknor... I know you posted this a while ago, and I don't know if you are interested, but I've found a document online that relates to this issue and used it to make my own little "work around" to this problem. Now I can select any spread that I need to for MT backtesting and do it weekends, weekdays, whatever. It's a great little work around that allows consistency in back testing. I'll subscribe to this topic and see if there is any interest and if so I'll sell the solution to you!
(Just kidding - if there's interest I'll post my work around...) ;-)
YES YES, there is interest, please, share with as.
Bucho
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
First of all I should state that this problem doesn't exist as long as your broker has permanently fixed spreads. If for any reason your spreads ever change, then this problem applies to you.
I was running the backtester and optimizer over some data using a strategy that uses no indicators. I would run it once, and it would report a profit ($55k on 1 fixed lot) over the timeframe April 5 to April 18. I would then run the same backtest again, and show a 33K loss. The only difference was *when* I pressed the start button. I checked for a large number of issues, tried it again and again. Sometimes the results were consistent and sometimes they were... completely different. The optimizer matched the backtester several times, and then they had nothing in common but I'd get the same results by repeatedly pressing the backtester.
Based on my initial disclaimer readers may have guessed the issue. The data in Market Watch is extrapolated to figure out what the spread should be for the entire history file. If you current Ask - Bid is 2 pips, the backtester will use a 2 pip spread going back to 1999. If the current Ask-Bid is 10 pips, that is what the backtester will use. So if you have a fixed spread, you're fine. If you don't, well you are not fine. I attempted to 'fix' this issue to go with the median spread my ECN has (1 pip +commission) but the data is stored in the ticks.raw and symbols.sel files. My weekend spread is 4 pips, a drastic difference for a small pip trading algorithm. I might be able to save copies of these files and use them in a disconnected mode in the future, but right now I don't even have a copy of a file I could use.
My preference for fixing this problem would be to put the spread in the history files to reflect changing spreads during news, off hours trading, etc. Many brokers change their spreads Wednesday afternoon just before NY close to prevent swap trade scalpers. This is also important for people who trade on ECN. Of course, this is nearly impossible in MT4.
My second proposed solution is to allow editing of the spread in the backtester. Some initialization variable might be set (MODE_SPREAD?) that we could override a 'normally safe' value. I thought about making this change myself, but memory is very protected and that would also violate MT4's license.
So much data is simply ignored by MT4, this very serious issue is yet another reason to build my own FIX+Python based trading platform (I have it logging in currently to ECNs). Some of the data I want to see but can't: ticks, spread, buy side vs sell side volume (which side of the market is paying the spread?), pending order sizes (ECN Level 2 data), market instead of broker volume, etc. I have a litany of aggravations with MT4, this just feels like beating a dead horse into glue and dog food. I estimate 2 man months to get the FIX+Python+MySQL trader running. I can do any part of it, but I can't do every part of it.