Accurate day-level historical FX data for major pairs for back-testing in Excel

 

Hi team,

I am an fx noob. Sorry if this has been discussed to death already but I have searched for hours with limited results.

I am trying to evaluate if mathematical methods can realistically be used for probable consistent gain in fx speculation. I am writing a program in Excel/VBA to try to simulate all live-trading conditions and test various strategies using historical day-level data for the major pairs.

I am struggling to figure out what accurate data, as would be used in live trading, looks like and where to find high-quality historical samples of it.

I have accessed free historical data from a few sources and it is all very different leading to very different results in my tests.

I started out using data exported from MT4 using a FOREX.COM practice account. After getting some reasonable results with this I wanted to confirm the data integrity by using data from a different source. I used data from MT4 again but using an FXDD practice account this time.

The data from FXDD had 2 day gaps for weekends instead of 1 day gaps (as in the FOREX.COM data). My results were radically different.

Please help. Why are the data formatted differently? Are these real historical data? Are these the data that are given in live accounts? What is the most reputable source for quality data for back-testing? Can you advise me of any other important issues to consider in evaluating data quality for back-testing?

Many thanks!

 

the integrity the datasource is heavy discussed. From what i took from this discussions:

  • As higher the timeframes, as higher the quality
  • Intra M1 data is only simulated.
  • As larger your TP/SL as more realistic are the results
  • Since you are simulating in Excel, a common mistake there is to forget to include the Speard in the calculations
  • Different gaps on weekends are common, it is broker dependent if they offer a Sunday Candle.
  • As best source for realistic data mostly the ducascopy datafeed is recommend. on google you find tutorials how to import that datafeed into MetaTrader, i don't know how you import them in excel but should not be that difficult.
//z

 

Thanks Z,

I downloaded D1 historical data for the major pairs from Dukascopy. I looked closely at the data and there are a lot of inconsistencies and anomalies in it. There are many days that are missing, sometimes several days in a row. Saturdays/Sundays are included/excluded inconsistently throughout the files. There are data for Christmas day and New Year ’s Day for some years and not others.

So I went back to the FOREX.COM and the FXDD.COM files to check them. They have similar issues. So I went to the USA reserve bank website and got the data from there. This data has similar issues too.

Doesn’t the FX market remain open over public holidays? Why is there missing data in all files that I look at?

Reason: