Downloading historical data?
I would like to get IG Markets data (and CMC Markets data) and would like to hear how you go about getting this data, which is round the clock?
When using the History Center at IG's MT4 platform to download historical data there seems to be restrictions on how far back you can download data from. This seems to be the case for all MT4 platforms
Reason I would like this is because IG for instance have price data around the clock (albeit they make up their own prices during night on indicies etc),
Does anybody have a good source to get IGs (or CMC markets price data), or any other CFD-firm that have round the clock price data (minus settlement period) and tips on how to download it? Would like it in a csv file preferbly and only need OHLC and timestamp and price data on say 5min, 10min, 15min, 30min, 1h, 4h bars , no volume.
I think you are misinformed or under a misconception! Most markets are 24/5 and not "round-the-clock" as you put it. Only a very few number of markets, like CryptoCurrencies are 24/7.
For most markets (Forex, most CFDs, etc.), they close during the week-end so the Dukascopy data is not missing anything. You probably just don't know how to properly setup TickStory for the correct Time-zone used by your broker.
Another problem, is that TickStory Lite is a very old version with several bugs and is not at all recommended for use with the most recent builds of MT4.
I think you are misinformed or under a misconception! Most markets are 24/5 and not "round-the-clock" as you put it. Only a very few number of markets, like CryptoCurrencies are 24/7.
For most markets (Forex, most CFDs, etc.), they close during the week-end so the Dukascopy data is not missing anything. You probably just don't know how to properly setup TickStory for the correct Time-zone used by your broker.
Another problem, is that TickStory Lite is a very old version with several bugs and is not at all recommended for use with the most recent builds of MT4.
Hi, thanks for the answer. I should clarify what I'm asking for.
I know markets close for the weekend. When I say round the clock I thought eveybody understood I meant Sun night - Friday evening.
Difference between DukasCopy price data and for instance IG Markets price data on say DAX30 is that Dukascopy have pricedata from 08:00.22:00, i.e. when the underlying is actually traded on an exchange, (I know forex is different and have round the clock pricedata but for indicies it's usually a different story).
IG Markets sets their own prices on DAX30 between 22:00-08:00 thus keeping the option open of trading this instrument when there is actually isn't any underlying being traded on any exchanges as they are closed. Between 08:00-22:00 roughly IG Markets gives you the actual underlying price of the asset in question with a spread of their liking.
I've built my own backtesting algorithms in C# and am after price data from IG Markets or a similar CFD-firm (who many actaully use IG Markets prices for their own assets prices when actual underlying market is closed, know this as I've been working in the dealing desk of other CFD-firms) so that I can run my algorithms on price data that ranges round the clock (for the 5 trading days of course).
Tickstory Lite (yes, I know how to set up time-zone and how to export to csv which I then convert to a json-object and make changes to before I use the data in my sql-database for my calculations) is the best option I've come across so far. Now I'm looking for another option as it's not convenient to download via MT4's history center as there is so little data available and using Tickstory Lite only gives me price data on indicies for DAX30 between 08:00-22:00.
Anybody have any other sources of downloading data from CFD-firm that offer price data outside exchanges trading hours? For weekdays of course.
Johan Strandberg: Hi, thanks for the answer. I should clarify what I'm asking for.
I know markets close for the weekend. When I say round the clock I thought eveybody understood I meant Sun night - Friday evening.
Difference between DukasCopy price data and for instance IG Markets price data on say DAX30 is that Dukascopy have pricedata from 08:00.22:00, i.e. when the underlying is actually traded on an exchange, (I know forex is different and have round the clock pricedata but for indicies it's usually a different story).
IG Markets sets their own prices on DAX30 between 22:00-08:00 thus keeping the option open of trading this instrument when there is actually isn't any underlying being traded on any exchanges as they are closed. Between 08:00-22:00 roughly IG Markets gives you the actual underlying price of the asset in question with a spread of their liking.
I've built my own backtesting algorithms in C# and am after price data from IG Markets or a similar CFD-firm (who many actaully use IG Markets prices for their own assets prices when actual underlying market is closed, know this as I've been working in the dealing desk of other CFD-firms) so that I can run my algorithms on price data that ranges round the clock (for the 5 trading days of course).
Tickstory Lite (yes, I know how to set up time-zone and how to export to csv which I then convert to a json-object and make changes to before I use the data in my sql-database for my calculations) is the best option I've come across so far. Now I'm looking for another option as it's not convenient to download via MT4's history center as there is so little data available and using Tickstory Lite only gives me price data on indicies for DAX30 between 08:00-22:00.
Anybody have any other sources of downloading data from CFD-firm that offer price data outside exchanges trading hours? For weekdays of course.
Yes, IG Markets is a "very special" broker that likes doing things differently. They were my first broker, but only for a short while until I saw the light and moved on!
In essence, if you want to back-test it, you will not find any other source of data for that particular time period, namely 22:00-08:00 of DAX30, because it is specific to that broker and only they can provided you with data for it. In other words, they are a Dealing Desk, and you are trading them directly and it can no longer be qualified as a CFD during those hours.
If you really want to develop a strategy for DAX30 that truely is independent of the Broker and is truely indexed against the underlying instrument, then restrict it to working only during the official trade hours and and use only real data from the underlying instrument.
I hope, you understand my subtext here!
As for the data, besides Dukascopy, you can also use MT5 with a broker that offers the DAX30, and then export it and convert it into whatever format you require.
Anybody have any other sources of downloading data from CFD-firm that offer price data outside exchanges trading hours? For weekdays of course.
Why are you doing that ? If you now how a dealing desk is working, it really surprises me you still want to trade "outside" exchange trading hours.
I am missing something ?
Why are you doing that ? If you now how a dealing desk is working, it really surprises me you still want to trade "outside" exchange trading hours.
I am missing something ?
Hi Alain,
I build algorithms for some retail small time clients, they mainly us IG and want to test my algorithms on round the clock price data like IG provides although I favor building algos that don't trade during the night when there is no real underlying being traded there are others who still want this information.
Yes, IG Markets is a "very special" broker that likes doing things differently. They were my first broker, but only for a short while until I saw the light and moved on!
In essence, if you want to back-test it, you will not find any other source of data for that particular time period, namely 22:00-08:00 of DAX30, because it is specific to that broker and only they can provided you with data for it. In other words, they are a Dealing Desk, and you are trading them directly and it can no longer be qualified as a CFD during those hours.
If you really want to develop a strategy for DAX30 that truely is independent of the Broker and is truely indexed against the underlying instrument, then restrict it to working only during the official trade hours and and use only real data from the underlying instrument.
I hope, you understand my subtext here!
As for the data, besides Dukascopy, you can also use MT5 with a broker that offers the DAX30, and then export it and convert it into whatever format you require.
You are right in the way that it's not optimal to trade with IG but not sure how to work around the problem of if build for instance an algo that trades SMA based on x number of bar charts based on price data from a broker that supplies price data between 08:00-22:00 then the algo will behave very differently when being used on IG's MT4.
- People want to see the SMA-lines in a chart in live conditions and feel that their algo is trading thereafter, this will be very different if price data for the algo is different from the broker data.
- IG and CMC(No MT4 yet) are the only brokers in my country that actually have a good reputation and physical offices, there are also a bunch of other reasons why I wish to work with these particular brokers which I don't want to disclose here but of course referals is one of them.
Can I download more historical data in MT5 then in MT4? This would be nice as I know another CFD-brokerage firm who uses IG prices to supply "their own" price data between 22:00-08:00 in MT4, probably the same for their MT5.
You are right in the way that it's not optimal to trade with IG but not sure how to work around the problem of if build for instance an algo that trades SMA based on x number of bar charts based on price data from a broker that supplies price data between 08:00-22:00 then the algo will behave very differently when being used on IG's MT4.
- People want to see the SMA-lines in a chart in live conditions and feel that their algo is trading thereafter, this will be very different if price data for the algo is different from the broker data.
- IG and CMC(No MT4 yet) are the only brokers in my country that actually have a good reputation and physical offices, there are also a bunch of other reasons why I wish to work with these particular brokers which I don't want to disclose here but of course referals is one of them.
Yes! MT5 can download much more data, including tick-data, but probably not as much as what is available in Dukascopy history database! It really depends on the broker providing that data and how far back they provided.
As for Dealing Desks, if you are only interested in selling your EAs to customers, then that is your business, but if it is for your own profitability, then I am afraid your EAs will only work for a short while until they catch on to the strategy being used (and I don't think your customers will be too pleased with such a result either).
Hi Alain,
I build algorithms for some retail small time clients, they mainly us IG and want to test my algorithms on round the clock price data like IG provides although I favor building algos that don't trade during the night when there is no real underlying being traded there are others who still want this information.
Hi,
When you are backtesting, the best is to use data directly coming from the broker you will be trading from, so you have no surprise cost/slippage etc...
I have gathered quite a bit of market data on index equities (including the dax30), but also fx and commodities since 2011. it's 10 second data bid ask from igindex, which can be converted into 1min 5min 30min... so it s quite a lot of data. The data is gathered whenever the underlying market is tradable by igindex, so it includes their own pricing during the closed time of the official market.
i also gathered options, which is even bigger in data amount.
Let me know if you are interested by it.
Regards,
I was getting different result of my same code and strategy with different Timeframes. As soon as I see the reality I have understood, I need ti clarify rather our backtesting and coding effort for nothing or worth! I was getting MT4 background messed up with misleading testing information. When I have doubted about, I have start testing MT4 result stability instead of my strategy. What I have seen; the reality of testing something other.
Forexample: iOpen-iClose-iHigh-iLow- and Open-Close-High-Low values with the same timeframe testing show different price information. It has shown Some values from feature, some from now and past. During the test and also visual test also shown different results. So I am testing with %99.9 quality tick data with High Performance workstation. I have understood all my effort was for nothing, Thats why real result and Backtesting is not match! I had to brake trading because of this fatal error! In this case I will start coding my own tester with simplest reliable information.
MetaQuotes need to explain! Is that a bug or Brokers Manipulation to make us loose!
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi,
To download historical data I am currently using Tickstory Lite which is Dukascopy's data, it is however not round the clock price data but follows the exchanges opening hours which is not what I'm looking for.
I would like to get <> and would like to hear how you go about getting this data, which is round the clock?
When using the History Center at <>'s MT4 platform to download historical data there seems to be restrictions on how far back you can download data from. This seems to be the case for all MT4 platforms
Reason I would like this is because <> for instance have price data around the clock (albeit they make up their own prices during night on indicies etc),
Does anybody have a good source to get<>, or any other CFD-firm that have round the clock price data (minus settlement period) and tips on how to download it? Would like it in a csv file preferbly and only need OHLC and timestamp and price data on say 5min, 10min, 15min, 30min, 1h, 4h bars , no volume.