Fast Fourier Transform - Cycle Extraction - page 11
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Very interesting topic. Seems it's very difficult to figure out the cycle, or to figure out the right cycle.
HaHa...it's so easy to extract cycles...just put a moving average of RSI...
FFT isn't Suitable for Forecasting
I tried that FFT indicator. Beautiful. Too bad it's completely useless for forecasting.
The reason you can't forecast with an FFT comes from the nature of the Fourier transform and the assumptions that are implicit in its design. Someone with only a passing familiarity with the Fourier Transform may believe (incorrectly) that the FFT will extract the cycles that the market is exhibiting. Sadly, it is not so.
The fundamental principle that the Fourier Transform (FT) is based upon is that any arbitrary time series can be produced by some superposition of frequencies, of appropriate magnitude and phase. The FT is a mathematical transformation that computes the set of frequencies necessary to produce that exact time series. So, why can't the output of that calculation then be used to project forward and predict what the market will do next? There are several reasons, and I'll take them up individually.
The first reason why the FT has no predictive power arises from the fact that although any time series can be produced by a superposition of waves, those waves don't necessarily correspond to any actual cyclical phenomena occurring in the time series. Consider a time series that is zero everywhere, except somewhere in the center there is a single value that is 1. Perhaps it represents a thinly traded market, and there was only one trade in that period. Well, according to Fourier theory, there is some combination of frequencies that will cancel each other out at every point except in the center, where they will reinforce and produce a spike. The FFT will gleefully find those frequencies for you. But that doesn't correspond to any actual physical process in the marketplace -- there aren't in fact any such cycles acting in that marketplace -- so any forecast that could be derived from extrapolating such cycles would mean nothing.
The second reason the FFT can't forecast is that implicit in the makeup of the Fourier Transform is the assumption that the frequencies extend from minus infinity to plus infinity in time, forever. That has the automatic effect of making the time series periodic, from minus infinity to plus infinity! That means that what the FT will "predict" will happen in the period n+1 will be an exact repeat of what occurred in period n. In the case of the single spike mentioned in the first example, if the waves are projected forward into the next time interval, they will again cancel everywhere except to reinforce in the center, producing another spike. The FFT will forecast an infinite train of spikes into the future. More generally, the set of waves derived from the FFT will simply draw the exact same pattern again and again in every block going forward.
And if those two reasons weren't enough, here is another one. The set of frequencies that you will get from the FFT are entirely determined by the block size of the FFT. If the block size of your FFT is 512 points, then the FFT will produce a fundamental tone with period of 512, a second harmonic of period 256, a third harmonic of period 512/3, etc, on up to the highest frequency, having a period of 2. If you choose 739 as the block size, then the FFT will produce a fundamental frequency of 1/739 and every harmonic of that. If the FFT block size is 64 and the market cycle is periodic in 39 bars, it won't hit it. The combination of frequencies it does give you will provide a reasonable fit for the time series, but they won't project well into the future. You can match the smaller frequencies more exactly by using a larger FFT block size. But the downside of that is the FFT will look for that frequency to be present and constant throughout the entire block. If there are only a few cycles present near the end of the block, it will not fit that frequency particularly well.
Every once in a while the FFT gets lucky -- the market happens to be manifesting a cyclic pattern which happens to be within the FFT's bag of frequencies. Then it suddenly produces an accurate forecast.
As for extracting actual cyclic behavior from market data, I hear the Goertzel is pretty good, though I haven't tried it yet.
Fourier Transform vs Series
I noticed some people mistaking something here. There is a difference between the Fourier Transform and Fourier Series.
When you calculate Fourier series, you are basically calculating the projection of a function onto a trigonometric basis in an interval. I.e. you are using an infinite sum of sines and cosines to represent the function exactly. An infinite number of frequencies are represented in the infinite series. The representation is exact if you can compute the infinite sum, however its only exact on the interval. This representation of the function will be periodic from -inf to +inf with its period being the size of the interval. Since a computer cannot calculate an infinite sum, you have to stop at some point and thus you get an approximation.
The Fourier Transform is an integral transform that literally "transforms" a function from one domain to another. I.e. it converts a function in a time domain to a function in a frequency domain. There is a tie between the Fourier Series and Fourier Transform, but I don't want to get into it.
Also, FFT typically stands for "Fast Fourier Transform", which is an algorithm for computing the Fourier transform of discrete data.
In the case of Forex, its obvious there is cyclic behavior, however it remains to be proven its periodic. No one knows the "function" that determines price, so we are looking at it like dark matter. I personally believe there is some usefulness of the Fourier transform during certain situations, however I am skeptical. I have done some backtesting on it, and it seems to work only when news isn't happening, and only with some tweaking of the interval its computed over. This being so, I think it would be an interesting topic to research how to make the Fourier Series dynamic under various volatility levels and market strengths if applied to Forex.
Never it remains to be proven its periodic.
No one knows the "function" that determines price, so we are looking at it like dark matter -- a SCREW UP clock work -- for your TIMEX watch !
the only exception is aussie trading hours
prediction -- tried it -- consolidation, it will be successful, if it is about the magnitude of PIP SPREAD ---- hehehe !
By the way. I just thought I would mention, I was lecturing about people mistaking Fourier Series for Fourier Transform and then I went ahead and accidently used "Fourier Transform" when I meant "Fourier Series" in my last paragraph. Irony?
Its because you obviously jumped to conclusions about what Crodzilla was implying because you never bothered to understand Cycles, Cycle Extraction, and Cycle Indicators.
The fact is, if someone can master Cycles and finds a natural rhythm and plots it on their chart, not only are patterns going to be highly more probable to fall through, it gives you a clear visual presentation of what might happen.
Please read Encyclopedia of Chart Patterns and you'll notice, trendlines + chart patterns has a failing rate, especially if you don't use confirmation.Just checking in. I don't see much work getting done here.
I have completed my Goertzel browser. As someone mentioned Goertzel may be better. And, I have found that it is, so far. Its much faster to compute, and we don't need any external DLL's to mess with, as with FFT.
I also had to write a windows based program to align the phases. In the attached picture, the first cycle (highest magnitude) is 180 degrees out of phase.
As someone else mentioned, you can have different tones with different frame lengths. That is both true and false. One reason why some tones may disappear is that they dissipate, the way waves do in a pond. And I have found that "persistent" waves can exist for finite lengths of time, which is what I want to take advantage of. I wrote a program to extract all the waves in all frames, in all possible lengths, then choose only those waves that "persist" in a current frame and past frames. Then, I align the individual waves by simulating "trading" each cycle and each combination of cycles to phase shift each cycle. The program shows me which cycles have had the highest hit rate. So far, I have not seen a combination of cycles (adding) that were profitable. It has always been a single persistent cycle that gets the highest hit rate.
There is still a lot to do, but I'm slowly making progress to see how cycles are prevalent in the market. My goal is to find and take advantage of "persistent" and current cycles if only to trade for one trade. Then re-evaluate, to see if the cycle is dissipating. And that may be done by evaluating a single cycle over time to see if its phase is shifting and its amplitude is falling. We may also be able to evaluate whether a single cycle is getting stronger by using the same method.
The attached pic is from my browser. The yellow cycle is the highest magnitude, and the purple is the second highest magnitude cycle.
Carl
PS. And don't forget, cycles can be projected forward in time to see when they peak/valley!!
How can I help?
Hi Crodzilla,
The work you are doing on cycles is very interesting. I agree with you that if cycles can be mastered they will form the core of a very good trading system.
I have been using Stochastics set to 8,3, 8 in conjunction with the Awesome Oscillator as a crude cycle indicator.
I wait until the Stochastics on the Daily is in the middle of it's cycle and use that as a direction of trade indicator.
If Stochastics on the Daily is indicating an up cycle I wait until H4 H1 M30 M15 M5 & M1 are at the bottom of their cycles before I enter. Using this approach I have been able on some occasions to enter within a few pips of the top or bottom of a move.
I know it is a very crude approach but all the same I have been able to increase my capital, real money, by over 500% since the 16th of September this year.
In your last post you wrote "Just checking in. I don't see much work getting done here."
What you are doing is a far more sophisticated, scientific approach than I have been doing, and I would like to help if I can ... problem is I don't have any high level programming skills. My programming is limited to some machine code I learnt for the Motorola 6800 series of processors in the middle 80's.
Is there is any way I can contribute ?
Stewart
Crodzilla cycles are very cool
I think the extraction of the cycles is brilliant !
Can I crunch some numbers for different pairs for you?
This will tell us direction, and that is half the battle.
wgreville