Predicting the future with Fourier transforms - page 2

New comment
 
ANG3110:

Only the attached indicator doesn't draw anything.

It does. Scroll the chart to the left, you will see.

 

In the parallel thread 'Not Mashka's Business!' there is a discussion about forecasting with 'perfect MA'. I believe that perfect smoothing can only be obtained using Fourier and thresholding, as done here. The most accurate estimate is just in the middle of the sample window. You need to do similar research, but base it on this algorithm rather than MA.

Secondly, you can mathematically precisely justify the value of the threshold(porog), thereby getting rid of trying (finding the optimal) value of this value.

Thirdly, to increase the forecasting time, you need to increase the accuracy of the current value measurements, it's possible in two ways - simultaneous analysis of multiple currencies, or increasing the number of suppliers of quotes . Optimization. DDE in VB (VBA)'.

 

To see the future,

The "Offset graph" checkbox must be selected in the graph window settings,

extern int InPast=0,

extern inttern Futur=100 for how many bars the forecast

 

As for porog, I will in principle redo this part, because when a frequency goes over the threshold, the graph changes sharply, maybe I will leave it as it was at klot, or I will make a smooth decrease.

 

Funny idyuk, thank you!

Tried it in the tester, so to speak, in real conditions. Sometimes it seems to be the same, but sometimes it is vice versa, especially before the strong moves or at the end of the strong moves.

Very interesting situation is formed when 3 indicators with different power of two, for example, 8, 9 and 10 are applied to the chart, and they often look in different directions. What if we try to use the average value of these 3 indicators?

Have you tried it?

 
m_keeper:

As for porog, in principle I will redo this part, because when any frequency crosses the threshold the graph changes sharply, maybe I will leave it as it was at klot or I will make a smooth decrease.

One more thing I'd like to do, to get rid of FFT (degree of two is not the best choice for finding the best window). Better a discrete transform.

About the threshold, there are Neumann Pearson criteria, ideal observer, look them up. Application of this theory will allow to get away from optimization (finding optimal value on history), and will allow to make an adaptive threshold, there are quite a lot of variants of realization.

 

Looks like the digitizer corner is now in this thread :), then here's a thought-question. If you take the average spectral density for the price chart, you get this kind of picture


You can see that for low frequencies it's not a bad 1/f, but for high frequencies there's a noticeable deviation from this law. This was discussed some time ago and the main version suggested that we have some sort of digitisation noise. The thought is that it might be useful to filter out this component. That is, leave a clean 1/f for the whole frequency range. And the question is how to do this. The idea is to have some procedure that takes as input an array of spectral density and outputs coefficients of digital filter. In principle, I can think of a simple weighting function for the Fourier amplitudes, but a digital filter would be more economical to use.
 

The library now contains __lib_FFT.mq4 file, while the indicator text contains #_lib_FFT.mq4, that's why it is not drawn.

 
Prival:
m_keeper:

As for porog, in principle I will redo this part, because when any frequency crosses the threshold the graph changes sharply, maybe I will leave it as it was at klot or I will make a smooth decrease.

One more thing I'd like to do, to get rid of FFT (degree of two is not the best choice for finding the best window). Better a discrete transform.

About the threshold, there are Neumann Pearson criteria, ideal observer, look them up. Application of this theory will allow to get away from optimization (finding optimal value on history), and will allow to make an adaptive threshold, there are quite a lot of implementation options.

In principle it is very easy to get rid of degree 2:

w=2*pi/T;

for (int k=0; k<=N; k++)
   {                                                              
      sum_cos=0.0; 
      sum_sin=0.0;                                           
      for (int n=0; n<T; n++) 
      {
         sum_cos+=Close[n]*MathCos(k*n*w); 
         sum_sin+=Close[n]*MathSin(k*n*w);
      }
      ak[k]=sum_cos*2/T; 
      bk[k]=sum_sin*2/T;
   }
   ak[0]=ak[0]/2;
   //------------------------------------
   for (n=T-1; n>=0; n--)
   {                                                        
      sum=0.0;                                       
      for (k=0; k<=N; k++)
      {                                                             
         sum+=ak[k]*MathCos(k*n*w)+bk[k]*MathSin(k*n*w);
      }                                                             
      fx[n]=sum;
   }

N - number of harmonics,

T - period.


The optimal window, or more precisely the optimal period T is easiest to choose by 2 criteria, by minimum RMS and by maximum of the sum of harmonic amplitudes. But due to multiple waves on the market, there will be several optimal windows.

And you should also take into account that when you try to extrapolate, due to periodic sines and cosines, previous periodic components will simply be added to the end and, even though the probability of repetition is quite high, the phase and especially the amplitude will still be different and in fact you should always adjust the phase, like a phase detector.


Phase definition as MathArctan() is no good because of arctan properties, at least +/- pi/2 must be taken into account.

Better to define the phase in a slightly different way.

 

There is no need to get away from FFT, in fact it is not necessary to divide the series exactly in half, it is divided to increase counting speed, and not because it is necessary to divide strictly by two. I once worked with FFT that works with series of any length, I'll rewrite it for mql sometime later.


I'm trying to find out what characteristics may indicate false positives

here are two pictures, with n=9 and n=10

In both cases, the window tail has hit a sharp spike that may appear in the future.

Now I'm experimenting with series preprocessing, maybe it will help.


Changed smoothing condition, no chart jumps now, everything runs smoothly.


As for the chart being redrawn, I do not see any problem.

The closing prices are not the past but the events that generated them - the waves (in Fourier's case) - are the past

and we learn more and more about them with every new bar.

Files:
fft_and_future_1.mq4  6 kb
123456789...55
New comment