Matlab curve fit equation does not match when plotted in MS Excel

I would really appreciate your help with curve fitting. Thank you in advance.

I am trying Fourier Series to following data:

t=[30,60,90,120,150,180,210,240,270,300,330,360,390,420,450,480,510,540,570,600]
x=[1.462,0.017,0.132,0.257,0.253,0.472,0.698,0.889,0.809,0.59,0.517,0.503,0.451,0.371,0.198,0.17,0.2781.18,1.364,1.59]

Matlab gives a really nice fit, the result is below. However when I use this equation along with its coefficients and plot them in Excel its a different plot. Is there something I am missing here ? It is no where even close to actual data. But it overlays very well when seen in Matlab. How ?

General model Fourier:

     f(x) = 
               a0 + a1*cos(x*w) + b1*sin(x*w) + 
               a2*cos(2*x*w) + b2*sin(2*x*w) + a3*cos(3*x*w) + b3*sin(3*x*w) + 
               a4*cos(4*x*w) + b4*sin(4*x*w) + a5*cos(5*x*w) + b5*sin(5*x*w) + 
               a6*cos(6*x*w) + b6*sin(6*x*w)
       where x is normalized by mean 315 and std 177.5
Coefficients (with 95% confidence bounds):
       a0 =   1.886e+08  (-5.755e+10, 5.793e+10)
       a1 =  -3.245e+08  (-9.96e+10, 9.895e+10)
       b1 =   9.628e+06  (-2.828e+09, 2.848e+09)
       a2 =    2.05e+08  (-6.239e+10, 6.28e+10)
       b2 =  -1.229e+07  (-3.623e+09, 3.598e+09)
       a3 =  -9.278e+07  (-2.832e+10, 2.813e+10)
       b3 =   8.495e+06  (-2.47e+09, 2.487e+09)
       a4 =   2.853e+07  (-8.611e+09, 8.669e+09)
       b4 =  -3.573e+06  (-1.036e+09, 1.029e+09)
       a5 =  -5.352e+06  (-1.617e+09, 1.606e+09)
       b5 =   8.662e+05  (-2.466e+08, 2.483e+08)
       a6 =   4.629e+05  (-1.379e+08, 1.388e+08)
       b6 =  -9.378e+04  (-2.648e+07, 2.629e+07)
       w =      0.3203  (-7.985, 8.626)

Goodness of fit: SSE: 0.03824 R-square: 0.9907 Adjusted R-square: 0.9707 RMSE: 0.07983

[Matt J]

No idea what you did to plot the fit in MATLAB. The parameters look totally unreasonable in their order of magnitude, considering the magnitude of your t,x data, and the code below which I used to plot the fit shows terrible agreement as well.

f = @(x) a0 + a1*cos(x*w) + b1*sin(x*w) + ...
a2*cos(2*x*w) + b2*sin(2*x*w) + a3*cos(3*x*w) + b3*sin(3*x*w)+ a4*cos(4*x*w) + b4*sin(4*x*w) + a5*cos(5*x*w) + b5*sin(5*x*w) + a6*cos(6*x*w) + b6*sin(6*x*w);...

    t=[30,60,90,120,150,180,210,240,270,300,330,360,390,420,450,480,510,540,570,600];

    x=[1.462,0.017,0.132,0.257,0.253,0.472,0.698,0.889,0.809,0.59,0.517,0.503,0.451,0.371,0.198,0.17,0.278,1.18,1.364,1.59];

   plot(t,x,t ,f(t));

Answer 1

[the cyclist]

I have essentially the same reaction as Matt J. It is unclear how you got your fit, what variable you are normalizing (and how), and what you are plotting. (Matt, I think it is the normalization that brings the seemingly out-of-whack values in line.)

You might want to post the complete code that you used to do the fit in MATLAB, and whatever code you used to plot and show the fit.

I am going to make a guess, though. You have 14 parameters to fit 20 data points. You are likely massively overfit, and this "fit" depends on some exquisite fine-tuning of the parameters.

MATLAB is internally using all 15 (or so) significant figures of its double-precision coefficients to get the fit. I am guessing that maybe you are carrying over only the 4 or 5 digits into Excel, and therefore you get a terrible fit.

Just a guess. If you post your whole code, we may be able to help you more.

Answer 2

[Nilesh]

Thank you for your replies Matt and cyclist. I am using the GUI. Below are the steps that got me that solution:

(1) type in t and x values 
(2) use 'cftool' command to open Curve Fitting Toolbox
(3) in the toolbox, select 't' for 'X data' and 'x' for 'Y data' from drop down menu
(4) then select 'Fourier' from drop down menu and '6' for 'Number of terms'
(5) run using Auto fit

The resulting plot in the toolbox showed raw data values and a overlay of fitted curve.

I tried different parameters by limiting a0=2 (below) which gave a fit that was reproducible in Excel.

Best Regards.

General model Fourier5: f(x) = a0 + a1*cos(x*w) + b1*sin(x*w) + a2*cos(2*x*w) + b2*sin(2*x*w) + a3*cos(3*x*w) + b3*sin(3*x*w) + a4*cos(4*x*w) + b4*sin(4*x*w) + a5*cos(5*x*w) + b5*sin(5*x*w) where x is normalized by mean 315 and std 177.5 Coefficients (with 95% confidence bounds): a0 = 2.00E+00 (-2.169e+06, 2.307e+06) a1 = -3.37E-01 (-2.484e+06, 2.352e+06) b1 = -2.21E+00 (-3.04e+06, 2.845e+06) a2 = -1.46E+00 (-4.741e+05, 4.205e+05) b2 = 5.58E-01 (-2.207e+06, 2.34e+06) a3 = 1.40E+00 (-8.197e+05, 8.779e+05) b3 = 1.535 (-4.654e+05, 4.53e+05) a4 = 1.115 (-2.365e+05, 2.254e+05) b4 = -1.973 (-9.811e+04, 8.755e+04) a5 = -0.7325 (-9510, 9204) b5 = -0.4044 (-2.785e+04, 2.967e+04) w = 0.004354 (-0.006288, 0.01235)

[the cyclist]

I don't have the curve fitting toolbox, so I can't replicate this. I believe there is a feature that will produce code to reproduce the coefficients programmatically, though.

[Matt J]

where x is normalized by mean 315 and std 177.5

We don't understand what this means, and we don't understand which 'x' this refers to. In some places you use 'x' to denote your input data (the function of t) and in other places 'x' denotes your input data, as in f(x).

Answer 3

[Nilesh]

Found it ! as you have pointed out the normalized with mean and SD was the key. The values for the 'x' variable in the Fourier equation need to be normalized before plotting them with the actual data.

In this example it would be for 't': Mean = 315; SD = 177.5

So instead of using putting '30' in the equation, it has to be (30-315)/177.5 = -1.605633803. And so on.

The plot looks good now.

Still need to understand how to interpret the fit with the normalized values as opposed to not using normalized values.

Thank you for your help. I really appreciate it. Best Regards.

Reference: http://www.mathworks.com/matlabcentral/newsreader/view_thread/145211

[Diablo]

Thank you very much. You solved my problem as well.

[Karthik Raghavan Ramaswamy]

I too have the same problem. But even after substituting the sd and mean, i am getting an awkward result. But my curve fitting toolbox result is accurate. any idea?

[Viviana Rota]

I too have the same problem.

I need to find the best fit of points in time. through the curve fitting app tool I have found the third order fourier equation. App fitting gives the coefficients of the equation and says the abscissa is centered with mean and standard deviation.

I would like to replicate the obtained curve fitting without app, but I cannot reconstruct the x-abscissa on which the fitting was calculated.

how do Ido that ? thanks

[Torsten]

The x-abscissa on which the fitting is based are

x_normalized = (x-mean(x))/std(x)

Here is an example:

load census
f=fit(cdate,pop,'poly3','Normalize','on','Robust','Bisquare')

f = 
     Linear model Poly3:
     f(x) = p1*x^3 + p2*x^2 + p3*x + p4
       where x is normalized by mean 1890 and std 62.05
     Coefficients (with 95% confidence bounds):
       p1 =     -0.4619  (-1.895, 0.9707)
       p2 =       25.01  (23.79, 26.22)
       p3 =       77.03  (74.37, 79.7)
       p4 =       62.81  (61.26, 64.37)

cdate_norm = (cdate-mean(cdate))/std(cdate);
hold on
plot(cdate,f.p1*cdate_norm.^3+f.p2*cdate_norm.^2+f.p3*cdate_norm+f.p4)
plot(cdate,pop,'o')
hold off
grid on

Answer 4

[Ilan Leibu]

I found the answer to the curve fitting coefficients using Fourier function Basically don't use the toolbox curve fitting ...nothing of the recommendations I saw in here helped (normalization to standard deviation and median...)

Just use fit function in the command window ... fit(var1,..,varn,'fourier8') multiply w by the value specified when getting the constants using coeffvalues() function

Answer 5

[Ilan Leibu]

Correction to the previous message: calculate value of w by using w= function name*w in command ; then multiply function by the value specified when getting constants values

Answer 6

[PY SUN]

I meet the same problem use curve fitting app.

I solve it by limit the lower and upper limit at fit option.