how to remove noise from curves and take their derivates

Hello,

I have some curves which are not smooth, I have to take their derivative. therefore first requirement is remove the noise and take the derivative.

I am doing this work through curve fitting using rat35, poly9 etc. and then taking the derivative. but everytime i run the script, result changes slighty.

i am attaching the curve , their zoom version and then warnings which appeared in workspace, would you please guide me how i should handle this issue.

Regards,

Kiran

[Rena Berman]

(Answers Dev) Restored edit

Answer 1

[Bruno Luong]

I would chose spline fitting rather than polynomial (bad) or rational.

Look at the shape of your data, I doubt any simple analytic model can fit it well.

c=readtable('100_li_4th_discharge1.xlsx');
x=table2array(c(:,1));
y=table2array(c(:,2));

% https://fr.mathworks.com/matlabcentral/fileexchange/25872-free-knot-spline-approximation
pp=BSFK(x,y,[],[],[],struct('sigma',1e-9,'KnotRemoval','none'));
ppd=ppder(pp); % derivative

xi=linspace(min(x),max(x),1024);

figure
h1=plot(x,y,'.g',xi,ppval(pp, xi),'r');
yyaxis right
h2=plot(xi,ppval(ppd,xi),'k')
legend([h1; h2],'data','fit','derivative');

[Bruno Luong]

Ops here is ppder.m

function ppd = ppder(pp)

ppd = pp;

coefs = ppd.coefs;
n = size(coefs,2);
ppd.coefs = coefs(:,1:n-1).*(n-1:-1:1);
ppd.order = ppd.order-1;

end

[Bruno Luong]

ppd is dy/dx, if you want dx/dy it is 1./ppd.

BTW in this case you could fit x vs y instead of the opposite.

c = readmatrix('https://fr.mathworks.com/matlabcentral/answers/uploaded_files/1265640/100_li_4th_discharge1.xlsx')

Q=c(:,1);
V=c(:,2);
% https://fr.mathworks.com/matlabcentral/fileexchange/25872-free-knot-spline-approximation
pp=BSFK(V,Q,[],[],[],struct('sigma',1e-9,'KnotRemoval','none'));
ppd=ppder(pp); % derivative
yi=linspace(min(V),max(V),1024);
figure
h1=plot(V,Q,'.g',yi,ppval(pp, yi),'r');
xlabel('V')
ylabel('Q')
yyaxis right
h2=plot(yi,ppval(ppd,yi),'k')
legend([h1; h2],'data','fit','derivative');
ylabel('dQ/dV)*')

[Matt J]

@IQRA KIRAN If it worked, then you should Accept-click the answer.

Answer 2

[Matt J]

If you provide a StartPoint in your fit options, the warning will go away and the fit will be the same when you rerun.

However, the differences you see with each run are not meaningful. You get a slightly different result because the start point of the iterative fitting process is different each time, but in this case they all seem to be converging to essentially the same curve.

[Matt J]

would you please elaborate how to provide a start point ?

The StartPoint is one of the options to fit(),

https://www.mathworks.com/help/curvefit/fit.html?searchHighlight=fit&s_tid=srchtitle_fit_1#d124e32760

and how to realize that which results should be considered the correct one

The second output from fit() is a goodness of fit measurement. You can experiment with different start points to see if one gives a significantly better gof, but I suspect they will all be similar.

[Matt J]

I would have to see the code. It puzzles me that it would be working for one fit type, but not the other.

[Matt J]

Ah, yes. The StartPoint option is only for nonlinear models. So, you cannot and don't need to supply one for the 'polyx' fit types.

Incidentally, order 9 for your polynomials is indeed a bit high, and will make the fit sensitive to small perturbations in the data. You should probably go lower.

[Matt J]

In which case?

[Matt J]

Why would you need another way?

[Matt J]

Use a lower order polynomial so that the parameter estimates won't be as sensitive. Or use a non-polynomial model. Image Analyst suggested an exponential fit. A smoothing spline might also work,

https://www.mathworks.com/help/curvefit/smoothing-splines.html

Basically, you should experiment with other models to see if a stiffer model would work.

Answer 3

[Image Analyst]

You can use sgolayfilt, which is a sliding polynomial fit within the sliding window, however if I were going to take the derivatives of your curves, I'd fit them to an exponential decay (to get a formula with numerical coefficients), which has a known derivative. See my attached exponential decay demo which uses fitnlm. I have many other functions fit with fitnlm, so it you want them, just ask.