Unrealistic fitting confidence levels in noisy data

2 views (last 30 days)
The fitting of simple exponential is giving a confidence level which is not realistic providing how noisy are the data. Any explanation please ?
Thank you. Code:
Data = xlsread('data_test.xlsx','Sheet1','A1:B3990');
Time = Data(:,1) - 2.6E-6 ; % time offset
Signal = Data(:,2)-0.078; % amplitude offset
x = Time;
y = Signal;
y = y/(0.8*max(y)); % data normalization to about 1
format long % just to get more precision digits
mdl = fittype(' a*(1-exp(-b*x)) ','indep','x')
mdl =
General model: mdl(a,b,x) = a*(1-exp(-b*x))
fittedmdl = fit(x,y,mdl,'StartPoint', [max(y) 1E5])
fittedmdl =
General model: fittedmdl(x) = a*(1-exp(-b*x)) Coefficients (with 95% confidence bounds): a = 0.9589 (0.955, 0.9628) b = 5.557e+06 (5.373e+06, 5.74e+06)
plot(x,y, 'bp', 'DisplayName','data')
hold on
xlabel('Time /s')
ylabel('Intensity /a.u')
ax = gca;
ax.FontSize = 15;
As you can see the error on (b) is very small (only 3% !) which is not realistic looking how noisy are the data:
coefficientValues = coeffvalues(fittedmdl);
a = coefficientValues(1);
b = coefficientValues(2);
Tau = (1/b)
Tau =
ConfIntervals = confint(fittedmdl);
b_err = (ConfIntervals(2,2) - ConfIntervals(1,2))/2;
DeltaTau = Tau * (b_err/b)
DeltaTau =
Error = (DeltaTau / Tau)*100
Error =

Answers (1)

Bjorn Gustavsson
Bjorn Gustavsson on 1 Sep 2022
This is most likely due to the large number of data-points you have. Compare with the simpler case of the uncertainty of the average of a number of random samples from a normal-distribution with a large standard deviation - it decreases roughly as the square-root of the number of samples. Here you have a similar situation. One thing you can do is to plot the distribution of the residuals, and investigare how it varies as you vary the parameters - and check how much it starts to skew off zero-centred and symmetric. Another thing you should look at are the residuals - to my naked eye there seems to be some systematic variation in the signal that aren't accounted for - some higher-frequency variations that do not look like random noise.


Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!