How to filter noise from a time series without losing important informations?
15 views (last 30 days)
Show older comments
Hello everyone,
as described in the topic I have a time series like this:
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/660485/image.png)
Zoomed in you can see there is a lot of noise.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/660490/image.png)
I need to filter the noise in the green rectangle without losing informations like the spike in the green circle.
As can be seen in the frequency domain there is a lot going on.
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/660495/image.png)
I have tried smoothing and filtering endlessly with Data Analyser Toobox and the Signal Processing Toolbox. I have tried the Filter Designer just like the filter funktion with various settings like Savitzky-Golay Filter or Exponential Moving Average Filter or Bandpass. But everytime I get rid of the noise I also lose to much important informations.
Can anybody tell if and how I can achive this?
I have attached the data.mat with the time series in case there is someone who likes to try.
Best regards
Fabian
2 Comments
Sharmin Kibria
on 25 Jun 2021
Edited: Sharmin Kibria
on 25 Jun 2021
Did you try denoising it using the Signal analyzer app? I tried to denoise the signal with symlet 4 wavelet (default setting) and got the attached denoised version. I was able to preserve 98% of the original signal energy in the denoising process. Do you think it is good enough?
Thanks
Sharmin
Accepted Answer
Mathieu NOE
on 28 Jun 2021
hello
this is my suggestion and the result : as you can see , the large amplitude spikes are not changed by the filtering (perfect overlay with raw data)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/667545/image.png)
clc
clearvars
load('data.mat');
x = data(:,1);
y = data(:,2);
t = data(:,3);
dt = mean(diff(t));
Fs = 1/dt;
ind = find(t>3.9 & t < 4);
t = t(ind);
x = x(ind);
y = y(ind);
samples = length(y);
figure(1)
plot(t,x);
%% "smart" data smoothing
N = 9;
xs = medfilt1(x, N,'truncate');
xs = medfilt1(xs, N,'truncate');
% replace noisy data by smoothed (only low amplitude signal is affected)
ind = find(abs(xs)<1);
xx = x;
xx(ind) = xs(ind);
figure(1)
% plot(t,x,'b',t,xs,'r',t,xx,'g');legend('Raw','Smoothed');
plot(t,x,'b',t,xx,'r');legend('Raw','"smart" Smoothed');
title(['Data samples at Fs = ' num2str(round(Fs)) ' Hz / Smoothed with medfilt1' ]);
grid on
2 Comments
Mathieu NOE
on 28 Jun 2021
hello
even simpler code , only one stage of filtering , and noisy low amplitude signal simply replaced by zeros
clc
clearvars
load('data.mat');
x = data(:,1);
y = data(:,2);
t = data(:,3);
dt = mean(diff(t));
Fs = 1/dt;
ind = find(t>3.9 & t < 4);
t = t(ind);
x = x(ind);
y = y(ind);
samples = length(y);
figure(1)
plot(t,x);
%% "smart" data smoothing
N = 9;
xs = medfilt1(x, N,'truncate');
% replace noisy data by smoothed (only low amplitude signal is affected)
ind = find(abs(xs)>3);
xx = zeros(size(x));
xx(ind) = x(ind);
figure(1)
% plot(t,x,'b',t,xs,'r',t,xx,'g');legend('Raw','Smoothed');
plot(t,x,'b',t,xx,'r');legend('Raw','"smart" Smoothed');
title(['Data samples at Fs = ' num2str(round(Fs)) ' Hz / Smoothed with medfilt1' ]);
grid on
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/667565/image.png)
More Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!