# Filling gaps in time series with Nan

28 views (last 30 days)
Andrea on 17 May 2013
Commented: Ruby Hurtado on 14 Sep 2022
Hi there!
I^m currently working with several series of data from different sensor.Each sensor is measuring different quantities, and, theoretically, should provide a value each 30s, so that in 1 day I should have 2880 values for each sensor. I have a matrix for each sensor with 2 columns: timestamp and the corresponding measured data. The problem is that sometime some sensors just misses the measure (very randomly) and they do not record gaps as anything, they simply skip to the next measure, so the length of the matrixes is never 2880, but always shorter and different between the sensors. What i would like to do is to detect these gaps in the timestamp column and to insert Nan values in the corresponding data column. If anyone could offer any suggestions it would be very much appreciated!
PS: the timestamp column is already in Matlab time format: (2013-03-01=735294) (30s=-3.4722e-004)
Thanks!!!!

José-Luis on 17 May 2013
Edited: José-Luis on 17 May 2013
Sounds like a job for intersect():
%Generating random dataset:
numVals = 2000;
all_seconds = 0:30/86400:1;
all_seconds(end) = [];
all_seconds = all_seconds' + floor(now);
your_seconds = all_seconds(sort(randperm(numel(all_seconds),2000)));
your_data = [your_seconds rand(numVals,1)];
filled_data = [all_seconds NaN(numel(all_seconds),1)];
[bla ia ib] = intersect(filled_data(:,1),your_data(:,1));
filled_data(ia,2) = your_data(ib,2);
##### 2 CommentsShowHide 1 older comment
Ruby Hurtado on 14 Sep 2022
What is numVals?

Matt Kindig on 17 May 2013
Edited: Matt Kindig on 17 May 2013
One way that considers floating point errors:
startValue = datenum(2013,5,17,0,0,0); %your starting observation
endValue = datenum(2013,5,17,23,59,59); %your ending observation
increment = datenum(2013,1,1,0,0,30)-datenum(2013,1,1,0,0,0); %30s increments
timePoints = startValue:increment:endValue; %desired time points
% timestamps is your existing time points
% values is your existing sensor measurements
% to illustrate, we will just remove some random time points
ix = sort(ceil(rand(1,10)*length(timePoints)));
timestamps = timePoints;
timestamps(ix) = [];
values = rand(size(timestamps)); %random values
%now get good points
n= histc(timestamps, [timePoints-(increment/3), timePoints(end)+(increment/3)]);
% n will be 1 if sample time is present, zero otherwise.
allValues = NaN(size(timePoints)); %initially set to NaN
allValues(n==1) = values; %fill in with "real" data