Fill in missing NaNs

I am trying to fill these NaNs following this rule: If there is a single NAN, I want the NAN to be filled in with the average of the numbers before and after. If there is more than one NAN. I want the NAN to be filled in with the nearest number. For example, row 2305 should be the average of 16.3 and 14.8, from 2298 to 2304 should be 16.3, and from 2306 to 2312 should be 14.8.

[darova]

Do you have any code? What problem are you facing?

[Cuong Nguyen]

L1 = length(maxTemp);

for i = 1:1:L1
    
    for j = 0:20
        if  i <27028 && isnan(maxTemp(i+j))
          a=j; 
        else
            break
        end
    end
    
    if isnan(maxTemp(i)) && isnan (maxTemp(i+a)) % 'a' distance away from i
       for k = 0:a % new variable which goes upto variable a
           f = a; %new variable which goes down from variable a
            
               if (a/2 > k) 
                   maxTemp(i+k)=maxTemp(i+k-1);
                   maxTemp(i+f)=maxTemp(i+f+1);
                   
               else
                   break
               end
              
                if mod(a,2)== 0 % if a is even
                   maxTemp(i+k)= avg(maxTemp(i+k-1),maxTemp(i+k+1));
                 
                elseif mod(a,2)~= 0 % if a is not even
                   maxTemp(i+k)=maxTemp(i+k-1);
                   maxTemp(i+f)=maxTemp(i+f+1);
                end
            f = f-1; % decrease by 1 
       
       end
    end
    end

That array is showing maximum temperature day by day (on the photo is day 2297 to day 2313). After I run my code, this is what I got:

I need row 2305 to be average of 16.3 and 14.8, and row 2298 - 2304 to be 16.3. Thanks a lot for your time.

[Image Analyst]

Why not use interp1() or regionfill() to linearly interpolate from one side to the other?

[Cuong Nguyen]

I am quite a newbie to matlab, can you make it more specific? Thanks.

Answer 1

[Ameer Hamza]

Try this:

x = [1;2;3;4;nan;nan;nan;nan;nan;5;7;8;nan;nan;nan;nan;11;11;12;nan;nan;nan;15];
bb = regionprops(isnan(x));

idx_nnan = find(~isnan(x));
idx_nan = find(isnan(x));
[~,idx] = min(abs(idx_nan - idx_nnan'), [], 2);
x(idx_nan) = x(idx_nnan(idx));

a = [bb.Centroid];
a(1:2:end) = [];
bb = bb(a==fix(a));
for i=1:numel(bb)
    idx_center = bb(i).Centroid(2);
    idx = cumsum(bb(i).BoundingBox([2 4])) + [-0.5 0.5];
    x(idx_center) = mean(x(idx));
end

Original x:

x =
     1
     2
     3
     4
   NaN
   NaN
   NaN
   NaN
   NaN
     5
     7
     8
   NaN
   NaN
   NaN
   NaN
    11
    11
    12
   NaN
   NaN
   NaN
    15  

New x:

x =
    1.0000
    2.0000
    3.0000
    4.0000
    4.0000 % <--- nearesr value
    4.0000 % <--- nearesr value
    4.5000 % average
    5.0000 % <--- nearesr value
    5.0000 % <--- nearesr value
    5.0000
    7.0000
    8.0000
    8.0000 % <--- nearesr value
    8.0000 % <--- nearesr value
   11.0000 % <--- nearesr value 
   11.0000 % <--- nearesr value
   11.0000
   11.0000
   12.0000
   12.0000 % <--- nearesr value
   13.5000 % average
   15.0000 % <--- nearesr value
   15.0000

[Cuong Nguyen]

Big thanks! Seems like it works for me. But can you explain more about this line ?

[~,idx] = min(abs(idx_nan - idx_nnan'), [], 2);

Also "Centroid" and "BoundingBox" as well?

[Ameer Hamza]

Cuong, this code uses a tricky way to find the center point of each NaN cluster by using a function regionprops() from the image processing toolbox. This function is normally used to detect similar regions in an image and give bounding boxes around those regions. I applied that function on a 1D array to find clusters of NaN in your input array. The Centroid gives the center point of NaN, and BoundingBox gives the edges of each NaN cluster.

Now the line

[~,idx] = min(abs(idx_nan - idx_nnan'), [], 2);

is used to find all the NaN which are not at center and find their nearest non-NaN number.

[Cuong Nguyen]

Oh nice nice. Thank you, I got something to learn from this!

[Ameer Hamza]

Glad to be of help.

Answer 2

[darova]

Use bwlabel

A1 = isnan(A);                              % find NaN
[L,n] = bwlabel(A1);                        % label each region
xx = 1:length(A);
for i = 1:n                                 % loop through each region
    BW = L==i;                              % select region
    if sum(BW(:))>1                         % if more than one 'NaN'
        ix1 = find(BW,1,'first');           % first index of region
        ix2 = floor(mean(BW.*xx));          % find mean index in region
        ix3 = find(B2,1,'last');            % last index of region
        A(ix1:ix2) = A(ix1-1);              % fill first part
        A(ix2+1:ix3) = A(ix3+1);            % fill second part
    else
        ix = find(BW);
        A(ix) = mean(A(ix([-1 1])));        % average
    end
end

[Cuong Nguyen]

I tried yours and I got this message "Array indices must be positive integers or logical values". It also shows that the error is in "A(ix) = mean(A(ix([-1 1])))" line.

Answer 3

[Andrei Bobrov]

This question is a repeat of this question:

x = [1;2;3;4;nan;nan;nan;nan;nan;5;7;8;nan;nan;nan;nan;11;11;12;nan;nan;nan;15];
out = [x,f1(x)]
x = [16.3;nan(15,1);14.8];
out = [x, f1(x)]

Here f1:

function out = f1(x)
    b1 = fillmissing(x,'linear');
    b2 = fillmissing(x,'nearest');
    d = [0;diff(bwdist(~isnan(x)),2);0]==-2;
    out = b2;
    out(d)  = b1(d);
end

[Cuong Nguyen]

Oh icic, and in your answer you said "In R2016b". Is there any feature in r2016b that other versions doesnt' have?

[Andrei Bobrov]

No! It only says that this version is available to me :) and this code will work with R2016b and later.

[Cuong Nguyen]

Ahh ok. Can you explain more about this line? It's something about the distance to the edges of NaN cluster I reckon, isn't it?

d = [0;diff(bwdist(~isnan(x)),2);0]==-2;