anonymous iff function error on table data

I'm trying to find bad data markers (-999) in a table and convert them to NaN. I have an anonymous iff function below that generates an error on the last line:

      % error correction -999 to NaN for humidity data
      iif = @(varargin) varargin{2 * find([varargin{1:2:end}], 1, 'first')}();
      negToNaNfunc = @(x) iif(x<-100,NaN,true,x);
      display(negToNaNfunc(-1000));
      display(data(1:10,:));
      data = varfun(negToNaNfunc,data);

Here is the output showing that the iff function works on a single data point, and the "Dimensions of matrices being concatenated are not consistent." error. See also the timetable data.

         NaN

        10×6 timetable

               Timestamps         InsideHumidity    InsideTemperature    InsideDewpoint    EarthTubeHumidity    EarthTubeTemperature    EarthTubeDewpoint
          ____________________    ______________    _________________    ______________    _________________    ____________________    _________________

          12-Jul-2013 00:00:15     43.8             73.76                50.41             -999                 -1766.2                 NaN              
          12-Jul-2013 00:00:40    43.82             73.76                50.42             -999                 -1766.2                 NaN              
          12-Jul-2013 00:01:08    43.92             73.76                50.48             -999                 -1766.2                 NaN              
          12-Jul-2013 00:02:20     44.3             73.76                50.71             -999                 -1766.2                 NaN              
          12-Jul-2013 00:03:23     44.6             73.76                50.89             -999                 -1766.2                 NaN              
          12-Jul-2013 00:03:48     44.6             73.72                50.86             -999                 -1766.2                 NaN              
          12-Jul-2013 00:04:13     44.7             73.76                50.95             -999                 -1766.2                 NaN              
          12-Jul-2013 00:04:38    44.86             73.76                51.05             -999                 -1766.2                 NaN              
          12-Jul-2013 00:05:03    44.78             73.76                   51             -999                 -1766.2                 NaN              
          12-Jul-2013 00:05:28       45             73.76                51.13             -999                 -1766.2                 NaN              

      Error using tabular/varfun>dfltErrHandler (line 412)
      Applying the function '@(x)iif(x<-100,NaN,true,x)' to the variable 'InsideHumidity' generated the following error:

      Dimensions of matrices being concatenated are not consistent.

      Error in tabular/varfun>@(s,varargin)dfltErrHandler(grouped,funName,s,varargin{:}) (line 191)
          errHandler = @(s,varargin) dfltErrHandler(grouped,funName,s,varargin{:});

      Error in tabular/varfun (line 354)
                  b_data{jvar} = errHandler(s,a_data{jj});

      Error in Aggregate hourly mean temperature for a day (House Humidity) (line 72)
          data = varfun(negToNaNfunc,data);

Please help.

Answer 1

[Walter Roberson]

This is not due to it being a table. This is due to the fact that your iff is expecting to be passed a scalar but it is being passed a vector. The function you pass to varfun needs to take an entire variable as input.

[Walter Roberson]

negToNaNfunc = @(x) x .* (x>=-100)./(x>=-100);

[Matt Cossalman]

Cool Walter. That worked well. Why doesn't the iff function take a vector, but your function does?

[Walter Roberson]

In the iif code, the [varargin{1:2:end}] part squishes every second argument into a single matrix using horzcat(). If the first and third arguments do not have the same number of rows then you get the error about mismatched sizes that you saw. If they do have the same number of rows then you are applying find() to what might be a matrix with multiple rows or multiple columns, and pulling out the first linear index at which the matrix is non-zero, multiply by 2 and try to use that to index varargin -- which is probably not going to be within range.

In a way you got lucky that the inputs happened to be columns: if they had happened to be rows then you would not have gotten the dimensions problem and might not have noticed for a while that you were getting the wrong answer.