# How to find variance and std in matlab without using zeros in matrix?

14 views (last 30 days)
Leesy on 22 Feb 2017
Commented: Rik on 12 Jan 2018
I have a matrix (pm2d), and i need to calculate the std and (population) variance in each column without using the zero values. I was wondering if i could use a for loop or an if statement?
For my variance i used:
var = sum(pm2d.^2)/(length(pm2d)-1) - (length(pm2d))*mean(pm2d).^2/(length(pm2d)-1)
But that took the zeros into account...
And for the standard deviation i used:
S = std(pm2d)
which definitely used the zeros.
Every code i try to write is not working. Any assistance would be appreciated! Thanks!

Vandana Rajan on 22 Feb 2017
Edited: Vandana Rajan on 22 Feb 2017
Hi,
You can use nanvar and nanstd functions in statistics toolbox.
>> b = pm2d; % just to retain the original matrix
>> b(b==0) = NaN;
>> nz_var = nanvar(b);
>> nz_std = nanstd(b);
Of course, this solution works only if you have license to statistics toolbox.
Leesy on 22 Feb 2017
I do not, is there another way to do it?
Rik on 22 Feb 2017
Edited: Rik on 12 Jan 2018
Yes. My solution, or the one Jan Simon suggested (which should have better performance).

Rik on 22 Feb 2017
In my solution, I abuse the option of omitting NaNs when using mean and std
pm2d_temp=pm2d;%create a copy
pm2d_temp(pm2d_temp==0)=NaN;%overwrite zeroes with NaN
var = sum(pm2d.^2)/(length(pm2d)-1) - (length(pm2d))*mean(pm2d_temp,'omitnan').^2/(length(pm2d)-1)
S=std(pm2d_temp,'omitnan');
Vandana Rajan on 22 Feb 2017
That's cool :)

Jan on 22 Feb 2017
Edited: Jan on 22 Feb 2017
It works with replacing the zeros by NaNs and ignoring the NaNs, but you can do this directly also:
function [m, v, s] = StatsNonZeros(x, dim)
if nargin < 2 % Default: first non-singelton dimension
dimv = [find(size(x) ~= 1), 1]; %#ok<MXFND>
dim = dimv(1);
end
n = sum(x ~= 0, dim); % Number of non zero elements along dim
m = sum(x, dim) ./ n; % Zeros are neutral in the sum
v = sum(bsxfun(@minus, x, m) .^ 2, dim) ./ (n - 1);
s = sqrt(v);
end
This is what happens inside nanmean and nanstd also, after the NaNs have been replaced by zeros. Therefore it is an indirection to replace the zeros by NaNs at first.
Call it as:
[m,v,s] = StatsNonZero(pm2d)
Franck Eitel on 29 Nov 2017
What's the meaning of 's' here? I think we were looking for the variance and standard deviation. Pls could you clarify it for me?
Rik on 12 Jan 2018
[m, v, s] are the mean, variance, and standard deviation, although I presume you will have found that by now.