How to calculate an exponentially weighted moving average on a conditional basis?

Hi, I am using MATLAB R2020a on a MacOS. I have a signal 'cycle_periods' consisting of the cycle periods of an ECG signal on which I would like to perform an exponentially weighted mean, such that older values are less weighted than newer ones. However, I would like this to be done on an element-by-element basis such that a given element is only included in the overall weighted mean if the weighted mean with the current sample does not exceed 1.5 times or go below 0.5 times the weighted mean without the element.

I have used the dsp.MovingAverage function as shown below to calculate the weighted mean, but I am really unsure as to how to manipulate the function to include my conditions.

% Exponentially weighted moving mean for stable cycle periods
    movavgExp = dsp.MovingAverage('Method', 'Exponential weighting', 'ForgettingFactor', 0.1);
    mean_cycle_period_exp = movavgExp(cycle_periods);

I would very much appreciate any help regarding this matter, thanks in advance.

[Mathieu NOE]

hello

your special case is not covered in the standard matlab package

you will have to create your own , but that doesn't seems a "mission impossible"

all the best

Answer 1

[Uday Pradhan]

Hi Cai,

In order to accomodate your custom conditions, you may think about creating a small script to mimic the algorithm used by the dsp.MovingAverage system object (exponential wieghing method). A very detailed explanation about the same is given in this link. I have created a dummy code for reference:

%Let t be a 1-D array of data points
lambda = 0.1; %forgetting factor
prevWeightFactor = 0; %initialise weight factor and moving average
prevAvg = 0;

for i = 1 : length(t)
    presentWeightFactor = lambda * prevWeightFactor + 1;
    presentAvg = (1 - (1/presentWeightFactor)) * prevAvg + (1/presentWeightFactor) * t(i);
    if (presentAvg < 0.5 * prevAvg) || (presentAvg > 1.5 * prevAvg)
        presentAvg = prevAvg;  %ignore this input, you might want to skip this step for the first sample
    else   %accept this input in the moving average
        prevWeightFactor = presentWeightFactor;
        prevAvg = presentAvg;
    end
end

I hope this helps!

[Cai Chin]

Hi Uday, thank you very much for this suggestion. I have tried to implement this as follows: At the moment, I have created a while loop which goes through my input signal 'cycle_periods' and calculates the moving mean using the algorithm. It then compares it to a scalar 'mavCnd' and either keeps or removes it. A separate index 'j' is a record of the indices of the previously accepted elements. At the end of the loop, the 'tail' of outliers is removed. However, I cannot seem to be able to remove the outliers and the signal is not smoothed. Furthermore, the vector 'x' (as shown) which should contain a list of the acceptable moving mean values, has a number of repeats of the first value of the signal 'cycle_periods' and I cannot figure out why.

% Calculate successive weights to test how they change with different FF
% values
lambda = 0.8;
w = zeros(length(cycle_periods),1); 
w(1) = 1; % initialize the weight for the first sample
for i = 2:length(cycle_periods)
    w(i) = lambda*w(i - 1) + 1; % calculate the successive weights
end
% Calculate exponentially weighted moving mean manually whilst removing
% outliers in real-time
x = cycle_periods; % array for exponentially weighted means
i = 2; % index for counting through input signal
j = 2; % index for counting through accepted exponential mean values 
while i <= length(cycle_periods)
    mavCnd = (1 - 1/w(j))*x(j - 1) + (1/w(j))*cycle_periods(i);
      if mavCnd < 1.5*(x(j - 1)) && mavCnd > 0.5*(x(j - 1))  % Identify high and low outliers
        x(j) = mavCnd;
        cycle_index(j) = i;
        j = j + 1;
      end
    i = i + 1;
end
% Scrap the tail
x(j - 1:end)=[];
% Indices of valid and invalid elements
idxCycle = [1:length(cycle_periods)];
cycle_periods(cycle_index) % List of valid points
cycle_periods(idxCycle(~ismember(idxCycle, cycle_index))); % List of removed outliers

x = [0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0050, 0.0040, 0.0040, 0.0050, 0.0040, 0.0050, 0.0030, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0040, 0.0050, 0.0050, 0.0050, 0.7830, 0.0040, 0.8010, 0.7720, 0.7560, 0.7800, 0.0050, 0.7960, 0.7840, 0.7600, 0.7690, 0.0040, 0.8110, 0.0040, 0.8590, 0.0050, 0.8350, 2.4050, 0.8200, 0.0040, 0.8140, 0.0050, 0.8090, 0.7890, 0.7740, 0.0030, 0.7480, 0.7420, 0.7650, 0.8110, 0.0040, 0.8350, 0.0040, 0.8330, 0.0040, 0.8280, 0.0040, 3.9230, 0.0040, 0.8390, 0.0040, 0.8430, 0.8110, 0.8050, 0.8060, 0.8230, 0.0040, 0.8260, 0.0040, 0.8330, 0.0040, 0.8250, 0.7930, 0.7810, 0.7840, 0.0050, 0.7990, 0.0050, 0.8130, 0.0050, 0.7910, 0.7950]

I would very much appreciate any help with solving this. Thanks in advance

[Uday Pradhan]

Cai, thanks for the detailed reply. I have a few questions here:

for i = 2:length(cycle_periods)
    w(i) = lambda*w(i - 1) + 1; % calculate the successive weights
end

Do you want to assign weights for all the data points irrespective of if the corresponding moving average is accepted or not? In my code I have assumed that we are not considering the weights of outlier data points for the calculation of successive weights.

Next I would suggest to take

x = zeros(length(cycle_periods),1); 
x (1) = cycle_periods(1);

to properly distinguish between the accepted and original data points. Also, if you could share the cycle_periods vector, I will be able to answer in a more clear manner.

[Cai Chin]

Hi Uday, thanks again. I don't need to assign weights for all the data points - as you said, I would only need them for the accepted values. However, I thought it would be easier to just refer to the accepted indices (j) in the calculation of the weighted mean afterwards.

I shall include what you have suggested in terms of assigning 'x'.

The vector cycle_periods is as follows:

cycle_periods = [0.0040, 0.7740, 2.3040, 0.8340, 0.0040, 0.8630, 0.8020, 1.5400, 3.0100, 0.7760, 0.0040, 0.7620, 0.7470, 0.7610, 0.0040, 0.7910, 0.0040, 0.7760, 0.7560, 0.7640, 0.7880, 0.0040, 0.8140, 0.0040, 0.8150, 0.7800, 0.7560, 0.7830, 0.0040, 0.8010, 0.7720, 0.7560, 0.7800, 0.0050, 0.7960, 0.7840, 0.7600, 0.7690, 0.0040, 0.8110, 0.0040, 0.8590, 0.0050, 0.8350, 2.4050, 0.8200, 0.0040, 0.8140, 0.0050, 0.8090, 0.7890, 0.7740, 0.0030, 0.7480, 0.7420, 0.7650, 0.8110, 0.0040, 0.8350, 0.0040, 0.8330, 0.0040, 0.8280, 0.0040, 3.9230, 0.0040, 0.8390, 0.0040, 0.8430, 0.8110, 0.8050, 0.8060, 0.8230, 0.0040, 0.8260, 0.0040, 0.8330, 0.0040, 0.8250, 0.7930, 0.7810, 0.7840, 0.0050, 0.7990, 0.0050, 0.8130, 0.0050, 0.7910, 0.7950]

Thanks again!

[Uday Pradhan]

Hello Cai,

I ran your code with the data you have provided and the implementation seems to be correct. I cannot find anything wrong in that regard. Looking at indices, say idx, at which the values are acepted I can see most of these correspond to the positions where cycle_periods(idx) is 0.0040, hence this value repeats often in x. I believe the results align with the condtions you want to establish on the input data:

lowerBound = 0.5*(x(j - 1));
upperBound = 1.5*(x(j - 1));

Our first element is 0.0040, so that makes the upper and lower bounds : 0.0020 and 0.0060. Values whose weighted average falls in this range are only accepted. These bounds do not change for a few indices hence we see 0.0040 repeatedly getting accepted.

[Cai Chin]

Hi Uday, this makes so much sense now, I have removed the erroneously small values and the result is now what I wanted. Thank you very much for all your help!