Greetings! I was working with the Altimetry Time Series data that contained outliers.I want to remove them with Tailored Outlier Method. Plz help me with the related script.

I want to perform the Tailored outlier removal methods and want to generate graphs having strips for upper and lower limits, as shown in figure. I have also performed some basic outlier removal procedures in Excel, which are not showing better results for the scattered outliers in Radar Altimetry time series.

Answer 1

[Simran]

Hi @Jasra rehman,

I understand that you want to perform the tailored outlier removal method and want to generate graphs having strips for upper and lower limits. You can follow the below steps to do so:

1.) Load your altimetry time series data, which could be your latitude or time data, elevation data.

latitude = ...; % Your latitude or time data 
    elevations = ...; % Your elevation data 

2.) You can then use a histogram to determine the mode of the elevation data.

3.) Then do initial outlier removal and remove data points outside a specified range from the mode. You can follow the below code script to do so.

initialRange = 1; % Adjust as needed 
inliers = elevations(abs(elevations - modeValue) <= initialRange); 

4.) Next, calculate the mean and RMS and iteratively remove further outliers. The below code script explain how to do it.

thresholdPercentage = 0.1; % Adjust as needed 
while length(inliers) > thresholdPercentage * length(elevations) 
    meanLevel = mean(inliers); 
    rmsValue = sqrt(mean((inliers - meanLevel).^2)); 
    inliers = inliers(abs(inliers - meanLevel) <= rmsValue); 
end 

5.) Then plot the original and filtered data with upper and lower limit strips.

figure; 
hold on; 
scatter(latitude, elevations, 'r', 'DisplayName', 'Original Data'); 
scatter(latitude(ismember(elevations, inliers)), inliers, 'b', 'DisplayName', 'Filtered Data'); 
yline(meanLevel + rmsValue, '--g', 'DisplayName', 'Upper Limit'); 
yline(meanLevel - rmsValue, '--g', 'DisplayName', 'Lower Limit'); 
xlabel('Latitude'); 
ylabel('Elevation'); 
legend('show'); 
title('Tailored Outlier Removal with Upper and Lower Limits'); 
hold off; 

I used some example data and got this graph where you can clearly see the original data and the filtered data, and upper and lower limit:

You can refer to the following documentation for more help:

“histcounts” - https://www.mathworks.com/help/releases/R2021a/matlab/ref/histcounts.html

“sqrt” -

https://www.mathworks.com/help/releases/R2021a/matlab/ref/sqrt.html

“mean” -

https://www.mathworks.com/help/releases/R2021a/matlab/ref/mean.html

Hope this helps!