Locate values in a matrix
Show older comments
I have multiple relatively large dataset.
each dataset represents loads on a ship hull, over a 24 hour period.
So the dataset is in matrix format, 17250922x11 table.
The first column is set to time, and the following 10 columns are values from different sensors on the ship hull.
My goal is to:
1) Graphically produce a readable and visually pleasing graph (or several) for when there are peak loads (whne the ship comes into contact with ice). I have managed to produce a graph, but it is very noisy and very compact. Maybe I should filter it?
2) Locate and display these peaks, as there are several of them in each column and each dataset. The values during these peaks will offcourse vary from the rest of the values due to heavy loads.
Anyway I am very new to MATlab so I struggle alot with this and will be grateful for any help.
Kindly
Kris
Answers (1)
Star Strider
on 11 Jul 2023
Edited: Star Strider
on 11 Jul 2023
1 vote
A reasonably general approach would be to use the Signal Processing Toolbox sgolayfilt function to smooth the data (there are oither ways as well, for example smoothdata) and then to locate the peaks, use the islocalmax function or the Signal ProcessingToolbox findpeaks function.
EDIT — (11 Jul 2023 at 16:43)
It would hellp to have the data, and a description of what you want to do with it.
I encourage you to use the datetime function instead of datenum to do date calculations, plotting, and other processing.
EDIT — (11 Jul 2023 at 17:41)
I understand about the data, however I cannot do anything specific without at least a sample of it. What I have already posted is essentially the best I can do.
5 Comments
Kristoffer
on 11 Jul 2023
Moved: Voss
on 11 Jul 2023
Kristoffer
on 11 Jul 2023
Star Strider
on 11 Jul 2023
Edited: Star Strider
on 12 Jul 2023
I am not certain how to define ‘peak load’ so here are some guesses —
% type('sample.txt')
% opts = detectImportOptions('sample.txt')
% v1 = getvaropts(opts,'Var1')
% opts = setvartype(opts,'Var1','datetime')
% opts = setvartype(opts,'Var2','datetime')
% opts = setvaropts(opts,'Var1','InputFormat','dd-MMM-uuuu')
% opts = setvartype(opts, 'Var2','datetime')
% opts = setvaropts(opts,'Var2','InputFormat','HH:mm:ss.SSSS')
T1 = readtable('sample.txt');
Date = datetime(T1.Var1, 'InputFormat','''''dd-MMM-uuuu');
Time = datetime(T1.Var2, 'InputFormat','HH:mm:ss.SSSS''''');
DateTime = Date + timeofday(Time);
DateTime.Format = 'dd-MMM-yyyy HH:mm:ss.SSSS';
T1 = addvars(T1, DateTime,'Before','Var1');
T1 = removevars(T1,{'Var1','Var2'});
T1 = renamevars(T1, compose('Var%d',3:12), compose('Var%d',1:10))
figure
plot(T1{:,1}, T1{:,2:end})
grid
title('Original')
VarsSGfilt = sgolayfilt(T1{:,2:end}, 3, 11); % Filter Noise
for k = 1:size(VarsSGfilt,2)
SEM = std(VarsSGfilt(:,k))/sqrt(numel(VarsSGfilt(:,k))); % Standard Error Of The MEan
VarCI95 = SEM*1.96 + mean(VarsSGfilt(:,k)); % +95% Confidence Interval For Data Set
[pks,locs] = findpeaks(VarsSGfilt(:,k), 'MinPeakProminence',VarCI95); % Detect & Store Peaks & Indices
Peaks{k} = pks;
Locs{k} = locs;
end
figure
plot(T1.DateTime, VarsSGfilt)
hold on
for k = 1:size(VarsSGfilt,2)
plot(T1.DateTime(Locs{k}), Peaks{k}, 'vr')
end
hold off
grid
title('Filtered')
figure
tiledlayout(5,2)
for k = 1:size(VarsSGfilt,2)
nexttile
plot(T1.DateTime, VarsSGfilt(:,k))
hold on
plot(T1.DateTime(Locs{k}), Peaks{k}, 'vr')
hold off
grid
title(sprintf('Var%d',k))
end
sgtitle('Filtered Plot Array')
This first filters the variables, and then uses the 95% confidence interval on the mean of each data set with 'MinPeakProminence' to define the peaks, since this varies with each signal. There are likely several ways to do this, and since this is quite outisde my areas of expertise, I defer to you for that. I used tiledlayout for the plot array. It has some advantages with respect to programming, however it also has limitations with respect to the individual axes. It you want more control over the axes, use subplot instead. The code is quite similar.
I had significant problems with the datetime variable, and finally gave up on importing it as a single variable with the options structure. It was difficult enough to import it and convert it as is.
I will help to refine this as desired.
EDIT — (12 Jul 2023 at 03:19)
Expanded explanation of the statistical calculation used to define the 'MinPeakProminence' value.
.
Kristoffer
on 12 Jul 2023
Star Strider
on 12 Jul 2023
Thank you!
Yes to both.
The column (variable) names should import automatically, and should appear in the table. (They were not part of ‘sample.txt’.)
Change ‘DateTime’ to ‘LocalTime’ wherever it appears, for consistency with the original file.
To get the variable names from the table, after you import it, create this variable:
VN = T1.Properties.VariableNames;
It is a cell array of the variable names, so you can then simply refer to them as ‘VN{1}’, ‘VN{2}’ and so forth, just as any other cell array. In the tileldlayout for loop the title call would then be:
title(VN{k})
You may need to experiment with it to get it to work as you want it to.
When you get the file imported as a table, one option to follow up with it here would be to use the writetable function to write perhaps the first 500 rows of it to a file, and then post that here as a follow-up to ‘sample.txt’.
For example:
writetable(T1(1:500,:), 'sampletable.txt')
That would make it easier for me to hellp you with it, if necessary.
.
Categories
Find more on Data Distribution Plots in Help Center and File Exchange
on 11 Jul 2023
on 12 Jul 2023
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
