How to convert 30s daily data into hourly data using MATLAB?

I have a daily 30s file.I need to convert it into hourly basis and plot the data on hourly basis. The file consists of 5 coloumns and 2858 rows. First coloumn is the time in second, second column is the second to hour conversion and all other columns are the error measurements(3,4,5).I have to plot each column data(3,4,5) in 24 hour duration with a time scale of 4hrs.I'm using old version of MATLAB.How can I combine these values and plot this.Can anyone please suggest me a solution.

[Dyuman Joshi]

"First coloumn is the time in second, second column is the second to hour conversion ..."

How does the conversion work? Just multiply times in seconds with seconds to hour conversion?

What would be the context of the data obtained?

"I have to plot each column data(3,4,5) in 24 hour duration with a time scale of 4hrs"

What do you mean by this?

How is the data obtained related to this?

[Aiswarya]

Thank you for the response. I changed the data time format to hh:mm:ss. The latest csv file consist for time in first column and the corresponding errors in the next columns. I have to plot the 30s data series on hourly basis in 4hrs interval and average the data for hours and plot it.

Answer 1

[Star Strider]

It would be nice to know what version you are using. Are you supposed to accumulate (for example take the mean) of the data over 4 hours, or just plot the data at 4-hour intervals, or something else?

This is straightforward using a timetable and the retime function —

T1 = readtable('sample_hourly_...estdata_E.csv')

T1 = 2858×5 table
    Var1      Var2        Var3      Var4       Var5 
    ____    _________    ______    _______    ______

     30     0.0083333    5.9288    0.96482     4.913
     60      0.016667    6.2877    0.97484    4.7815
     90         0.025    5.9988       1.06    4.8505
    120      0.033333    5.6999     1.0159    5.6315
    150      0.041667    5.7833     1.0239    4.7822
    180          0.05    6.0166    0.75248    4.7387
    210      0.058333    5.9688    0.97885    4.8818
    240      0.066667    6.0499    0.87468    4.9862
    270         0.075    5.7744     1.2132    4.3773
    300      0.083333     5.541    0.96182    4.9297
    330      0.091667     5.871    0.89271    5.0361
    360           0.1    5.8955     1.2292    4.8141
    390       0.10833    5.5332     1.3695    5.2932
    420       0.11667    6.1322     1.1832    5.1075
    450         0.125    6.5078    0.99988    3.9719
    480       0.13333    5.7188     1.1411    4.9198

Time = datetime([2024 01 01]) + seconds(T1{:,1});
T2 = [table(Time) T1(:,3:5)];
T2.Time.Format = 'HH:mm:ss'

T2 = 2858×4 table
      Time       Var3      Var4       Var5 
    ________    ______    _______    ______

    00:00:30    5.9288    0.96482     4.913
    00:01:00    6.2877    0.97484    4.7815
    00:01:30    5.9988       1.06    4.8505
    00:02:00    5.6999     1.0159    5.6315
    00:02:30    5.7833     1.0239    4.7822
    00:03:00    6.0166    0.75248    4.7387
    00:03:30    5.9688    0.97885    4.8818
    00:04:00    6.0499    0.87468    4.9862
    00:04:30    5.7744     1.2132    4.3773
    00:05:00     5.541    0.96182    4.9297
    00:05:30     5.871    0.89271    5.0361
    00:06:00    5.8955     1.2292    4.8141
    00:06:30    5.5332     1.3695    5.2932
    00:07:00    6.1322     1.1832    5.1075
    00:07:30    6.5078    0.99988    3.9719
    00:08:00    5.7188     1.1411    4.9198

VN = T2.Properties.VariableNames;

q4h1 = (rem(hour(T2.Time), 4)  == 0) & (minute(T2.Time) == 0) & (second(T2.Time) == 0);
T2(q4h1,:)

ans = 5×4 table
      Time       Var3        Var4       Var5  
    ________    _______    ________    _______

    04:00:00     2.0807      1.8583    -1.0893
    08:00:00    -3.8786      6.5277      -11.2
    12:00:00    -6.4366    -0.82104    -7.0585
    16:00:00     2.1863    -0.14896     4.6413
    20:00:00      5.141     -0.4865     6.2848

% nnz(q4h)

figure
plot(T2{q4h1,1}, T2{q4h1,2:end})
grid
xlabel('Time')
ylabel('Value')
title('Every 4 Hour Values')
legend(VN{2:end}, 'Location','best')

q4h2 = (rem(hour(T2.Time), 4)  == 0) & (minute(T2.Time) == 0) & (second(T2.Time) == 30);
T2(q4h2,:)

ans = 6×4 table
      Time       Var3        Var4       Var5  
    ________    _______    ________    _______

    00:00:30     5.9288     0.96482      4.913
    04:00:30     2.2096      2.0275    -0.8845
    08:00:30    -3.7675      6.6449    -11.714
    12:00:30    -7.0522    -0.76595    -6.5867
    16:00:30      2.143     0.28073     4.6995
    20:00:30     4.6265    -0.56563     5.4408

% nnz(q4h)

figure
plot(T2{q4h2,1}, T2{q4h2,2:end})
grid
xlabel('Time')
ylabel('Value')
title('Every 4 Hour Values + 30 Seconds')
legend(VN{2:end}, 'Location','best')

TT2 = table2timetable(T2);
TT2.Time.Format = 'HH:mm:ss';
TT2r = retime(TT2, 'regular','mean','TimeStep',hours(4))

TT2r = 6×3 timetable
      Time       Var3        Var4        Var5  
    ________    _______    _________    _______

    00:00:00     4.6571      0.40753     3.0897
    04:00:00     -2.852       2.2892    -6.5746
    08:00:00    -5.5369      -1.0768    -7.5661
    12:00:00    -5.6504      -1.4982     1.6178
    16:00:00     3.5855     -0.10615     3.7615
    20:00:00     6.0721    -0.015406      5.938

figure
plot(TT2r.Time, TT2r{:,1:end})
grid
xlabel('Time')
ylabel('Value')
title('Meaned 4 Hour Values (‘retime’)')
legend(VN{2:end}, 'Location','best')

.

[Aiswarya]

Dear Sir,

Thank you for the assistance.I'm using Matlab2015b.I'm facing issues while loading the CSV file using readtable command.I also converted the time format to hh:mm:ss.The new csv file is attached here.I need separate plot for each error value(east,north,up) with respect to time in hour.The sample plot is attached here.Two types of plot is required 30s data into hourly average and simple 4hr interval plot. I have huge file of datasets like this.I'm working on it.Can you please suggest me a solution.

[Star Strider]

I am not certain what you want.

Making appropriate changes to my previous code with your new file —

T1 = readtable('sample_data_shlg.csv', 'VariableNamingRule','preserve')

T1 = 2858×4 table
      GPST      East Error    North Error    UP Error
    ________    __________    ___________    ________

    00:00:00      5.9288        0.96482        4.913 
    00:00:30      6.2877        0.97484       4.7815 
    00:01:00      5.9988           1.06       4.8505 
    00:01:30      5.6999         1.0159       5.6315 
    00:02:00      5.7833         1.0239       4.7822 
    00:02:30      6.0166        0.75248       4.7387 
    00:03:00      5.9688        0.97885       4.8818 
    00:03:30      6.0499        0.87468       4.9862 
    00:04:00      5.7744         1.2132       4.3773 
    00:04:30       5.541        0.96182       4.9297 
    00:05:00       5.871        0.89271       5.0361 
    00:05:30      5.8955         1.2292       4.8141 
    00:06:00      5.5332         1.3695       5.2932 
    00:06:30      6.1322         1.1832       5.1075 
    00:07:00      6.5078        0.99988       3.9719 
    00:07:30      5.7188         1.1411       4.9198 

% Time = datetime([2024 01 01]) + seconds(T1{:,1});
% T2 = [table(Time) T1(:,3:5)];
% T2.Time.Format = 'HH:mm:ss'
VN = T1.Properties.VariableNames;

[h,m,s] = hms(T1.GPST);
q4h1 = (rem(h, 4)  == 0) & (m == 0) & (s == 0);
T1(q4h1,:)

ans = 6×4 table
      GPST      East Error    North Error    UP Error
    ________    __________    ___________    ________

    00:00:00      5.9288        0.96482        4.913 
    04:00:00      2.2096         2.0275      -0.8845 
    08:00:00     -3.7675         6.6449      -11.714 
    12:00:00     -7.0522       -0.76595      -6.5867 
    16:00:00       2.143        0.28073       4.6995 
    20:00:00      4.6265       -0.56563       5.4408 

% nnz(q4h)

figure
plot(T1{q4h1,1}, T1{q4h1,2:end})
grid
xlabel('Time')
ylabel('Value')
title('Every 4 Hour Values')
legend(VN{2:end}, 'Location','best')

q4h2 = (rem(h, 4)  == 0) & (m == 0) & (s == 30);
T1(q4h2,:)

ans = 6×4 table
      GPST      East Error    North Error    UP Error 
    ________    __________    ___________    _________

    00:00:30      6.2877        0.97484         4.7815
    04:00:30      2.1796         1.9073      -0.068799
    08:00:30     -3.6608         6.8412        -12.339
    12:00:30     -6.7611       -0.84708         -6.695
    16:00:30      1.6852       0.015303         5.0054
    20:00:30       4.901       -0.32725         6.8059

% nnz(q4h)

figure
plot(T1.GPST(q4h2), T1{q4h2,2:end})
grid
xlabel('Time')
ylabel('Value')
title('Every 4 Hour Values + 30 Seconds')
legend(VN{2:end}, 'Location','best')

TT1 = table2timetable(T1);
% TT1.GPST.Format = 'HH:mm:ss';

TT1r1 = retime(TT1, 'hourly','mean')

TT1r1 = 24×3 timetable
      GPST      East Error    North Error    UP Error
    ________    __________    ___________    ________

    00:00:00       5.6884        0.43172      4.8334 
    01:00:00        5.515         0.1972       3.794 
    02:00:00       4.5204      -0.082578      3.3626 
    03:00:00        2.883         1.0959     0.33387 
    04:00:00     -0.87624         1.9189     -5.0525 
    05:00:00      -1.1392         2.0974      -3.468 
    06:00:00      -4.2934         1.5156     -4.1435 
    07:00:00      -5.1489         3.6637     -13.719 
    08:00:00      -5.7427          2.207     -11.422 
    09:00:00      -5.8088        -1.2876     -1.5988 
    10:00:00      -4.6776        -1.0911     -6.7865 
    11:00:00      -5.9398        -4.1967     -10.423 
    12:00:00      -8.1786        0.69842      3.5836 
    13:00:00      -7.9199       -0.31029      5.8302 
    14:00:00      -5.8264        -5.0859     -7.2518 
    15:00:00     -0.60508        -1.2893      4.4066 

figure
plot(TT1r1.GPST, TT1r1{:,1:end})
grid
xlabel('Time')
ylabel('Value')
title('Meaned Hourly Values (‘retime’)')
legend(VN{2:end}, 'Location','best')

TT1r2 = retime(TT1, 'regular','mean','TimeStep',hours(4))

TT1r2 = 6×3 timetable
      GPST      East Error    North Error    UP Error
    ________    __________    ___________    ________

    00:00:00      4.6517         0.41056       3.081 
    04:00:00     -2.8644          2.2989     -6.5957 
    08:00:00     -5.5422         -1.0921     -7.5575 
    12:00:00     -5.6325         -1.4968      1.6421 
    16:00:00      3.5916        -0.10685      3.7649 
    20:00:00      6.0742       -0.014377      5.9372 

figure
plot(TT1r2.GPST, TT1r2{:,1:end})
grid
xlabel('Time')
ylabel('Value')
title('Meaned 4 Hour Values (‘retime’)')
legend(VN{2:end}, 'Location','best')

Please post back to clarify any changes needed to get what you want.

.

[Aiswarya]

Dear Sir,

Sorry for the late reply. Thank you for the continuous assistance. I'm facing some issues while loading the CSV file using readtable command in Matlab2015b. But I try these codes in another version. It's working fine.

[Star Strider]

My pleasure!

If my Answer helped you solve your problem, please Accept it!

In R2015b, you problably need to use ( 'PreserveVariableNames',true ). I believe that changed in R2020a (although I am not certain about the exact version).

.

Answer 2

[Dinesh]

Hi Aiswarya,

To plot your data on an hourly basis for each error column, you'll need to group the 30 second interval data into hourly averages and then plot each error measurement. The following MATLAB snippet demonstrates how you might average one of the error measurements (3rd column from your data) over each hour. Repeat the process for the other error columns as well.

% Load "data" from your CSV file
timeHours = ceil(data(:, 2)); % Convert to hourly
error1HourlyAvg = accumarray(timeHours, data(:, 3), [], @mean); % Averages for error 1

plot(error1HourlyAvg);

The following link is the documenation for accumarray:

https://www.mathworks.com/help/matlab/ref/accumarray.html

[Aiswarya]

Thank you for the suggestion.

Answer 3

[Alexander]

Hi Aiswarya,

I also can only assume your intention. My solution would be:

clear

data = dlmread('sample_hourly_shlg_testdata_E.txt',',');

plot(data(:,2), data(:,3:5));grid minor;

xticks([0 4 8 12 16 20 24]); % scale to 4 hours

xlabel('Time [h]');ylabel('your dimension');title('Your Title')

legend('M1','M2','M3');

Hopefully it helps.

[Aiswarya]

Hi Alexander,

Thank you for the answer.I tried it, but not working.I changed the format to hh:mm:ss.The latest csv file is attached here. I need separate plot for each error values (east, north and up).Two types of plots are required 30s data into hourly average and just plot the data in 4hr interval.

[Alexander]

Just for my understanding, what means "not working". Do you get an error message? Or does the code didn't meet you expectations?

[Alexander]

Are you aware, that you are loosing about 12 min of data between 2623 and 2624?