merge many mat files in one big vector

Hi! I have many mat files which i would like to merge in one vector. these mat files have all the same length (psd(65536,1)), since i got 500 of them, i just create first a vector zeros(500*65536,1), and write in a for loop, some lines so that my new psd_all=[psd1;psd2;psd3;...psdn]..I dont know why but my code is since yesterday still am running... here is my code:

close all
clear all
%Spectrogram für einen Monat of PSDs Daten

clc
tic

dt=0.02;
s_rate=200;
% load tc
Path='C:\Masterarbeit_code\LaacherSee_DataAnalyse\A01_01\hhz';     %Path=D:\LaacherSee_Projekt\AllDateien\__MiniSeeds_Laacher_See_2021\1_A01\Cube-Dateien\_Stunden';
P=strcat(Path,'\PSDs\1220\')     %P=strcat(Path,'\hhe\06\');
S=(fullfile(P,'*.mat'));
AllFiles=dir(S);
mo=1220   %Monat
f1=65536; %length PSD Vektor
f2=numel(AllFiles);
f3=f1*f2;
PSD_Monat=zeros(f3,1);


for k=1:numel(AllFiles)      %144:586 %:numel(AllFiles)
    File=AllFiles(k).name;
    Data = load(strcat(P,File));
    % X=rdmseed(strcat(P,File)); %('AllFiles(k).name'); 
    % d = load(strcat(P,File));   %cat(1,X.d);
    % Data =load(fullfile(P,'PSDwert' num2str(k) '.mat');
    d=Data.PSD_real1;
    
    for k=1
    PSD_Monat([1:k*f1],1)=d(:,1)   %d([1:k*f1],1);
    end
    for k=2:numel(AllFiles)
        PSD_Monat([(k-1)*f1+1:k*f1],1)=d(:,1);
    end
end

toc

i think something is not working on my code, can someone help me on that?

Answer 1

[Jan]

Why do you overwrite the first part of PSD_Monat in each iteration?

 for k=1
    PSD_Monat([1:k*f1],1)=d(:,1)   %d([1:k*f1],1);
 end

This is a waste of time.

Including the indices in q´square brackets wastes time also: X(a:b) replies the result without creating the vector a:b explicitly. This requires tests, if a and b are inside the allowed range for only these 2 indices.

In opposite to this, X([a:b]) creates the index vector explicitly and Matlab has to check each element of it, if it is a valid index.

This looks even stranger:

    for k=2:numel(AllFiles)
        PSD_Monat([(k-1)*f1+1:k*f1],1)=d(:,1);
    end    

You overwrite the complete output with the current data. But why?

Your code write 500*500*65536*8 Bytes. Omit the overwriting:

Path = 'C:\Masterarbeit_code\LaacherSee_DataAnalyse\A01_01\hhz';
P = fullfile(Path, '\PSDs\1220\');
AllFiles = dir(fullfile(P,'*.mat'));
f1 = 65536;
f2 = numel(AllFiles);
PSD_Monat = zeros(f1, f2);   % Easier
for k = 1:numel(AllFiles)
    File = AllFiles(k).name;
    Data = load(fullfile(P, File));
    PSD_Monat(:, k) = Data.PSD_real1(:);
end
PSD_Monat = PSD_Monat(:);   % Make it a column vector afterwards

BY the way, clear all does not have benefits. It deletes all functions from the RAM and reloading them from the slow disk wastes time only. This is "crago cult programming" and should be avoided. If you want to keep the workspace clean, use functions instead of scripts.