Preallocating table makes it slower?

I have to add data to a table with a loop. I can estimate how large the table approx will be at the end. But i dont know the exact size. I also dont know the exact size of the data added with every iteration. I notized that the needed time grows nearly exponetialy when using vertcat. So i tried to preallocate the table. It made things worse!

Can some one help me out?

thanks in advance

Christian

Wrote some code to demostrate the problem:

rows = 1e6;
columns = 6;
aproxRows = rows * 1.5;


Table= table();
preallTable = array2table(zeros(aproxRows, columns));

preallTime = []; 
preallTime(1) = 0;
Time = []; 
Time(1) = 0;
start = 1;

for  i = 1:1e3
    i
    
    addTable = array2table(rand(1000, columns)); %size of table is unknown in reality for every loop pass
    
    End = start-1 + height(addTable);
    
    preallTic = tic;
    preallTable(start:End, :) = addTable;           
    preallTime(end+1) = preallTime(end) + toc(preallTic);
    
    start = End+1;   
    
    Tic = tic;
    Table = vertcat(Table,addTable);
    Time(end+1) = Time(end) + toc(Tic);  
    
    
end    

%cut off unused space
preallTable(start:end, :) = [];

%plot Time
figure
hold on
plot(preallTime)
plot(Time)
legend({'preallTime', 'Time'})

Answer 1

[Star Strider]

I would preallocate the array, then do the array2table call at the end, after the array was created and is stable (nothing more to be added to it).

[Christian Tieber]

The data to add, comes as a table. Not shure if a can change that.

[Star Strider]

Youi can easily change it to an array using the table2array function. If the variable names in the table are important, you can extract them with:

VarNames = T.Properties.VariableNames;

where ‘T’ is the original table name. When you have finished changing the array, you can the re-define the variable names in the new table (created with array2table) using ‘VarNames’ as the cell array value for the 'VariableNames' name-value pair (providing that the number of the variable names has not changed from the original table).