create multiple tables from a CSV in a for loop

hello everyone,
I have a large CSV file (too large to upload ~ 50 GB) that I stored in a table. the table columns are time,energy.
I'd like to save segments from this table into separate tables. for that I need the new table name to be changed in some way, lets say:
T=1000;
for i = 1:6
segment = find(old_table.time<T);
new_table(i) = old_table(segment,:); %note
T = T+1000;
end
the row with the note is what I'm trying to do but not sure how.

5 Comments

"for that I need the new table name to be changed in some way"
No, you don't need to change the table name.
Changing the table name would be a slow, more complex, and very inefficient approach.
The neat, simple, and very efficient solution would be to use basic MATLAB indexing and a cell array.
"the row with the note is what I'm trying to do but not sure how."
As an alternative you could just use one tall array: https://www.mathworks.com/help///matlab/large-files-and-big-data.html
thanks for your answer!
the problam is that when I'm trying to write it into a predetermined call array if fails since not all segments contain the same number of rows. I predetermined the row in the array to be 1E7 (which is more than the max segment I'm expecting).
the error is "unable to perform the assingment because the size of the left side in 1e7-by-2 and the right side is 56468-by-2"
I don't think you need to predetermine the segment size to store the table in cell array. You can just edit your code as follows.
new_table{i} = old_table(segment,:);
left_side(1:size(right_side,1), 1:size(right_side,2)) = right_side;
Thank you all, it solved my problam!

Sign in to comment.

Answers (0)

Categories

Asked:

on 14 Sep 2021

Commented:

on 19 Sep 2021

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!