How to create datastore from cell array containing a lot of data?

27 views (last 30 days)
I am trying to run an LSTM code with a frequency domain input with a cell array that is 184 x 1 called freqNorm. Each cell is a vector array of size 1 x 100001. When I try to run my code I get the following error:
Requested 400x127x100001 (18.9GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive.
I read that I should create a datastore but I am not quite sure how to do this. The input data needs to still be in a 184 x 1 cell array. I tried using the following code, but i kept getting an error (as seen below the code).
save('freqNorm.mat','freqNorm');
freqNorm = datastore('C:\Users\Kim Cardillo\Desktop\Research\PRTools_Matlab\AEsignals_4to48kc_3.5-8mm\freqNorm.mat');
Error using datastore (line 125)
Cannot determine the datastore type for the specified location.
Specify the 'Type' name-value pair argument to indicate the type of datastore to create.
I am not quite sure what the 'Type' is nor do I know if what I am trying to do is correct. Can someone please help me.

Accepted Answer

Devineni Aslesha
Devineni Aslesha on 24 Aug 2020
Hi Kimberly,
Please use the below code to create datastore from cell array freqNorm.
tA = tall(freqNorm);
write('C:\Users\Kim Cardillo\Desktop\Research\PRTools_Matlab\AEsignals_4to48kc_3.5-8mm\freqNorm.mat', tA);
tds = datastore('C:\Users\Kim Cardillo\Desktop\Research\PRTools_Matlab\AEsignals_4to48kc_3.5-8mm\freqNorm.mat','Type','tall');
For more information, refer the following links,
  2 Comments
Kimberly Cardillo
Kimberly Cardillo on 26 Aug 2020
Thank you. When I ran the code you posted, I got this error:
Invalid write location: C:\Users\Kim Cardillo\Desktop\Research\PRTools_Matlab\AEsignals_4to48kc_3.5-8mm\freqNorm.mat
Folder must either be empty or non-existing.
What does that mean?
Devineni Aslesha
Devineni Aslesha on 27 Aug 2020
Hi Kimberly,
In the write doc, you can see that the location should be
  • Existing empty folder
  • New folder that write creates
In your case, freqNorm.mat should be empty if it is already existing.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!