Array exceeds maximum array size preference
78 views (last 30 days)
Hi, given the following
v = [1 2 3 4 1 1 1 1 1 2 3 2];
P = unique(perms(v),'rows');
Error using zeros
Requested 479001600x12 (42.8GB) array exceeds maximum array size preference. Creation of arrays
greater than this limit may take a long time and cause MATLAB to become unresponsive. See array
size limit or preference panel for more information.
Do you know how to fix the problem to get all the possible permutations? One idea was to divide in different cases. For example is the total nuber of permutations is 1000, is there a way to modify the command perms to get just the first 200 elements? Is there a more efficient way?
Hari Krishna Ravuri on 29 Jul 2019
I understand that, you are trying to generate all permutations of the array ‘v’ resulting in generating a vector of very large size, which exceeds maximum array size preference. To know more about the variable size preferences, please refer https://www.mathworks.com/help/matlab/matlab_env/set-workspace-and-variable-preferences.html.
As you have mentioned that you are trying to solve NP-complete problem and need all the permutations, I suggest you write a small script which generates all the permutations by rearranging the elements in the array itself i.e. with using constant memory space. Now, in that script, for the first Iteration, create a .mat file and write the permutation to the file and for the remaining iterations, you just need to append every permutation to the file. For writing data to .mat file you can use save function with the syntax save(filename, variables,'-append'). For more information, please refer https://www.mathworks.com/help/matlab/ref/save.html
By writing data to .mat file, you can access all the data without bringing the entire file into the main memory.
“is there a way to modify the command perms to get just the first 200 elements?”
No, as of now, there is no such way mentioned in the documentation. Please refer https://www.mathworks.com/help/matlab/ref/perms.html
“Is there a more efficient way?”
As you are solving an NP-complete problem and need all the permutations for sure to get the optimal solution, the best way is to write all the permutations to a file and accessing them, as the entire array will not fit in the main memory. To know more about handling mat files, please refer to https://blogs.mathworks.com/loren/2011/10/14/new-mat-file-functionality-in-r2011b/
If you need a good solution but not the optimal one, you can use the problem heuristics and try to get a good solution in less time.
Muniba Ashfaq on 11 Sep 2020
I was also getting this error. I deleted all unnecessary variables from workspace to free up memory. I just keep those variables that was needed for my MATLAB code execution.
This worked amazingly.