CVX Solver: Exceeding Maximum Array Size with Large Dataset
8 views (last 30 days)
Show older comments
I am using the CVX solver to solve an optimisation problem with a dataset of input and output vectors of size 1400x1. When I take a subsample of 200 datapoints, this results in 40,002 equality constraints. Ideally I would use all 1400 datapoints for my problem, but I then receive the error:
Requested 1962801x1401 (20.5GB) array exceeds maximum array size preference (16.0GB). This might cause MATLAB to become unresponsive.
I've looked into "Preferences" > "Workspace" > "MatLab Array Size Limit" > checked off "Limit the maximum array size to a percentage of RAM" (already set to 100%).
However, I have seen other posts where people have worked with over 2 million equality constraints, albeit with a runtime of 40 minutes. Is this simply a problem where I need to use a computer with more memory and a better processor, or is there any way that I can modify the settings or the type of solver I use, even if the runtime may take longer?
1 Comment
Torsten
on 18 Mar 2025
Edited: Torsten
on 18 Mar 2025
It would help to know your optimization problem in a mathematical formulation and the usage of this 20.5 GB array in CVX.
Is your problem sparse ? If yes: can you exploit this sparsity in CVX ?
Maybe asking your question in the CVX forum has a better chance for getting a useful suggestion:
Answers (0)
See Also
Categories
Find more on Surrogate Optimization in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!