GPU fft encounters error

1 view (last 30 days)
Jingzhao Zhang
Jingzhao Zhang on 20 Mar 2015
Commented: Joss Knight on 1 Apr 2015
I am using gpu parallel to accelerate computation with Tesla 20, Matlab2015a. However, the program always encounters error at the same gpu operation. Below is the error message:
Error using gpuArray/fft2
MATLAB encountered an unexpected error in evaluation on the GPU. If you continue to see this error, reset the GPU device or restart MATLAB and try again.
I guess that this results from limited gpu memory since when I smaller the data size, the program runs smoothly. However, when the program breaks, I find that there are still 3G space available according to gpuDevice. What could be the problem? Does the system only grants Matlab a limited portion of gpu memory? How can I solve it? Thanks,
  5 Comments
Saad Mufti
Saad Mufti on 1 Apr 2015
I've had similar problems with a simple test program. To benchmark my gpu (same as OP), I steadily increased the array size from 2000 to 10,000. At 10,000 Matlab returned the same error as reported above. However, the puzzling thing is that restarting Matlab lets me get on with more GPU work, but resetting the GPU does not improve the sotuation - still get the same error...
Joss Knight
Joss Knight on 1 Apr 2015
There is a bug in the cuFFT library which causes it to go into an unrecoverable state after a GPU allocation failure in certain circumstances. What I would expect to see, if you are hitting this bug, is first an error which tells you there was an allocation failure, and then from then on, no matter what you do, any call to fft returns error code EXEC_FAILED. Is this what you are seeing? You can continue using everything else in MATLAB, you just cannot do FFTs on the GPU until you restart MATLAB.
Unfortunately the workaround for this for the time being is to avoid such allocation failures by being careful with your resources. You can only recover fully from this problem by restarting MATLAB. Many apologies for this.

Sign in to comment.

Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!