Why is my GPU not detected by Parallel Computing Toolbox?

When I execute "gpuDevice" command in MATLAB command prompt on a machine with a supported GPU installed, I get the following error message:

ERROR: >> gpuDeviceError using gpuDevice (line 26)
No supported GPU device was found on this computer. To learn more about supported GPU devices, see
www.mathworks.com/gpudevice.

Answer 1

[MathWorks Support Team]

From R2024b onwards:

Run the validateGPU('all') command to perform a mix of system and GPU tests to confirm the GPUs in your machine can be used by MATLAB.

Pre-R2024b:

Run gpuDevice() to get the most helpful error message. gpuDeviceTable() and gpuDeviceCount() will not provide troubleshooting information.

Common Issues:

1. Check your GPU is supported

Check you have a supported GPU card, see NVIDIA website to check compute capability (https://developer.nvidia.com/cuda-gpus for older cards consult https://developer.nvidia.com/cuda-legacy-gpus). 

2. Use the latest NVIDIA graphics drivers for your card

Update your NVIDIA graphics drivers to their latest version for your card. It is not necessary to download the CUDA toolkit. https://www.nvidia.com/Download/index.aspx

3. Enable GPU Forward compatibility (From R2020b onwards)

If your GPU card is from a newer chipset architecture than your release of MATLAB you may need to enable GPU forward compatibility within MATLAB by using:

parallel.gpu.enableCUDAForwardCompatibility

4. Check if device is detectable by nvidia-smi:

If you still cannot access your CUDA enabled device from within MATLAB check whether the device is correctly identified by your operating system using a program like nvidia-smi. If it is not seek advice from your sysadmin or NVIDIA support.

To run nvidia-smi:

• Windows: nvidia-smi from a cmd window. Normally installed into C:\Windows\System32
• Linux: nvidia-smi from terminal window. Normally installed into /usr/bin/

5. Check if the device is in "Exclusive" mode and in use by another process:

By default CUDA enabled GPUs can be accessed by multiple processes at the same time. This is controlled by the NVIDIA setting of  "Compute Mode" selected for the card and would be visible in the nvidia-smi output from step 4. If this is set to "Exclusive" access to the GPU will be limited to a single process. This setup is most often encountered in computing clusters or shared machine environments.  If you are encountering issues with "Exclusive" mode seek advice from your sysadmin or NVIDIA support.

6. Check for device masking environment variables

In some environments such as computing clusters access to CUDA capable devices on machines are handled at a system level by techniques like control groups or general resource allocation. Checking for the presence of these systems can explain why GPUs are visible but restricted in access for other programs. This is more likely to exist on controlled HPC clusters.

CUDA_VISIBLE_DEVICES is an environment variable which will only allow access to the valid CUDA devices listed in the variable. The environment variable acts as a mask on any underlying CUDA devices. 

CUDA_VISIBLE_DEVICES = '0'          % Only device 0 is accessible and usable.
CUDA_VISIBLE_DEVICES = '0, 1'       % Devices 0 and 1 are accessible and usable.
CUDA_VISIBLE_DEVICES= 'NoDevApps'   % (or any invalid input) no CUDA devices are accessible.

If CUDA_VISIBLE_DEVICES is unset then all CUDA devices are unrestricted.

If the steps in this support solution have not helped you resolve the problem, please contact MathWorks support directly:

http://www.mathworks.com/support/contact_us/

 

[Walter Roberson]

There is the issue that each process or thread is associated with a single GPU device; if you have more processes or thread than you have GPUs then by default the extra are not associated with any GPU.

Manually selecting a GPU device within a process or thread is not recommended; doing so will reset the GPU, interrupting any current work. Sharing a single GPU between multiple processes or threads is really not recommended.

[Alison Eele]

Thanks Walter, I have updated this to include the edge case regarding GPUs set in 'Exclusive' compute mode by the driver.