[BUG] WSL - No NVIDIA GPU detected but nvidia-smi works and shows device #680
Labels
Comments
adriantorrie
added
? - Needs Triage
|
Hi @adriantorrie, I'm not very familiar with WSL2, but I see that you are using a NVIDIA GeForce GTX 1080 Ti which is a Pascal generation GPU. Unfortunately, Pascal support was dropped starting with RAPIDS 24.02 (https://docs.rapids.ai/notices/rsn0034/). You could try a |
|
Cool, I'll give it a go. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
nvidia-smican see the device2.1. in Windows
2.2. in WSL
2.3. from within the running RapidsAI container (in WSL), via the
ipythonprompt2.4. from within the running RapidsAI container (in WSL), via the run command override
For step 1 above (from WSL):
$ docker run --gpus all --pull always --rm -it \ --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \ nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11 24.04-cuda12.2-py3.11: Pulling from nvidia/rapidsai/base Digest: sha256:a9ec3f43016242a11354abf70f545abdd9623239b0a3a1c9a1da65ddd75f3d55 Status: Image is up to date for nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11 This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.download.nvidia.com/licenses/NVIDIA_Deep_Learning_Container_License.pdf Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] Type 'copyright', 'credits' or 'license' for more information IPython 8.22.2 -- An enhanced Interactive Python. Type '?' for help. In [1]: import cudf /opt/conda/lib/python3.11/site-packages/cudf/utils/_ptxcompiler.py:61: UserWarning: Error getting driver and runtime versions: stdout: stderr: Traceback (most recent call last): File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 254, in ensure_initialized self.cuInit(0) File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 327, in safe_cuda_api_call self._check_ctypes_error(fname, retcode) File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 395, in _check_ctypes_error raise CudaAPIError(retcode, msg) numba.cuda.cudadrv.driver.CudaAPIError: [500] Call to cuInit results in CUDA_ERROR_NOT_FOUND During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<string>", line 4, in <module> File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 292, in __getattr__ self.ensure_initialized() File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 258, in ensure_initialized raise CudaSupportError(f"Error at driver init: {description}") numba.cuda.cudadrv.error.CudaSupportError: Error at driver init: Call to cuInit results in CUDA_ERROR_NOT_FOUND (500) Not patching Numba warnings.warn(msg, UserWarning) /opt/conda/lib/python3.11/site-packages/cudf/utils/gpu_utils.py:149: UserWarning: No NVIDIA GPU detected warnings.warn("No NVIDIA GPU detected")Steps/Code to reproduce bug
Note: For step 5 above, the Release Selector provided me with the following docker command:
docker run --gpus all --pull always --rm -it \ --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \ nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11Expected behavior
The expectation was that the following line of python would execute, and not raise an exception with "No NVIDIA GPU detected":
Environment details (please complete the following information):
Ubuntu-22.02andUbuntu-24.04Additional context
Ubuntu-22.04Ubuntu-24.04$ docker run --gpus all --pull always --rm -it \ --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \ nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11 24.04-cuda12.2-py3.11: Pulling from nvidia/rapidsai/base 24.04-cuda12.2-py3.11: Pulling from nvidia/rapidsai/base Digest: sha256:a9ec3f43016242a11354abf70f545abdd9623239b0a3a1c9a1da65ddd75f3d55 Status: Image is up to date for nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11 This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.download.nvidia.com/licenses/NVIDIA_Deep_Learning_Container_License.pdf Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] Type 'copyright', 'credits' or 'license' for more information IPython 8.22.2 -- An enhanced Interactive Python. Type '?' for help. In [1]: import cudf /opt/conda/lib/python3.11/site-packages/cudf/utils/_ptxcompiler.py:61: UserWarning: Error getting driver and runtime versions: stdout: stderr: Traceback (most recent call last): File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 254, in ensure_initialized self.cuInit(0) File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 327, in safe_cuda_api_call self._check_ctypes_error(fname, retcode) File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 395, in _check_ctypes_error raise CudaAPIError(retcode, msg) numba.cuda.cudadrv.driver.CudaAPIError: [500] Call to cuInit results in CUDA_ERROR_NOT_FOUND During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<string>", line 4, in <module> File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 292, in __getattr__ self.ensure_initialized() File "/opt/conda/lib/python3.11/site-packages/numba/cuda/cudadrv/driver.py", line 258, in ensure_initialized raise CudaSupportError(f"Error at driver init: {description}") numba.cuda.cudadrv.error.CudaSupportError: Error at driver init: Call to cuInit results in CUDA_ERROR_NOT_FOUND (500) Not patching Numba warnings.warn(msg, UserWarning) /opt/conda/lib/python3.11/site-packages/cudf/utils/gpu_utils.py:149: UserWarning: No NVIDIA GPU detected warnings.warn("No NVIDIA GPU detected") In [2]: !nvidia-smi Sat Jun 1 14:22:59 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 555.42.03 Driver Version: 555.85 CUDA Version: 12.5 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce GTX 1080 Ti On | 00000000:23:00.0 On | N/A | | 40% 25C P8 17W / 250W | 1632MiB / 11264MiB | 3% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 33 G /Xwayland N/A | | 0 N/A N/A 38 G /Xwayland N/A | | 0 N/A N/A 45 G /Xwayland N/A | | 0 N/A N/A 62 G /Xwayland N/A | +-----------------------------------------------------------------------------------------+Ubuntu-22.04andUbuntu-24.04):docker run --gpus all --pull always --rm -it \ --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 \ nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11 nvidia-smi 24.04-cuda12.2-py3.11: Pulling from nvidia/rapidsai/base Digest: sha256:a9ec3f43016242a11354abf70f545abdd9623239b0a3a1c9a1da65ddd75f3d55 Status: Image is up to date for nvcr.io/nvidia/rapidsai/base:24.04-cuda12.2-py3.11 This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.download.nvidia.com/licenses/NVIDIA_Deep_Learning_Container_License.pdf Sat Jun 1 14:36:23 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 555.42.03 Driver Version: 555.85 CUDA Version: 12.5 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce GTX 1080 Ti On | 00000000:23:00.0 On | N/A | | 40% 24C P8 22W / 250W | 1637MiB / 11264MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 33 G /Xwayland N/A | | 0 N/A N/A 38 G /Xwayland N/A | | 0 N/A N/A 45 G /Xwayland N/A | | 0 N/A N/A 62 G /Xwayland N/A | +-----------------------------------------------------------------------------------------+The text was updated successfully, but these errors were encountered: