DensifyPointCloud: error while loading shared libraries: libcuda.so.1

[sean-kenops]

What is the problem?

DensifyPointCloud: error while loading shared libraries: libcuda.so.1

What should be the expected behavior?

libcuda.so.1 should be found in the docker image.

Looks like the expected path is missing in the image. Adding this path corrects the issue.

mkdir /usr/local/nvidia
ln -s /usr/local/cuda-11.2/compat /usr/local/nvidia/lib
ls -l /usr/local/nvidia/lib/

How can we reproduce this? (What steps did you do to trigger the problem? Be detailed)

Further details are located in the forum: https://community.opendronemap.org/t/densifypointcloud-error-while-loading-shared-libraries-libcuda-so-1/20178/1

[INFO] Running dense reconstruction. This might take a while.
[INFO] Estimating depthmaps
[INFO] running "/code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud" "/var/www/data/254e5ea9-97b8-4129-a552-56a9218aed51/submodels/submodel_0000/opensfm/undistorted/openmvs/scene.mvs" --resolution-level 2 --dense-config-file "/var/www/data/254e5ea9-97b8-4129-a552-56a9218aed51/submodels/submodel_0000/opensfm/undistorted/openmvs/Densify.ini" --max-resolution 4032 --max-threads 8 --number-views-fuse 2 --sub-resolution-levels 2 --archive-type 3 -w "/var/www/data/254e5ea9-97b8-4129-a552-56a9218aed51/submodels/submodel_0000/opensfm/undistorted/openmvs/depthmaps" -v 0 --cuda-device -2
/code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

[sean-kenops]

This may be from a poorly formed LD_LIBRARY_PATH variable in the container.

[lupionpe]

Any updates here?

[pierotofy]

It should have been fixed. Update and report back if it's still an issue? 🙏

[lupionpe]

I am still getting the same issue:

...
[INFO]    running "/code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud" "/var/www/data/a40c3d2a-36a3-4b82-91d4-6bfe48d7b02f/opensfm/undistorted/openmvs/scene.mvs" --resolution-level 1 --dense-config-file "/var/www/data/a40c3d2a-36a3-4b82-91d4-6bfe48d7b02f/opensfm/undistorted/openmvs/Densify.ini" --max-resolution 5280 --max-threads 32 --number-views-fuse 2 --sub-resolution-levels 2 --archive-type 3 -w "/var/www/data/a40c3d2a-36a3-4b82-91d4-6bfe48d7b02f/opensfm/undistorted/openmvs/depthmaps" -v 0 --cuda-device -2
/code/SuperBuild/install/bin/OpenMVS/DensifyPointCloud: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory
...

console.txt

My NodeODM:

[pierotofy]

How was the container launched? Does it work with:

docker pull opendronemap/nodeodm:gpu
docker run -ti --rm -p 3000:3000 --gpus all opendronemap/nodeodm:gpu

?

[lupionpe]

How was the container launched? Does it work with:

docker pull opendronemap/nodeodm:gpu
docker run -ti --rm -p 3000:3000 --gpus all opendronemap/nodeodm:gpu

?

Using the command:

docker run -p 3000:3000 -q 1 --max_images 5500 --cleanup_uploads_after 60 --cleanup_tasks_after 1440 --gpus all opendronemap/nodeodm:gpu

[pierotofy]

That's not a valid docker run command? The args to NodeODM have to be passed after opendronemap/nodeodm:gpu.

If you run docker run -ti --rm --entrypoint bash --gpus all opendronemap/nodeodm:gpu -c "/usr/bin/nvidia-smi" you should get:

docker run -ti --rm --entrypoint bash --gpus all opendronemap/nodeodm:gpu -c "/usr/bin/nvidia-smi"
Fri Feb 28 19:31:24 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.02              Driver Version: 560.94         CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        On  |   00000000:01:00.0  On |                  N/A |
|  0%   51C    P8             31W /  350W |     588MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A        28      G   /Xwayland                                   N/A      |
|    0   N/A  N/A        39      G   /Xwayland                                   N/A      |
+-----------------------------------------------------------------------------------------+