Add all directories configured in PATH with Windows and Python 3.9

[giohappy]

Expected behavior and actual behavior.

As a solution for #851 Fiona uses add_gdal_dll_directories to add the DLL directories from ./libs.
This doesn't work for wheels like these. In cases like this, the ./libs folder is not bundled and the GDAL binaries are bundled inside another wheel.

I wonder if this logic could be extended to add all the directories configured within the PATH env var, also considering that the libs folder is also automatically added to the PATH here.

With the following lines I was able to load fiona without errors:

import osgeo

#export PATH=%PATH%;<path to directory with GDAL DLLs>
os.environ["PATH"] = os.getenv('PATH') + ";" + osgeo.__path__[0]
paths = os.getenv('PATH').split(";")
for path in paths:
    try:
        os.add_dll_directory(path)
    except Exception as e:
        print(e)

import fiona

Operating system

Windows 10

Fiona and GDAL version and provenance

GDAL‑3.4.1‑cp39‑cp39‑win_amd64.whl
Fiona‑1.8.21‑cp39‑cp39‑win_amd64.whl

obtained from https://www.lfd.uci.edu/~gohlke/pythonlibs/

[rbuffat]

The wheels https://www.lfd.uci.edu/~gohlke/pythonlibs/ are not created by the Fiona project. It would be best to get in contact with the author of these wheels for packaging-related issues.

Afaik they contain a patch to load GDAL from osgeo (In fiona/init.py of the wheels):

# Unofficial Windows Binaries:
# Use gdal DLLs and data files from the osgeo package
try:
    import numpy
    import osgeo
except ImportError:
    pass

[giohappy]

@rbuffat I know that it's a specific case, but I think that having a generic way to load the DLLs from PATH could be beneficial to similar generic cases, without hurting Fiona.

The custom patch inside the wheel fails silently because the wheel doesn't install numpy. I will report this.
Actually, by importing osgeo first there's no need need to call os.add_dll_directory.
Anyway my proposal goes beyond this specific case.

[rbuffat]

@giohappy What use case do you have in mind? There are two binary wheels provider for Windows. Gohlke and conda-forge. Both have their own mechanism to load the GDAL dlls. The code in _loading.py is mainly used in the Windows CI. If somebody compiles Fiona on Windows on its own they can place the dlls in the libs folder. import osgeo would introduce a dependency to osgeo, which in general is not advisable. Fiona needs to be compiled against specific dll versions. If osgeo package is updated, Fiona would potentially break if this update changes the dlls and the fiona package is not recompiled against the new set of dlls.

[giohappy]

@rbuffat my code wasn't meant to be adopted as it is. Take the osgeo and the append to PATH lines away. The only relevant part is the cycling on the PATH dirs. If for any reason the GDAL DLLs folder are set inside the PATH they will be loaded, otherwise not.

Considering that __init__.py appends ./libs to the PATH, if these ./libs folder exists (from locale compiling) that will always have precedence over the ones set inside the PATH, which makes sense to me.

[sgillies]

@giohappy thank you for the report. I'm sorry, but to keep things simple we're not going to add DLL directories except the ones in the fiona wheels.