Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Circular imports for Python 3.8 on Windows if DLLs are already loaded #944

Closed
rbuffat opened this issue Sep 4, 2020 · 46 comments
Closed

Comments

@rbuffat
Copy link
Contributor

rbuffat commented Sep 4, 2020

WIth fiona 1.8.15, the issue is fixed now in the conda-forge builds, only the Windows py3.8 build is still failing: conda-forge/fiona-feedstock#167

The failure is with the general error message of partially initialized module, so doesn't give much pointers (at least to me):

running run_test.py
Traceback (most recent call last):
  File "D:\bld\fiona_1599200296823\test_tmp\run_test.py", line 6, in <module>
    import fiona
  File "D:\bld\fiona_1599200296823\_test_env\lib\site-packages\fiona\__init__.py", line 84, in <module>
    import fiona._loading
  File "D:\bld\fiona_1599200296823\_test_env\lib\site-packages\fiona\_loading.py", line 44, in <module>
    import fiona.ogrext
  File "fiona/ogrext.pyx", line 29, in init fiona.ogrext
  File "D:\bld\fiona_1599200296823\_test_env\lib\site-packages\fiona\env.py", line 14, in <module>
    with fiona._loading.add_gdal_dll_directories():
AttributeError: partially initialized module 'fiona' has no attribute '_loading' (most likely due to a circular import)

Originally posted by @jorisvandenbossche in #941 (comment)

@rbuffat
Copy link
Contributor Author

rbuffat commented Sep 4, 2020

The issue is with

Fiona/fiona/_loading.py

Lines 43 to 46 in 52ad006

try:
import fiona.ogrext
except ImportError as e:
search_gdal_dll_directories()

Starting with Python 3.8 dll directories must be added with add_dll_directory() on Windows. (#851)

On Fionas appveyor, the import of import fiona.ogrext will fail as there are no DLL presents at the time of import, or fiona.ogrext is already loaded. But in the case of conda, the DLLs are already loaded. Thus the import of fiona.ogrext does not raise an ImportError and fiona.ogrext will import fiona.env, that again imports fiona._loading, which leads to a circular import.

A quick fix could be to change the test import to something that does not import anything else from Fiona. E.g.

 try: 
     import fiona._err 
 except ImportError as e: 
     search_gdal_dll_directories() 

But this will be quite dangerous because with adding additional imports the problem can arise again.

@rbuffat rbuffat changed the title Circular imports for Python 3.8 on Windows Circular imports for Python 3.8 on Windows if DLLs are already loaded Sep 4, 2020
@sgillies
Copy link
Member

sgillies commented Sep 4, 2020

@rbuffat as we solve this, let's also move this code

if platform.system() == 'Windows' and sys.version_info >= (3, 8):
    # if loading of extension modules fails, search for gdal dll directories
    try:
        import fiona.ogrext
    except ImportError as e:
        search_gdal_dll_directories()

into add_gdal_dll_directories(). Code that runs on import only is difficult to test.

@rbuffat
Copy link
Contributor Author

rbuffat commented Sep 4, 2020

I created a PR in #945. If you have already implemented something feel free to close it.

@rbuffat
Copy link
Contributor Author

rbuffat commented Oct 11, 2020

Fixed in Fiona 1.8.16

@rbuffat rbuffat closed this as completed Oct 11, 2020
@iboates
Copy link

iboates commented Dec 7, 2020

I'm leaving a comment here because it's the only Google result which matched my error exactly. Hopefully it can save someone else a similar headache.

I am using a conda environment when getting this error and it was solved by starting a new environment where I made sure that every package was being taken from the conda-forge channel (except for some libraries only available from the esri channel.

@Toblerity Toblerity locked and limited conversation to collaborators Dec 7, 2020
@Toblerity Toblerity unlocked this conversation Dec 7, 2020
@janhurst
Copy link

I'm leaving a comment here because it's the only Google result which matched my error exactly. Hopefully it can save someone else a similar headache.

I am using a conda environment when getting this error and it was solved by starting a new environment where I made sure that every package was being taken from the conda-forge channel (except for some libraries only available from the esri channel.

I am seeing the same behaviour with conda 4.9.2 on Windows 10 trying to get geopandas up and running. Creating an environment and forcing only conda-forge packages. As best as I can tell everything is "correctly" coming from conda-forge.

various versions are:
gdal 3.2.2
fiona 1.8.18
geopandas 0.9.0

@jorisvandenbossche
Copy link
Member

@janhurst can you post the output of conda list in the environment?

@jjohnarios
Copy link

@jorisvandenbossche I'm getting the same error with the same conda and packages versions as @janhurst . I have some non conda-forge packages which i believe have nothing to do with geopandas. I'm posting the exact error i'm getting and my conda list output.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-a62d01c1d62e> in <module>
----> 1 import geopandas as gpd

~\Anaconda3\envs\nilmtk-env\lib\site-packages\geopandas\__init__.py in <module>
      5 from geopandas.array import points_from_xy  # noqa
      6 
----> 7 from geopandas.io.file import _read_file as read_file  # noqa
      8 from geopandas.io.arrow import _read_parquet as read_parquet  # noqa
      9 from geopandas.io.arrow import _read_feather as read_feather  # noqa

~\Anaconda3\envs\nilmtk-env\lib\site-packages\geopandas\io\file.py in <module>
     18 
     19 try:
---> 20     from fiona import Env as fiona_env
     21 except ImportError:
     22     try:

~\Anaconda3\envs\nilmtk-env\lib\site-packages\fiona\__init__.py in <module>
     83 
     84 import fiona._loading
---> 85 with fiona._loading.add_gdal_dll_directories():
     86     from fiona.collection import BytesCollection, Collection
     87     from fiona.drvsupport import supported_drivers

AttributeError: module 'fiona' has no attribute '_loading'

conda_list.txt

@janhurst
Copy link

I noticed that the environment seems to work ok when i try to read a file in a notebook when using jupyter lab, yet the same notebook will throw the error I posted when running inside vscode.

@jjohnarios do you happen to be using vscode?

I dug a bit further. If I start vscode from a Powershell prompt (eg code . and note using the base conda environment), then open a notebook, and set the notebook kernel to my Python interpreter I am seeing the issue. I confirmed that sys.version and sys.executable seem to be pointing to the "geo" conda environment correctly, just the import geopandas call failing as indicated above.

However, If I first activate the "geo" environment (eg conda activate geo) in Powershell, and then do the same process of starting vscode and set the notebook kernel etc the error does not occur!

I'm pretty confused now :) ... I don't know if this is a vscode issue, or perhaps something screwy in conda and its Powershell integration etc.

@jorisvandenbossche for reference my conda list output is attached, as well as the environment yaml.

geo.yml.txt
conda_list.txt

@jjohnarios
Copy link

@janhurst I've tried vscode and import works fine. I was using jupyter notebook and spyder and the error kept occurring there. I made sure i was using the same kernel and environment. Consequently, i realised that : import geopandas works for me in python Anaconda prompt, in VScode, in Sublime Text and NOT IN jupyter notebook and spyder.

@andrewnell
Copy link

I have the exact same issue, where import geopandas works for me in the anaconda prompt and sublime, however, I get the same error as above in Jupyter Notebook, Jupyter Lab or Spyder.

It should also be noted that this is only for an environment I have created using Python 3.6, however, if I create an environment with Python 3.8 I do not have this issue. @jorisvandenbossche any ideas what may be causing this issue?

@rbuffat
Copy link
Contributor Author

rbuffat commented Mar 24, 2021

@jorisvandenbossche do you have an idea how to debug this? I don't have access to a Windows machine at the moment, so the only possibility would be to create a CI workflow that would reproduce this. But for that, we would require the exact steps to reproduce this issue.

Maybe placing print(dir(fiona)) in front of from fiona import Env as fiona_env in https://github.com/geopandas/geopandas/blob/master/geopandas/io/file.py#L20
might reveal some more useful information.

@jorisvandenbossche
Copy link
Member

I also don't have a Windows machine readily available.
Since the issue seem to be triggered by using an environment that is not directly activated, but used as a kernel from some UI (notebook, spyder), one possible issue could be that fiona is not finding/looking for the correct GDAL location (although I still find it a strange error message for that ..).

@janhurst @jjohnarios @andrewnell can you try if setting the following environment variable before importing geopandas/fiona inside the notebook / spyder helps? It should be the equivalent of:

@set "GDAL_DATA=%CONDA_PREFIX%\Library\share\gdal"

so in terms of python code you can run in the notebook before importing geopandas/fiona, I think it would be something like

import os
os.environ["GDAL_DATA"] = os.environ["CONDA_PREFIX"] + "\Library\share\gdal"

@jjohnarios
Copy link

I also don't have a Windows machine readily available.
Since the issue seem to be triggered by using an environment that is not directly activated, but used as a kernel from some UI (notebook, spyder), one possible issue could be that fiona is not finding/looking for the correct GDAL location (although I still find it a strange error message for that ..).

@janhurst @jjohnarios @andrewnell can you try if setting the following environment variable before importing geopandas/fiona inside the notebook / spyder helps? It should be the equivalent of:

@set "GDAL_DATA=%CONDA_PREFIX%\Library\share\gdal"

so in terms of python code you can run in the notebook before importing geopandas/fiona, I think it would be something like

import os
os.environ["GDAL_DATA"] = os.environ["CONDA_PREFIX"] + "\Library\share\gdal"

I've tried that but doesn't seem to work. Same error message.

@jorisvandenbossche
Copy link
Member

@jjohnarios OK, thanks for trying!

Aonther question: you posted above your conda list output, and indeed I don't directly see anything suspicious (the packages no from conda-forge but from pypi shouldn't have an impact on fiona). But could you try if you can reproduce it with a minimal, newly created environment as well?
Eg like the sequence of commands at https://geopandas.readthedocs.io/en/latest/getting_started/install.html#creating-a-new-environment

@janhurst
Copy link

Also didn't see any difference setting GDAL_DATA.

@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Mar 24, 2021

Another question (sorry for the many questions, but that's at the moment the only way for me to help debug it): do you only see it with the latest fiona 1.8.18? If you explicitly install fiona 1.8.17 in the environment, do you still see the error?

Or if you explicitly ask for an older GDAL? (you both seem to have 3.2.2 installed, which is relatively new (which might explain that it's a new issue)) Do you also have the error when creating the environment with fiona=1.8.18 gdal=3.1?

@janhurst
Copy link

But could you try if you can reproduce it with a minimal, newly created environment as well?
Eg like the sequence of commands at https://geopandas.readthedocs.io/en/latest/getting_started/install.html#creating-a-new-environment

I tried to follow the getting started linked as exactly as possible. The only addition is that the ipykernel package is needed to actually switch to the kernel in vscode's notebook environment.

Unfortunately I am still seeing the same behaviour - starting vscode from the base environment and trying to select the geo_env kernel results in the module 'fiona' has no attribute '_loading' message in an AttributeError exception.

For reference the conda list output for the environment as described in the geopandas getting started plus conda install ipykernel is attached.
geo_env_conda_list.txt

@janhurst
Copy link

Do you also have the error when creating the environment with fiona=1.8.18 gdal=3.1?

Missed this, its late here but I'll give this a shot in the next day or two.

@RicardoSQueen
Copy link

RicardoSQueen commented Mar 24, 2021

Hey, there,

Having the same issue as @janhurst.

Another question (sorry for the many questions, but that's at the moment the only way for me to help debug it): do you only see it with the latest fiona 1.8.18? If you explicitly install fiona 1.8.17 in the environment, do you still see the error?

Tried with fiona 1.8.17 and 1.8.16 and still no success (same error)

Or if you explicitly ask for an older GDAL? (you both seem to have 3.2.2 installed, which is relatively new (which might explain that it's a new issue)) Do you also have the error when creating the environment with fiona=1.8.18 gdal=3.1?

Tried that too and still no success. Here is my current conda list output:
conda_list.txt

Just reforcing: on anaconda prompt the import occurs successfully, on vscode it does not work.

@fargiolas
Copy link

Probably a different bug, but I'm seeing a similar circular import error when I import geopandas (fiona 1.18.18, geopandas 0.9.0, gdal 3.2.2, installed with pip and brew on macos). The error goes away if I import fiona before geopandas.

@jorisvandenbossche
Copy link
Member

OK, still a mystery .. (and thanks all for the testing!)
Some other things to verify / try:

1. Does this only happen with geopandas 0.9, or also with older versions (0.8)?

2. Something that changed in the latest geopandas version, is that we delayed the fiona import (geopandas/geopandas#1775), but which might have as concrete effect that the order of imports of the different geo-reletad packages changed.
So maybe something to try, does one of the following import sequences also trigger the error? (in the notebook/spyder/vscode environment)

import pyproj
import fiona
import shapely
import fiona

3. Another possible "pattern" in the posted conda list outputs, is that you all have pyproj 3.0.1. Could someone try creating an environment but with an older version of pyproj? (3.0.0 or 2.6.1)?
(not directly an idea how that would influence it, but you never know ... and to eliminate possibilities)

@ogelin
Copy link

ogelin commented Mar 25, 2021

Hi, on MacOS using pip installs I also had this error when importing geopandas. Python 3.9.2.

I changed pyproj from 3.0.1 to 3.0.0 and now the error goes away as long as I import fiona before geopandas.

If I only import geopandas without fiona first, I still get the "Circular import" error message that others reported.

@LSgeo
Copy link

LSgeo commented Mar 26, 2021

I noticed that the environment seems to work ok when I try to read a file in a notebook when using jupyter lab, yet the same notebook will throw the error I posted when running inside vscode.

@jjohnarios do you happen to be using vscode?

I dug a bit further. If I start vscode from a Powershell prompt (eg code . and note using the base conda environment), then open a notebook, and set the notebook kernel to my Python interpreter I am seeing the issue. I confirmed that sys.version and sys.executable seem to be pointing to the "geo" conda environment correctly, just the import geopandas call failing as indicated above.

However, If I first activate the "geo" environment (eg conda activate geo) in Powershell, and then do the same process of starting vscode and set the notebook kernel etc the error does not occur!

I'm pretty confused now :) ... I don't know if this is a vscode issue, or perhaps something screwy in conda and its Powershell integration etc.

@jorisvandenbossche for reference my conda list output is attached, as well as the environment yaml.

geo.yml.txt
conda_list.txt

Hi, I've stumbled across here too, and wanted to add my two cents. I'm running a script in VScode on windows 10, and only encounter the issue when executing via a Jupyter Interactive notebook session.

I've encountered a lot of other dll issues with other libraries in the Jupyter Interactive window, that don't occur when running the script normally via the command line. There are a lot (of vaguely similar problems) spread across various github issues, but e.g. , vscode-jupyter and numpy

As you mention in the above comment, activating an environment and calling code to start vscode works around the issue for me. I'd love to learn why this is, and if there is a better solution.

@jorisvandenbossche
Copy link
Member

Some general background context: conda has the concept of "environment activation" (which happens if you manually activate an environment in the console with conda activate env_name). This step sets up the environment to work properly, and for example sets the appropriate environment variables (eg it adds the env directory and the env's /bin directory to the PATH).

Specifically for fiona, we rely on GDAL, and the libgdal package from conda-forge has a specific activation script (see https://github.com/conda-forge/gdal-feedstock/tree/master/recipe/scripts), which sets the GDAL_DATA environment variable. In addition, I think fiona relies on being able to find the GDAL binary based on PATH.

If those environment variables are not properly set, packages will have problems finding the correct binaries / libraries.

And I suppose this is the problem that causes the issues you all encounter when using jupyter notebook or vscode.

For Jupyter Notebook with using kernels based on conda environments, this is a known problem that this usage pattern doesn't automatically activate the environment when starting the kernel (jupyter/notebook#3588). One workaround is to install notebook in the environment you want to use, first activate the environment, and then start the notebook from that environment (this defeats the purpose of having multiple kernels a bit, though). Another solution is to use a tool like https://github.com/Anaconda-Platform/nb_conda_kernels that ensures kernels are activated properly. Finally, you could also prepend the appropriate ..\\envs\\env_name\\Library\\bin to your PATH manually in the notebook before importing fiona/geopandas.

For VSCode, I have less experience with this, but according to https://code.visualstudio.com/docs/python/environments#_select-and-activate-an-environment it seems that it should be possible to adjust your settings to ensure the environment is activated.


Now, what I am less sure about is why it seems that there now new issues with this (given that multiple people posted here in the last weeks), and if something changed in fiona / gdal / the conda packages to make this problem worse / easier to run into.
(unless this is just coincidental)

@fargiolas
Copy link

fargiolas commented Mar 28, 2021

Now, what I am less sure about is why it seems that there now new issues with this (given that multiple people posted here in the last weeks), and if something changed in fiona / gdal / the conda packages to make this problem worse / easier to run into.
(unless this is just coincidental)

I've been trying to track this on my setup (macos, arm64, packages installed with brew and pip3, no conda) with little success. The first fiona import geopandas/io/file.py fails with:

>>> import geopandas Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/opt/homebrew/lib/python3.9/site-packages/geopandas/__init__.py", line 7, in <module> from geopandas.io.file import _read_file as read_file # noqa File "/opt/homebrew/lib/python3.9/site-packages/geopandas/io/file.py", line 20, in <module> from fiona import Env as fiona_env File "/opt/homebrew/lib/python3.9/site-packages/fiona/__init__.py", line 85, in <module> with fiona._loading.add_gdal_dll_directories(): AttributeError: partially initialized module 'fiona' has no attribute '_loading' (most likely due to a circular import)

But if I wrap a try/except around all the fiona imports in fiona/__init__.py after with fiona._loading.add_gdal_dll_directories(): the exception I get is the following:

>>> import geopandas exception: dlopen(/opt/homebrew/lib/python3.9/site-packages/fiona/ogrext.cpython-39-darwin.so, 2): Symbol not found: _GEOSArea Referenced from: /opt/homebrew/opt/libspatialite/lib/libspatialite.7.dylib Expected in: flat namespace in /opt/homebrew/opt/libspatialite/lib/libspatialite.7.dylib Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/opt/homebrew/lib/python3.9/site-packages/geopandas/__init__.py", line 7, in <module> from geopandas.io.file import _read_file as read_file # noqa File "/opt/homebrew/lib/python3.9/site-packages/geopandas/io/file.py", line 12, in <module> import fiona File "/opt/homebrew/lib/python3.9/site-packages/fiona/__init__.py", line 112, in <module> __gdal_version__ = get_gdal_release_name() NameError: name 'get_gdal_release_name' is not defined

The same doesn't happen if I import fiona directly, only from geopandas. I tried setting gdal datadir with export GDAL_DATA=$(gdal-config --datadir) but it does not help.

There are some old mac specific bugs about the missing symbol in libspatialite, see e.g. Homebrew/homebrew-core#5161 maybe some recent change triggered them again.

Still digging, I'll comment if I find anything new, if you have some test you'd like me to run let me know.

@jorisvandenbossche
Copy link
Member

I have a question for those of you who commented here and where "import fiona before geopandas" fixes the issue.

To be able to investigate this issue, I would like to reproduce it (I have access to a Linux and Windows machine), but I currently don't know how.

If some people that have this specific issue could describe their environment / how they installed geopandas and fiona / etc, that would be very helpful!

@fargiolas
Copy link

@jorisvandenbossche I'm one of those but I only reproduced it on a macbook m1 (arm64) with packages installed from brew and pip, not sure if I can help. It's been a while since I looked into this, I should be able to run a quick test this evening to see if the issue is still there.

@jorisvandenbossche
Copy link
Member

@fargiolas if you would be able to check if you can still reproduce the issue, that would certainly be welcome! (let me know if you succeed, at which point I have a bunch of questions / things to test :) Are you using gitter? that might be a bit easier to chat about it)

@nmarchio
Copy link

Not sure if this is useful but I encountered this exact circular import error today and was able to resolve it by pinning my pygeos version to 0.10.2 instead of the most recent 0.12.0. I'm running it on a Macbook M1 (arm64) architecture. Note, this conda environment installs Fiona version 1.8.20.

conda create --name pygeospatial python=3.9.7 --yes
source activate pygeospatial 
conda install -c conda-forge pygeos=0.10.2 --yes
conda install -c conda-forge geopandas=0.10.2 --yes 

miccferr added a commit to miccferr/scikit-mobility-feedstock that referenced this issue Feb 27, 2022
Prioritize `fiona` import to avoid circular dependencies (see [this coment](Toblerity/Fiona#944 (comment)) )
@KevinChen9412
Copy link

One workaround is to install notebook in the environment you want to use, first activate the environment, and then start the notebook from that environment

I am having this Circular imports issue only when i try to import fiona in jupyter notebook (everything works fine in my Pycharm). This workaround fixed my issue. @jorisvandenbossche

@adamjstewart
Copy link
Contributor

We just hit the same circular import issue in our CI today, should this issue be reopened?

@theroggy
Copy link

A few months ago I had the issue on my CI tests on MacOS, but it disappeared again a few weeks later, and since yesterday the problem popped up on the windows tests...

@adamjstewart
Copy link
Contributor

I think it's related to this new version of the conda feedstock released yesterday: conda-forge/fiona-feedstock#191

@adamjstewart
Copy link
Contributor

If only we had Windows wheels and didn't need to rely on conda...

@MattBrst
Copy link

MattBrst commented Aug 23, 2022

I had the same error messages as mentioned above on a fresh Python 3.10 install from Anaconda on Windows 10:

import fiona

resulting in the following error message: ImportError: DLL load failed while importing ogrext: Kan opgegeven procedure niet vinden.

import geopandas
import fiona

resulting in another error message: AttributeError: partially initialized module 'fiona' has no attribute 'loading' (most likely due to a circular import).

The issue was solved by the following order of importing packages:

from osgeo import gdal
import fiona
import geopandas

@mlnrt
Copy link

mlnrt commented Sep 19, 2022

I also ran into that same issue and it drove me crazy for days.
In my code, in one of my own library I have the following imports:

import pandas as pd
import pygeos
import geopandas as gpd
from fiona.errors import DriverError

This resulted in the below behaviors:

  • In the conda Terminal, once activating the environment, I could import the library without any issue as above
  • In PyCharm/DataSpell/VSCode/Jupyter I gut the following error "AttributeError: partially initialized module 'fiona' has no attribute '_loading' (most likely due to a circular import)"
  • If I inverse the order of the fiona and geopandas error, I get a different DLL error "ImportError: DLL load failed for fion.ogrext import Iterator, ItemsIterator, KeysIterator"
  • Starting Jupyter from the activated conda environment terminal, resulted in the same issue as with PyCharm/DataSpell/VSCode

The packages were installed as follows using conda:

conda install -c conda-forge -y fiona==1.8.21 geopandas==0.10.2 pygeos==0.12.0

I tried installing GDAL from the GISInternals website as recommended on the Fiona PiPy page. But it did not solve the issue.
The Fiona installation instruction also mention "You will also need to download the compiled libraries and headers (include files)." But there is no instructions as to where to put those downloaded files.
I tried to install Fiona from wheel file which can be obtained from Christoph Gohlke’s website. But that did not make a difference compared to the conda-forge installation.

Also in PyCharm/DataSpell/VSCode/Jupyter, the conda environment seems to be activated correctly. The os.environ["PATH"] and os.environ["GDAL_DATA"] are there and correct.

None of the mentioned solutions mentioned in the thread above (or other forums) worked:

  • importing fiona before geopandas. Again this resulted in a different DLL error
  • starting Jupyter Notebook from the activated conda environment terminal
  • pinning down the pygeos version to 0.10.2
  • installing GDAL from the GISInternals website as recommended on the Fiona PiPy page
  • installing GDAL from the wheel file from Christoph Gohlke’s website
  • erase the kernel specs in my Windows folder
  • install nb_conda_kernels
  • adding a from osgeo import gdal before importing fiona.errors

In the end what worked for me was the following:

  1. Uninstall the direct Python 3.9 and 3.10 installations I had on top of Anaconda 3
  2. Uninstall Anaconda 3
  3. Reinstall Anaconda 3 with the options "Add Anaconda to my PATH environment variable" and "Register Anaconda as my default Python 3.9"
  4. Recreate my conda environment

I also noticed the following issue: If I import tensorflow before fiona.errors and geopandas then I get into the same error again "AttributeError: partially initialized module 'fiona' has no attribute '_loading' (most likely due to a circular import)"

@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Sep 22, 2022

@mlnrt I am sorry to hear the troubles it caused, but thanks for the detailed write-up (it is a really annoying issue ..).
In the unlikely case you would still be up for some more testing (now that you finally have it working ..), I would be interested to know if you run into the same issue if installing pyogrio instead of fiona (if it works there, that might give us some pointers if we can change something about fiona)

@mlnrt
Copy link

mlnrt commented Sep 24, 2022

@jorisvandenbossche Thank you for reach out.
I tried to create a new conda environment with pyogrio but it only installs pygeos. geopandas is a requirement but it doesn't installs it by default so I had to add it to the list of package and geopandas will install fiona. So by installing pyogrio and geopandas , compare to before I just end up with pyogrio on top of fiona, geopandas and pygeos. So I am not sure why it would make a difference. I mean it worked, but since I completely rebuilt my Windows Anaconda and Python environment it also works without pyogrio.
The other problem is that to really test, I would need to go back to my previous Anaconda + Python 3.9 + Python 3.10 environment which I don't have anymore. In my current windows environment Fiona, Geopandas and PyGeos do work correctly.

@jorisvandenbossche
Copy link
Member

compare to before I just end up with pyogrio on top of fiona, geopandas and pygeos. So I am not sure why it would make a difference.

It could still be the case that importing fiona fails, and importing pyogrio works. But of course that's only useful to test in the env where fiona was actually failing to be imported. And if you don't have that installation at hand anymore, don't worry about it (not need to go back to your previous not-working setup!)

@gmerritt123
Copy link

I had the same error messages as mentioned above on a fresh Python 3.10 install from Anaconda on Windows 10:

import fiona

resulting in the following error message: ImportError: DLL load failed while importing ogrext: Kan opgegeven procedure niet vinden.

import geopandas
import fiona

resulting in another error message: AttributeError: partially initialized module 'fiona' has no attribute 'loading' (most likely due to a circular import).

The issue was solved by the following order of importing packages:

from osgeo import gdal
import fiona
import geopandas

+1 this worked for me. Thanks so much. On a python 3.8 conda forge environment built up a few weeks ago.

If this resolves the issue, the issue has to be that the gdal path is messed up eh? Hard importing gdal in gives fiona/geopd the right path to point to but without it it fails?

This is the second conda environment on this machine that needs gdal/fiona/geopandas (porting stuff from 3.7 to 3.8)... but there's only one path/one specific path each fiona/geopd can point to for all of them somehow? Any advice on updating that path (where and to what) would likely perma-solve the issue....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests