ImportError: libtorch_cpu.so: cannot enable executable stack as shared object requires: Invalid argument #350
Comments
|
I've just asked around, and the answer I got is that glibc 2.41 prohibits dlopening libraries with executable stacks. We'll have to figure out which library does this, though. |
|
Yep, I can reproduce. I wish I'd knew that before i upgraded glibc today :-). Please note that generally it isn't safe to downgrade glibc. |
|
Ok, so apparently it isn't coming from any of the shared dependencies but from |
|
Ok, so far confirmed that;
|
I guess that's the problem. |
|
This has the potential to turn into a bigger problem actually, because GCC's nested function extension uses an executable stack - see here for some background. We can only hope that this changed with more recent GCC versions, but I'm not sure if that's even possible due to ABI. Otherwise any project that uses said extension on linux might be affected. |
|
Thanks for the information! So currently there are two workarounds:
Related discussions:
|
GCC can use the heap for the trampoline for some ABIs now. |
|
Thanks for the information! Do you also happen to know since which GCC version(s) |
|
GCC 14 (r14-4821-g28d8c680aaea46 and a few commits after) implemented it with
The default is still on the stack - I don't know what the blocker is there for changing it to heap where appropriate (@iains?) I don't, at a glance, see these in the Changes pages for GCC 14, which I'm going to handle now (EDIT: patch). |
|
Thank you very much for all that. 🙏 If it becomes too painful, we might have to accelerate our switch to GCC 14 (normally we wait until the patch release after the next major version, so 14.3). And I guess ppc will just have to live with the workarounds until GCC ever learns |
|
For SysV targets, I would expect copying the approach used for X86_64/Aarch64 would work. For BSD-style targets maybe copying the approach used by Darwin. I guess it's too late for GCC-15 (PPC) since the patch would need to be approved by the platform maintainer (but of course you could make a patch and see if it is acceptable) |
@h-vetinari Out of curiosity, could you please give some examples on what libraries are using the GCC's nested function extension? |
|
I don't have a good answer for you on that, but it's a long-standing extension that very likely has a bunch of users. Probably less so on projects that focus on cross-platform support (because there the code then isn't usable on osx/win, at least not with the default compilers of that platform), but I'd imagine several linux-focused projects to have used this. It's a useful pattern, even though it's not standard C. For more details see the article I posted. |
|
You should also be aware that other core languages (Ada, Fortran) also use nested functions "under the hood" so it's not just explicit use in C that matters to the overall "downstream". |
Explicitly pass `-Wl,-z,noexecstack` to the linker, to ensure that `libpytorch_cpu.so` is compiled without an executable stack. This is necessary because the raw assembly in oneDNN triggers: ``` $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: ittptmark64.S.o: missing .note.GNU-stack section implies executable stack ``` ...and glibc 2.41 no longer permits loading libraries with executable stack. Fixes conda-forge#350
|
#355 fixes it for me. |
Explicitly pass `-Wl,-z,noexecstack` to the linker, to ensure that `libpytorch_cpu.so` is compiled without an executable stack. This is necessary because the raw assembly in oneDNN triggers: ``` $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: ittptmark64.S.o: missing .note.GNU-stack section implies executable stack ``` ...and glibc 2.41 no longer permits loading libraries with executable stack. Fixes conda-forge#350
Explicitly pass `-Wl,-z,noexecstack` to the linker, to ensure that `libpytorch_cpu.so` is compiled without an executable stack. This is necessary because the raw assembly in oneDNN triggers: ``` $BUILD_PREFIX/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: ittptmark64.S.o: missing .note.GNU-stack section implies executable stack ``` ...and glibc 2.41 no longer permits loading libraries with executable stack. Fixes #350
Solution to issue cannot be found in the documentation.
Issue
I create a minimal environment with PyTorch (with command
mamba create -n ttt python=3.11 pytorch), then I get the following error when imporing PyTorch (withpython -c 'import torch'):I am using Arch Linux and the kernel version is 6.13.2-arch1-1, this issue seems to appear after a recent system upgrade (I can try rolling it back, but since it would be a huge effort for me, I have to do it later).
I am not sure if here is the most appropriate place to ask. But since I can import torch successfully if installed with pip, I would appreciate any hint or suggestion.
Installed packages
Environment info
The text was updated successfully, but these errors were encountered: