Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow skipping load_site_packages #3693

Closed

Conversation

maresb
Copy link
Contributor

@maresb maresb commented Dec 16, 2024

As part of the hacks used in conda-lock, we create a fake Conda environment containing just the package metadata in $PREFIX/conda-meta/, and we query this by running micromamba list -p fake-conda-env --json in a subprocess.

The v2 changes have broken this process because micromamba now checks not just for conda packages but newly for pip packages in the PrefixData::load_site_packages function by running python -q -m pip inspect --local.

The problem I'm now having is that load_site_packages expects a python executable to be present in $PREFIX/bin. Moreover, it's this python executable is then used in the python -q -m pip inspect --local command. But in conda-lock's fake environment, we don't have a $PREFIX/bin, so when load_site_packages calls get_python_path the result is an empty string, and I get the following error message:

critical libmamba could not load prefix data: failed to run python command :
      error: The parameter is incorrect.
      command ran:  -q -m pip inspect --local
      env options:("PYTHONIOENCODING", "utf-8") ("NO_COLOR", "1") ("PIP_NO_COLOR", "1") ("PIP_NO_PYTHON_VERSION_WARNING", "1")

Note that /path/to/some/python is missing before -q -m ....

What conda-lock is doing with the fake environment is hacky, and is not the responsibility of libmamba. But I just want to get it running again with v2, so I hope that you consider accepting this patch to allow me to disable the functionality from conda-lock by setting an environment variable. I expect only conda-lock to use this, so I don't think we need to document it.

(Note that I have tried other workarounds such as adding a fake python script to my fake conda environment, but on Windows the which_in helper only seems to detect .exe files and not .bat scripts, so that would require me to compile binaries which is not so viable for a pure Python project like conda-lock. I am open to other suggestions.)

Thanks for your consideration!

@maresb
Copy link
Contributor Author

maresb commented Dec 16, 2024

CC @Hind-M; thank you so much for all your help with fixing all of these conda-lock related issues.

@Hind-M
Copy link
Member

Hind-M commented Dec 17, 2024

Hi @maresb and thank you for reporting this!
It seems that conda provides a --no-pip flag to the list command to ignore pip installed packages. I'm not sure whether this is working properly there because of this.
I also tried locally but the flag seems to be buggy in conda.
I guess beside your use case there could be others, and since we were planning to support more missing flags and options in mamba to be more compatible with conda (cf. this issue to track them), this PR is superseding yours, so I'm closing this one :)

@Hind-M Hind-M closed this Dec 17, 2024
@maresb
Copy link
Contributor Author

maresb commented Dec 17, 2024

Amazing, thanks so much @Hind-M for prioritizing this for me, your solution looks better!

I agree with closing this PR, and I will track yours instead.

@maresb maresb deleted the allow-skipping-load-site-packages branch December 17, 2024 14:52
maresb added a commit to maresb/conda-lock that referenced this pull request Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants