Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support discovery of multiple python versions that are not on path #11308

Open
mneumei opened this issue Feb 7, 2025 · 8 comments
Open

Support discovery of multiple python versions that are not on path #11308

mneumei opened this issue Feb 7, 2025 · 8 comments
Labels
enhancement New feature or improvement to existing functionality needs-decision Undecided if this should be done

Comments

@mneumei
Copy link

mneumei commented Feb 7, 2025

Summary

We have multiple python versions that we do not want to put on path. For a single venv, we run uv with --python and there is no issue, but when we want to run tox, we can not tell uv where to find those different python versions anymore (original flag: discover, with as many python versions as you want).

Ideally, you could e.g. give multiple --python options, and uv will select the most suitable (matching the python requirements, newest, first)

Linked Issue:
tox-dev/tox-uv#170 (comment)

Example

No response

@mneumei mneumei added the enhancement New feature or improvement to existing functionality label Feb 7, 2025
@zanieb
Copy link
Member

zanieb commented Feb 7, 2025

I don't think we'll allow --python to be repeated like this, it sounds too complicated. Can't tox invoke uv with the proper --python invocation for each requested version? Or why can't the PATH be changed just for the uv invocation?

@gaborbernat
Copy link
Contributor

While tox could do this, I don't think is unreasonable for uv to handle it. uv already does some kind of Python interpreter discovery, it's not unreasonable to expect to be able to extend the list of Pythons to consider with a list of static paths 🤔 that uv can then also inspect part of the discovery.

@zanieb
Copy link
Member

zanieb commented Feb 7, 2025

I don't quite understand the use-case though. Like, why support things in addition to PATH? Interpreter discovery is already very complicated.

@gaborbernat
Copy link
Contributor

gaborbernat commented Feb 7, 2025

Is an escape hatch to force a given Python interpreter to be used, basically pushes that interpreter as the highest priority onto the discovery chain. Has a few advantages:

  • does not require changing an environment variable and can be explicit use this first without needing to change scripts.
  • if that PATH contains multiple Python scripts you don't' have to add the ones you don't want to use.

It follows the idea of explicit is better than implicit. It's primary use case is that imagine you have a script that requires Python 3.13. You have two builds of that Python on your system, one with debug symbols and one without. In general you want to use the non-debug symbol one as is faster. But you ran into a bug and you need more information so you want to temporarily switch to the debug variant.

In your script you specify uv venv -p 3.13. Now how can you temporarily switch between the two variants? You could add to the beggining of the PATH the one you want to use (but perhaps there are other Python versions too, and you're now sure which will be picked), however now you're changing an environment variable that could have other side effects to. Or you can change your script, but this now means you modified a file that you don't want to commit.

A not complicated workaround for these use cases that virtualenv has is the https://virtualenv.pypa.io/en/latest/cli_interface.html#try-first-with flag. This allows the user to short circuit the discovery, and from virtualenv POV is just first trying those passed in if compatible and use that, if they are not can fallback to the more complicated discovery mechanism. This issue is asking for something similar in uv; via a flag and/or environment variable.

@zanieb
Copy link
Member

zanieb commented Feb 7, 2025

Is an escape hatch to force a given Python interpreter to be used, basically pushes that interpreter as the highest priority onto the discovery chain.

We do support this already, via -p <path>.

I'm pushing against accepting multiple Python paths and choosing between them.

--try-with-first is... interesting. I've never heard of that. It seems peculiar.

Anyway, happy to consider — but I probably won't prioritize adding this without more feedback about why it's needed / use-cases / user requests.

@gaborbernat
Copy link
Contributor

gaborbernat commented Feb 7, 2025

We do support this already, via -p <path>.

This doesn't work if you're script specifies the generic -p 3.13 and you don't want to change your script to hardcode a path.

Another use case was of forcing thread free Python versions before we explicitely added support for them. Because before adding support the virtualenv discovery didn't know the difference between a threaded and non thread versions, so by settting that environment variable the user could force start/use this interpreter even if they are other compatible versions on the machine.

@mneumei can add their own use case, but I present one above 👍

@zanieb zanieb added the needs-decision Undecided if this should be done label Feb 7, 2025
@AdrianCert
Copy link

Our use case is that we don’t have an internet connection during our CI process, so we cannot rely on Python being managed by UV. Instead, we have a predefined set of Python versions available that we can use.

@gaborbernat
Copy link
Contributor

Our use case is that we don’t have an internet connection during our CI process, so we cannot rely on Python being managed by UV. Instead, we have a predefined set of Python versions available that we can use.

Can you explain why you can't just set the PATH environment variable for your use case? Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement to existing functionality needs-decision Undecided if this should be done
Projects
None yet
Development

No branches or pull requests

4 participants