install doesn't detect conflicts with already installed packages

[rbtcollins]

NB: I haven't created a reproducer for this, but I'm fairly sure its an issue :).

Given A with two versions 1.0 and 2.0 on PyPI
B install_requires A~=1.0
C install_requires A~=2.0

pip install B
pip install C

will complete without error, leaving B broken.

[qwcode]

yea, B will be broke.

[xavfernandez]

Related to #2492

[rbtcollins]

So decisions:
1)
pip install X which depends on Y==1.0
pip install Z which requires Y==$otherversion
Should that a) error or b) change X to a version that supports otherversion
I think b) because pip install X again would already automatically change Y, without needing -U.

2. should there be an escape hatch to say 'don't consider already installed things', and if so what spelling to give it?

[qwcode]

I would say a) error. changing the version of X seems radical to me.

as for "don't consider already installed things", we have --ignore-installed... could it live under that?

[piotr-dobrogost]

b) change X to a version that supports otherversion

What about packages that depend on X? I mean doesn't trying to fix version of Y forces us to make full dependency resolution? Where do we draw a line? I have a feeling that there are only two options; either there's very simple (dump?) dependency resolution like currently – take the first match or we are forced to make full dependency resolution. I can't image how some middle ground could look like.

changing the version of X seems radical to me.

It sure feels like this taking into account there's almost no dependency resolution in pip now.

[rbtcollins]

@qwcode we already change versions of things the user didn't specify. I"m confused why you'd find doing that for something that conflicts radical.

--ignore-installed - yes,that seems reasonable.

@piotr-dobrogost yes, this would work transitively. I'm working on this in the context of having full dependency resolution...

[dstufft]

I think it makes sense to change the versions of things even if they are already installed. It's a deviation of how pip has worked in the past -- generally we treated things as one shot installs while ignoring what was already installed (to some degree, besides conflict detection).

I think this is trickier then it appears at first, in a way this is instead changing it so that pip install, instead of just doing a one shot install, instead it adds a new desired item to the environment and then resolves it. I think this means that we'll need to track why items were installed and treat it in the dependency resolution as if all of the pip install commands were really all run together, so that the behavior of pip install foo<3 bar has the same result as pip install foo<3 && pip install bar (assuming no changes in the index between the two commands).

[rbtcollins]

So, we'll need a persistent store of the user supplied constraints. I'm happy to implement $whatever, but do you pip maintainers have any specific constraints on/around this? E.g. concurrency safe, place to store it, format to use?

Off hand I'm thinking: a directory in site-packages with a name that isn't a legitimate package containing one file per user-supplied constraint, with the content of the file being in requirements.txt format, updates via temp+rename on linux, with temp+delete+rename on windows.

[xavfernandez]

Maybe we could use https://www.python.org/dev/peps/pep-0376/#requested file to store the user constraints ?

[qwcode]

we already change versions of things the user didn't specify.
I"m confused why you'd find doing that for something that conflicts radical.

I'm ignoring what we currently do, since it's not we want.

my concern is changing a top-level requirement due to a sub-dependency in a different execution.

let's say I explicitly install "X==1", then later install "Y==1", which depends on "X==2".

I think that should fail, vs it automatically assuming, it should change my version of X.

@dstufft 's point about tracking the history of executions as requirements in the environment makes sense, and would alleviate my concern I think.

[ionelmc]

Does it really make sense to store "user supplied constraints", as opposed to "package provided constraints" (from install_requires or whatever)? What do you do when the user changes their mind about whatever constraints he had in the past? Looks like a user experience quandary to me.

IMO, pip should only check against constraints of already installed packages when upgrading packages, not some past constraints provided at cmdline 1 year ago (that the user surely forgot, and would make for a really bad surprise factor).

[dstufft]

Closing this as a duplicate of #988.