[2020-resolver] pip seems to download older package versions

[atodorov]

Description

A pip install where many dependencies use exact versions leads to quite unexpected versions which later leads to a pylint warning about method signature mismatch and will lead to more problems because our software uses libraries which had removed methods or changed behavior in certain versions.

Maybe related to #9922

Expected behavior

Latest versions are installed.

pip version

pip 21.1

Python version

3.6.13

OS

Ubuntu (GitHub Actions)

How to Reproduce

git clone https://github.com/kiwitcms/github-app && cd github-app/
sudo apt-get install libkrb5-dev  # needed for some dependencies
pip install -U pip
pip install -r devel.txt

Output

https://github.com/kiwitcms/github-app/pull/50/checks?check_run_id=2475410935

Successfully installed Django-3.1.4 Faker-8.1.2 Markdown-3.3.3 PyGithub-1.53 PyJWT-2.1.0 Pygments-2.7.3 asgiref-3.3.4 astroid-2.5.6 beautifulsoup4-4.9.3 bleach-3.2.1 bleach-allowlist-1.0.3 certifi-2020.12.5 cffi-1.14.5 chardet-4.0.0 coverage-5.5 coveralls-3.0.1 cryptography-3.4.7 defusedxml-0.7.1 deprecated-1.2.12 django-attachments-1.8 django-colorfield-0.3.2 django-contrib-comments-1.9.2 django-extensions-3.1.0 django-grappelli-2.14.3 django-guardian-2.3.0 django-modern-rpc-0.12.1 django-recaptcha-2.0.6 django-simple-history-2.12.0 django-tenants-3.2.1 django-tree-queries-0.4.3 django-uuslug-1.2.0 django-vinaigrette-2.0.1 docopt-0.6.2 factory-boy-3.2.0 flake8-3.9.1 idna-2.10 importlib-metadata-4.0.1 isort-5.8.0 jira-2.0.0 kiwitcms-8.9 kiwitcms-tenants-1.4.3 lazy-object-proxy-1.6.0 mccabe-0.6.1 mock-4.0.3 oauthlib-3.1.0 packaging-20.9 pbr-5.6.0 psycopg2-2.8.6 pycodestyle-2.7.0 pycparser-2.20 pyflakes-2.3.1 pylint-2.8.2 pylint-django-2.4.4 pylint-plugin-utils-0.6 pyparsing-2.4.7 python-bugzilla-3.0.2 python-dateutil-2.8.1 python-gitlab-2.5.0 python-redmine-2.3.0 python-slugify-4.0.1 python3-openid-3.2.0 pytz-2021.1 requests-2.25.1 requests-oauthlib-1.3.0 requests-toolbelt-0.9.1 six-1.15.0 social-auth-app-django-4.0.0 social-auth-core-4.1.0 soupsieve-2.2.1 sqlparse-0.4.1 text-unidecode-1.3 toml-0.10.2 topicaxis-opengraph-0.5 typed-ast-1.4.3 typing-extensions-3.7.4.3 urllib3-1.26.4 webencodings-0.5.1 wrapt-1.12.1 zipp-3.4.1

From the "github-app" application:

$ cat devel.txt
-r requirements.txt

factory_boy
flake8
coveralls
PyGithub
pylint
pylint-django
psycopg2
kiwitcms>8.6

$ cat requirements.txt 
social-auth-app-django
social-auth-core>=3.3.0
kiwitcms-tenants

From the kiwitcms package:

• latest version at time of writing is 10.0
• it has Django==3.1.7 and PyGithub==1.54.1 as pinned requirements.

What I can spot from the output above is:

• Installed Django version is 3.1.4, not 3.1.7 like I expect
• Installed PyGithub version is 1.53, not 1.54.1 like I expect
• Installed kiwitcms version is 8.9, not 10.0 like I expect.

Additional information:
The problem & the full logs can be seen at kiwitcms/github-app#50

The previous commit which executed the same test jobs and reported PASS was kiwitcms/github-app@c3a47e1 2 months ago on Feb 14th 2021 (minus the pip install -U pip part). This is the test job for that commit: https://github.com/kiwitcms/github-app/runs/1899170145.

From there I see: Successfully installed Django-3.1.5 ... PyGithub-1.54.1 ... kiwitcms-9.0 which is correct at the time! I don't see a reference for the pip version but it isn't telling me that a new version exists so probably latest-ish at the time.

### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).

[uranusjr]

From your logs

INFO: pip is looking at multiple versions of kiwitcms to determine which version
is compatible with other requirements. This could take a while.

This means that pip tried multiple versions of kiwitcms (likely including the one you wanted), but settled on this exact version since all later versions would cause conflicts. This is the intended behaviour. If you want exactly kiwitcms==10.0, you should say that in your requiremements file; if you don’t say that, pip is free to download another version as it sees fit.

As for why pip thinks kiwitcms==10.0 is unsatisfiable, I would suggest looking into your dependency set and identify the conflicts. The environment variable PIP_RESOLVER_DEBUG may also help you look into what pip was doing during resolution.

I am going to close this as invalid since everyhing is working as intended according to the information you made available. Feel free to reopen with more detailed description if your investigation reveals pip is doing something unreasonable.

[atodorov]

From your logs

INFO: pip is looking at multiple versions of kiwitcms to determine which version
is compatible with other requirements. This could take a while.

This means that pip tried multiple versions of kiwitcms (likely including the one you wanted), but settled on this exact version since all later versions would cause conflicts. This is the intended behaviour. If you want exactly kiwitcms==10.0, you should say that in your requiremements file; if you don’t say that, pip is free to download another version as it sees fit.

That is a very big change in behavior and definitely different from prior behavior. And I imagine it will cause issues for many people downstream. I wouldn't dismiss the issue so quickly.

As for why pip thinks kiwitcms==10.0 is unsatisfiable, I would suggest looking into your dependency set and identify the conflicts. The environment variable PIP_RESOLVER_DEBUG may also help you look into what pip was doing during resolution.

I will dig into this, my assumption is some of the many packages in the mix depend on slightly different versions of the same package. And that's a problem for layered products like mine for example.

Do we have a way to force pip into not trying multiple versions other than pinning every single version across every single repository I have ?

[uranusjr]

That is a very big change in behavior and definitely different from prior behavior. And I imagine it will cause issues for many people downstream. I wouldn't dismiss the issue so quickly.

Yes, it was a big change introduced in the 20.3 release, and we publicalised the change as much as we could both before and after the release, for example:

https://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html
https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020

This stirred things a lot last year, it’s unfortunate you missed the message at the time.

Do we have a way to force pip into not trying multiple versions other than pinning every single version across every single repository I have?

We are discussing a --upgrade-strategy=never flag, but that will take some time to happen (especially since the legacy resolver has not gone away yet). With that said, it is very recommended for you to pin the requirements in your every single repository’s requirements file. This is considered package management best practice nowadays, and pip is encouraging this approach going forward, so it’s better to ride along than flight against it.

[ifokeev]

@uranusjr I did pin versions but it's still happening