Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate switching to urllib3 #2751

Open
jku opened this issue Dec 11, 2024 · 1 comment · May be fixed by #2762
Open

Investigate switching to urllib3 #2751

jku opened this issue Dec 11, 2024 · 1 comment · May be fixed by #2762

Comments

@jku
Copy link
Member

jku commented Dec 11, 2024 •
edited
Loading

We currently use requests in the default ngclient fetcher (RequestsFetcher).

Internally requests uses urllib3 and we don't seem to use any functionality outside of what urllib3 provides: connection pool management and streaming are supported. Switching to urllib3 would mean:

  • we would have (at least) one less dependency
  • the API we would use is arguably better maintained

This could be achieved by first writing an alternative FetcherInterface implementation that uses urllib3: if it looks good we can then make it the default and remove the requests-based implementation.
@NicholasTanz NicholasTanz linked a pull request Jan 6, 2025 that will close this issue
Open
@jku
Copy link
Member Author

jku commented Jan 29, 2025 •
edited
Loading

Documenting findings from the PR:

  • urllib3 API is almost a 1:1 match
  • the one exception is that requests uses certifi for TLS certs and urllib3 by default does not. We could keep using certifi but this seems like a good time to switch to system certificates instead. Reasons are well documented in https://truststore.readthedocs.io/en/latest/
  • We likely want to support RequestsFetcher for at least one release even if the new fetcher is the default to provide easy workaround in case users have issues with the new default
  • Once we can remove the requests dependency we also drop idna, charset-normalizer and certifi: this feels like a win to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

replace RequestsFetcher for Urllib3Fetcher NicholasTanz/python-tuf
1 participant
@jku