VCS install using https credentials in env variables fails for Pipfile #6195
Comments
dennisvang
changed the title
dennisvang
changed the title
|
I am wondering if anyone can setup a sample private repo for testing this case -- I'd like to help improve the code around it. |
|
@matteius I've tried to reproduce this using one of my private github repos, because those are easier to set up. However, in this case For example, pip install "git+https://${USERNAME}:${PASSWORD}@github.com/myusername/my-private-repo.git"but pipenv install "git+https://${USERNAME}:${PASSWORD}@github.com/myusername/my-private-repo.git"fails with It looks unrelated to the present issue. Should I open a new issue for this? |
matteius
added
Category: Security
|
@dennisvang Could you test this report against this branch with I believe does better with the VCS env variables: #6242 |
Analysis of Issue #6195:1. Problem SummaryThe issue reports a discrepancy in Pipenv's handling of HTTPS credentials for VCS installations when specified via environment variables. While direct command line installation with embedded credentials works as expected, using the same URL format with environment variables in a 2. Comment Discussion Analysis
3. Proposed ResolutionThe issue likely stems from Pipenv not correctly substituting environment variables within the Here's a detailed breakdown:
4. Potential Code ChangesEnhance from pipenv.utils.shell import safe_expandvars
class Project:
# ... existing code ...
def get_vcs_deps(self, dev=False):
# ... existing code ...
# Expand environment variables in the URL.
for k, v in packages.items():
if is_vcs(v) or is_vcs(k):
if isinstance(v, dict):
for key, value in v.items():
if key in VCS_LIST:
v[key] = safe_expandvars(value)
return packages or {}
# ... existing code ...This modification expands environment variables within the VCS URL retrieved from the 5. Additional Steps/Investigations
By addressing the variable expansion issue and potentially refining VCS credential management, Pipenv can ensure consistent and secure handling of private repositories, enhancing its usability and reliability for diverse projects and environments. |
|
Could you check @dennisvang if #6276 solves for this (I think it should). |
Issue description
Trying to install a package over https, from a private AWS CodeCommit repo, on an Ubuntu system.
Instead of using git
credential.helper, we would like to inject credentials into our Pipfile using env variables.So, in a bash terminal, we define
USERNAMEandPASSWORD(url-encoded).Installing into an empty dir, directly from the command line, using these credentials, appears to work without issue:
pipenv install -e "git+https://${USERNAME}:${PASSWORD}@git-codecommit.eu-west-3.amazonaws.com/v1/repos/mypackage@main#egg=mypackage"This implies that the credentials and user permissions are correct.
However, it turns out the username and password end up in both
PipfileandPipfile.lock...That does not sound like a good idea.
Luckily, the docs for Injecting credentials into Pipfile via environment variables say:
However, if we try to install from a
Pipfileinto an otherwise empty dir, using the exact same url with sameUSERNAMEandPASSWORD, we get a status403(Forbidden):yields
Expected result
I expect installation from
Pipfileto work without issue, just like installation from the command line.Actual result
command line output
Steps to replicate
first
pyproject.toml, as follows, and setup a user with git credentials (perhaps a private github repo would also work, but haven't tried that yet)USERNAMEandPASSWORD(e.g. usingread -s) with the corresponding values (url-encoded)pipenv install -e "git+https://${USERNAME}:${PASSWORD}@git-codecommit.eu-west-3.amazonaws.com/v1/repos/mypackage@main#egg=mypackage"then
Pipfile, generated above, into the new dir, and replace the actual username and password in the file by the corresponding env variables, as in:pipenv installstatus 403error$ pipenv --support
Pipenv version:
'2024.0.1'Pipenv location:
'/home/me/.local/lib/python3.11/site-packages/pipenv'Python location:
'/home/me/.pyenv/versions/3.11.6/bin/python3.11'OS Name:
'posix'User pip version:
'24.0'user Python installations found:
PEP 508 Information:
System environment variables:
SHELLSESSION_MANAGERQT_ACCESSIBILITYPIPENV_VENV_IN_PROJECTCOLORTERMPYENV_SHELLXDG_CONFIG_DIRSSSH_AGENT_LAUNCHERXDG_MENU_PREFIXGNOME_DESKTOP_SESSION_IDLANGUAGELC_ADDRESSGNOME_SHELL_SESSION_MODELC_NAMESSH_AUTH_SOCKGIT_PS1_SHOWDIRTYSTATEXMODIFIERSDESKTOP_SESSIONLC_MONETARYGTK_MODULESPWDLOGNAMEXDG_SESSION_DESKTOPXDG_SESSION_TYPESYSTEMD_EXEC_PIDXAUTHORITYHOMEUSERNAMEIM_CONFIG_PHASELC_PAPERLANGLS_COLORSXDG_CURRENT_DESKTOPVTE_VERSIONWAYLAND_DISPLAYGNOME_TERMINAL_SCREENGNOME_SETUP_DISPLAYLESSCLOSEXDG_SESSION_CLASSTERMLC_IDENTIFICATIONLESSOPENUSERGNOME_TERMINAL_SERVICEDISPLAYSHLVLLC_TELEPHONEQT_IM_MODULELC_MEASUREMENTPAPERSIZEXDG_RUNTIME_DIRPYENV_ROOTLC_TIMEXDG_DATA_DIRSPATHGDMSESSIONDBUS_SESSION_BUS_ADDRESSLC_NUMERIC_PIP_DISABLE_PIP_VERSION_CHECKPYTHONDONTWRITEBYTECODEPYTHONFINDER_IGNORE_UNSUPPORTEDPipenv–specific environment variables:
PIPENV_VENV_IN_PROJECT:1Debug–specific environment variables:
PATH:/home/me/.pyenv/versions/3.9.18/bin:/home/me/.pyenv/versions/3.8.18/bin:/home/me/.pyenv/versions/3.11.6/bin:/home/me/.pyenv/versions/3.8.13/bin:/home/me/.pyenv/shims:/home/me/.npm-global/bin:/home/me/.local/bin:/home/me/.pyenv/bin:/home/me/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/binSHELL:/bin/bashLANG:en_CA.UTF-8PWD:/home/meThe text was updated successfully, but these errors were encountered: