Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test discovery fails when virtualenv name starts with 'python' #22932

Open
aitorciki opened this issue Feb 19, 2024 · 6 comments
Open

Test discovery fails when virtualenv name starts with 'python' #22932

aitorciki opened this issue Feb 19, 2024 · 6 comments
Assignees
@eleanorjboyd
Labels
area-testing bug Issue identified by VS Code Team member as probable bug

Comments

@aitorciki
Copy link

Type: Bug

Behaviour

Expected vs. Actual

When the active virtualenv's name starts with python (e.g. ./.venv/python-3.12/{bin,include,lib}), test discovery will fail trying to execute the discovery command using the venv directory as the binary:

[info] > ./.venv/python-3.12 -m pytest -p vscode_pytest --collect-only --color=yes

instead of

[info] > ./.venv/python-3.12/bin/python -m pytest -p vscode_pytest --collect-only --color=yes

If the virtualenv is manually re-selected (it's already active and working except for test discovery) before navigating to the Testing tab, discovery will complete successfully. If Testing is selected before explicit venv re-selection, discovery will never complete though:

[error] Test discovery already in progress, not starting a new one.

Filtering out directories named python* from binary selection was proposed in #21711, but was discarded due to the slow nature of statSync.

Output without explicit venv re-selection:

2024-02-19 19:51:33.190 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-02-19 19:51:33.190 [info] Experiment 'pythonREPLSmartSend' is active
2024-02-19 19:51:33.190 [info] Experiment 'pythonSurveyNotificationcf' is active
2024-02-19 19:51:33.190 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-02-19 19:51:33.190 [info] Experiment 'pythonTestAdapter' is active
2024-02-19 19:51:33.190 [info] Default formatter is set to charliermarsh.ruff for workspace /Users/aitor/tmp/vscode-test-discovery
2024-02-19 19:51:33.190 [info] VS Code was launched from an activated environment: 'python-3.12', selecting it as the interpreter for workspace.
2024-02-19 19:51:33.190 [info] Test server listening.
2024-02-19 19:51:33.190 [info] Python interpreter path: ./.venv/python-3.12/bin/python
2024-02-19 19:51:33.387 [info] > . ./.venv/python-3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariables.py
2024-02-19 19:51:33.387 [info] shell: zsh
2024-02-19 19:51:33.404 [info] > conda info --json
2024-02-19 19:51:33.414 [info] > /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariables.py
2024-02-19 19:51:33.415 [info] shell: zsh
2024-02-19 19:51:33.441 [info] > . ./.venv/python-3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariables.py
2024-02-19 19:51:33.441 [info] shell: bash
2024-02-19 19:51:33.456 [info] Send text to terminal: /usr/bin/python3 /Users/aitor/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariablesToFile.py /Users/aitor/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/deactivate/zsh/envVars.txt
2024-02-19 19:51:34.238 [info] Starting Pylance language server.
2024-02-19 19:51:34.548 [info] Discover tests for workspace name: vscode-test-discovery - uri: /Users/aitor/tmp/vscode-test-discovery
2024-02-19 19:51:34.548 [info] Running discovery for pytest using the new test adapter.
2024-02-19 19:51:34.548 [info] All environment variables set for pytest discovery for workspace /Users/aitor/tmp/vscode-test-discovery: [ redacted ]

2024-02-19 19:51:34.550 [info] > ./.venv/python-3.12 -m pytest -p vscode_pytest --collect-only --color=yes
2024-02-19 19:51:34.550 [info] cwd: .

After setting virtualenv manually:

2024-02-19 19:51:37.247 [info] Found: /usr/bin/python3 --> /usr/bin/python3
2024-02-19 19:51:37.248 [info] > ./.venv/python-3.12/bin/python -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:51:37.249 [info] Found: /Users/aitor/tmp/vscode-test-discovery/.venv/python-3.12/bin/python --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:51:37.250 [info] Found: /Users/aitor/tmp/vscode-test-discovery/.venv/python-3.12/bin/python3 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:51:37.250 [info] Found: /Users/aitor/tmp/vscode-test-discovery/.venv/python-3.12/bin/python3.12 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:51:37.251 [info] Found: /Users/aitor/.asdf/installs/python/3.12.2/bin/python --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:51:37.251 [info] Found: /Users/aitor/.asdf/installs/python/3.12.2/bin/python3 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:51:37.251 [info] Found: /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:51:37.253 [info] Found: /opt/homebrew/bin/python3 --> /opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
2024-02-19 19:51:37.253 [info] Found: /opt/homebrew/bin/python3.11 --> /opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
2024-02-19 19:51:37.253 [info] Found: /opt/homebrew/bin/python3.12 --> /opt/homebrew/Cellar/[email protected]/3.12.2/Frameworks/Python.framework/Versions/3.12/bin/python3.12
2024-02-19 19:51:37.257 [info] > /usr/bin/python3 -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:51:37.278 [info] > ~/.asdf/installs/python/3.12.2/bin/python -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:51:37.299 [info] > /opt/homebrew/bin/python3 -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:51:37.322 [info] > /opt/homebrew/bin/python3.12 -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:51:39.414 [info] Discover tests for workspace name: vscode-test-discovery - uri: /Users/aitor/tmp/vscode-test-discovery
2024-02-19 19:51:39.414 [info] Running discovery for pytest using the new test adapter.
2024-02-19 19:51:39.414 [error] Test discovery already in progress, not starting a new one.

Output with explicit venv re-selection:

2024-02-19 19:57:48.962 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-02-19 19:57:48.962 [info] Experiment 'pythonREPLSmartSend' is active
2024-02-19 19:57:48.962 [info] Experiment 'pythonSurveyNotificationcf' is active
2024-02-19 19:57:48.962 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-02-19 19:57:48.962 [info] Experiment 'pythonTestAdapter' is active
2024-02-19 19:57:48.962 [info] Default formatter is set to charliermarsh.ruff for workspace /Users/aitor/tmp/vscode-test-discovery
2024-02-19 19:57:48.962 [info] VS Code was launched from an activated environment: 'python-3.12', selecting it as the interpreter for workspace.
2024-02-19 19:57:48.962 [info] Test server listening.
2024-02-19 19:57:48.962 [info] Python interpreter path: ./.venv/python-3.12/bin/python
2024-02-19 19:57:49.183 [info] > conda info --json
2024-02-19 19:57:49.213 [info] > . ./.venv/python-3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariables.py
2024-02-19 19:57:49.214 [info] shell: zsh
2024-02-19 19:57:49.226 [info] > . ./.venv/python-3.12/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariables.py
2024-02-19 19:57:49.227 [info] shell: bash
2024-02-19 19:57:49.246 [info] > /usr/bin/python3 ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariables.py
2024-02-19 19:57:49.246 [info] shell: zsh
2024-02-19 19:57:49.435 [info] Send text to terminal: /usr/bin/python3 /Users/aitor/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/printEnvVariablesToFile.py /Users/aitor/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/deactivate/zsh/envVars.txt
2024-02-19 19:57:50.016 [info] Starting Pylance language server.
2024-02-19 19:57:50.574 [info] Found: /usr/bin/python3 --> /usr/bin/python3
2024-02-19 19:57:50.575 [info] > ./.venv/python-3.12/bin/python -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:57:50.576 [info] Found: /Users/aitor/tmp/vscode-test-discovery/.venv/python-3.12/bin/python --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:57:50.576 [info] Found: /Users/aitor/tmp/vscode-test-discovery/.venv/python-3.12/bin/python3 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:57:50.577 [info] Found: /Users/aitor/tmp/vscode-test-discovery/.venv/python-3.12/bin/python3.12 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:57:50.577 [info] Found: /Users/aitor/.asdf/installs/python/3.12.2/bin/python --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:57:50.577 [info] Found: /Users/aitor/.asdf/installs/python/3.12.2/bin/python3 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:57:50.577 [info] Found: /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12 --> /Users/aitor/.asdf/installs/python/3.12.2/bin/python3.12
2024-02-19 19:57:50.579 [info] Found: /opt/homebrew/bin/python3 --> /opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
2024-02-19 19:57:50.579 [info] Found: /opt/homebrew/bin/python3.11 --> /opt/homebrew/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/3.11/bin/python3.11
2024-02-19 19:57:50.579 [info] Found: /opt/homebrew/bin/python3.12 --> /opt/homebrew/Cellar/[email protected]/3.12.2/Frameworks/Python.framework/Versions/3.12/bin/python3.12
2024-02-19 19:57:50.582 [info] > /usr/bin/python3 -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:57:50.600 [info] > ~/.asdf/installs/python/3.12.2/bin/python -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:57:50.622 [info] > /opt/homebrew/bin/python3 -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:57:50.650 [info] > /opt/homebrew/bin/python3.12 -I ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/get_output_via_markers.py ~/.vscode/extensions/ms-python.python-2024.0.1/pythonFiles/interpreterInfo.py
2024-02-19 19:57:52.242 [info] Discover tests for workspace name: vscode-test-discovery - uri: /Users/aitor/tmp/vscode-test-discovery
2024-02-19 19:57:52.242 [info] Running discovery for pytest using the new test adapter.
2024-02-19 19:57:52.244 [info] All environment variables set for pytest discovery for workspace /Users/aitor/tmp/vscode-test-discovery: [ redacted ] 

2024-02-19 19:57:52.247 [info] > ./.venv/python-3.12/bin/python -m pytest -p vscode_pytest --collect-only --color=yes
2024-02-19 19:57:52.247 [info] cwd: .
2024-02-19 19:57:52.357 [info] �[1m============================= test session starts ==============================�[0m
platform darwin -- Python 3.12.2, pytest-8.0.1, pluggy-1.4.0
rootdir: /Users/aitor/tmp/vscode-test-discovery

2024-02-19 19:57:52.358 [info] collected 1 item

<Dir vscode-test-discovery>
  <Module test_stuff.py>
    <Function test_foo>

2024-02-19 19:57:52.361 [info] Test server connected to a client.
2024-02-19 19:57:52.361 [info] 
�[32m========================== �[32m1 test collected�[0m�[32m in 0.01s�[0m�[32m ===========================�[0m

2024-02-19 19:57:52.365 [info] ResultResolver EOT received for discovery.
2024-02-19 19:57:52.376 [info] Disposing data receiver for /Users/aitor/tmp/vscode-test-discovery and deleting UUID; pytest discovery.

Steps to reproduce:

  1. python -m venv .venv/python-3.12
  2. source .venv/python-3.12/bin/activate
  3. pip install pytest
  4. echo "def test_foo():\n assert True == True" > test_stuff.py
  5. code .

Diagnostic data

  • Python version (& distribution if applicable, e.g. Anaconda): 3.12.2
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
  • Value of the python.languageServer setting: Default
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Pasted in issue description.

User Settings 


languageServer: "Pylance"

testing
• pytestArgs: "--color=yes"
• pytestEnabled: true

Extension version: 2024.0.1
VS Code version: Code 1.86.2 (903b1e9d8990623e3d7da1df3d33db3e42d80eda, 2024-02-13T19:42:12.210Z)
OS version: Darwin arm64 23.3.0
Modes:

System Info
Item Value
CPUs Apple M2 Pro (10 x 24)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgl2: enabled
webgpu: enabled
Load (avg) 2, 2, 2
Memory (System) 16.00GB (0.07GB free)
Process Argv --crash-reporter-id 05132e86-5a47-43bc-81a1-8feaf931ac4e
Screen Reader no
VM 0%
A/B Experiments 
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscorecescf:30445987
vscod805:30301674
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:30899288
c4g48928:30535728
azure-dev_surveyone:30548225
a9j8j154:30646983
962ge761:30959799
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
2e7ec940:30962249
pythontbext0:30879054
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
b1063948:30958189
fegfb526:30961430
7j2b6412:30967274
bg6jg535:30958357
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Feb 19, 2024
@eleanorjboyd
Copy link
Member

@karrtikr any thoughts here on the environment selection? Seems like it is selecting the wrong environment as the interpreter.

@github-actions github-actions bot added the info-needed Issue requires more information from poster label Mar 27, 2024
@karrtikr
Copy link

I think the issue is here:

vscode-python/src/client/testing/testController/pytest/pytestDiscoveryAdapter.ts

Line 112 in 27783ce

const execService = await executionFactory?.createActivatedEnvironment(creationOptions);

When we call into that, we're using a deprecated API .pythonPath should no longer be used:

vscode-python/src/client/common/process/pythonExecutionFactory.ts

Line 114 in e9edfc0

: this.configService.getSettings(options.resource).pythonPath;

Instead we should call into IInterpreterService.getActiveInterpreter() API.

@karrtikr karrtikr removed their assignment Mar 27, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 7, 2024

Hey @eleanorjboyd, this issue might need further attention.

@aitorciki, you can help us out by closing this issue if the problem no longer exists, or adding more information.

@bhenderson-r7
Copy link

bhenderson-r7 commented Jun 12, 2024 •
edited
Loading

FWIW I'm experiencing the same issue with the use of direnv

[info] > ./.direnv/python-3.8 -m pytest -p vscode_pytest --collect-only

instead of

[info] > ./.direnv/python-3.8/bin/python -m pytest -p vscode_pytest --collect-only

@aitorciki
Copy link
Author

I can also still reproduce on latest versions of VSCode and the Python extension, yes.

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label Jun 15, 2024
@eleanorjboyd eleanorjboyd removed the triage-needed Needs assignment to the proper sub-team label Dec 16, 2024
@rousseldenis
Copy link

rousseldenis commented Dec 19, 2024 •
edited
Loading

I think I have the same problem but I don't have a venv name with same string.

python 3.12.3 on the system, python 3.11.11 for virtual environment.

@karthiknadig karthiknadig added the bug Issue identified by VS Code Team member as probable bug label Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eleanorjboyd eleanorjboyd

Labels
area-testing bug Issue identified by VS Code Team member as probable bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@aitorciki @karthiknadig @karrtikr @rousseldenis @eleanorjboyd @bhenderson-r7