Extension v2024.4.0 and v2024.4.1 create indefinite "Discover Python Interpreters" #23452

dokutoshi · 2024-05-20T01:26:04Z

Type: Bug

Behaviour

Expected vs. Actual

Install v2024.4.0 or v2024.4.1 extension and the discovering Python interpreters run indefinitely. This also restarts the kernel periodically - a MAJOR PROBLEM when excecuting ML notebooks - lost variables imply retraining or reloading a pandas DF etc.
XXX

Steps to reproduce:

To stop the kernel from periodic restarts use v2024.2.1

Diagnostic data

Python version (& distribution if applicable, e.g. Anaconda): 3.11.9
Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
Value of the python.languageServer setting: Pylance

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

User Settings


languageServer: "Pylance"

Extension version: 2024.2.1
VS Code version: Code 1.88.0 (5c3e652f63e798a5ac2f31ffd0d863669328dc4c, 2024-04-03T13:28:09.495Z)
OS version: Darwin arm64 23.5.0
Modes:

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	5, 5, 5
Memory (System)	16.00GB (0.05GB free)
Process Argv	--crash-reporter-id aaabbd8f-9170-4ae6-b93a-f11487600abe
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
tftest:31042121
vswsl492cf:30256860
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pythonmypyd1:30879173
pythoncet0:30885854
2e7ec940:31000449
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
chatpanelc:31048052
dsvsc021:30996838
9c06g630:31013171
fchga941:31046352
pythonprc:31047982
dwnewjupytercf:31046870
26j00206:31048877

The text was updated successfully, but these errors were encountered:

anthonykim1 · 2024-05-20T18:41:33Z

Hi @dokutoshi. Thank you for filing the issue. Can you set your log level to trace in command palette and share your Python output info accessible in output for Python?

dcx · 2024-05-21T07:15:55Z

I'm having this issue too, and there's nothing available in the Output dialog.

anthonykim1 · 2024-05-22T18:48:37Z

Interesting..Does Clear Cache and reload Python extension fix the problem?
(Accessible via ctrl+shift+p) or cmd+shift+p on mac.

anthonykim1 · 2024-05-22T18:48:52Z

Also wondering if problem persists in version below v2024.4.0

dokutoshi · 2024-06-01T15:49:42Z

I will look into this later today. Prescriptive measure include setting log level == 'trace' and cmd+shift+p On May 22, 2024, at 14:49, Anthony Kim ***@***.***> wrote: Also wondering if problem persists in version below v2024.4.0 — Reply to this email directly, view it on GitHub<#23452 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACNWBJOTDMYMQLMVSSOMG6TZDTSCVAVCNFSM6AAAAABH63ATVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRVGUZDMNRRGI>. You are receiving this because you were mentioned.Message ID: ***@***.***> This email and any attachments may contain private, confidential and privileged material for the sole use of the intended recipient. Any review, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and delete this email and any attachments afterwards.

dokutoshi · 2024-06-04T13:58:07Z

Clearing the cache and reloading the window does solve the infinite running issue.

However, the reason i opened this ticket relates to this bug:

48 hours into testing, Discovering process starts up and terminates a running job. ML training takes weeks. What happened? Using VSCode worked for over 2 years in DL training. Now I can barely use it for a day without Discovering Python Interpreters appearing and halting training. @anthonykim1

anthonykim1 · 2024-06-04T17:54:41Z

@dokutoshi Can you bring the requested logs and also check if same problem exists in version below 2024.4.0 that you have mentioned?

dokutoshi · 2024-06-04T18:40:11Z

Dealing with a related issue in Vscode Insiders that crashes Pylance and triggers Discovering Python Interpreters.
note the error using the -I flag in the Vscode code base.
Pylance Crash

For reference, here is the Python log output.

2024-06-04 18:31:35.095 [info] shell: bash
2024-06-04 18:31:35.732 [info] > /usr/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py
2024-06-04 18:31:35.802 [info] > ./miniforge-pypy3/envs/tf_2.15/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py

2024-06-04 18:31:35.816 [warning] [Error: Command failed: /usr/bin/python -I /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py
Unknown option: -I
usage: /usr/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.

at ChildProcess.exithandler (node:child_process:423:12)
at ChildProcess.emit (node:events:530:35)
at maybeClose (node:internal/child_process:1105:16)
at Socket.<anonymous> (node:internal/child_process:457:11)
at Socket.emit (node:events:518:28)
at Pipe.<anonymous> (node:net:337:12)] {

code: 2,
killed: false,
signal: null,
cmd: '/usr/bin/python -I /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py'
}
2024-06-04 18:31:35.816 [error] Support for Python 2.7 has been dropped by the Python extension so certain features may not work, upgrade to using Python 3.
2024-06-04 18:31:35.843 [info] > /usr/bin/python ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py
2024-06-04 18:31:36.115 [info] > ./miniforge-pypy3/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py
2024-06-04 18:31:36.279 [info] > /usr/local/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py
2024-06-04 18:31:36.554 [info] Starting Pylance language server.

dokutoshi · 2024-06-04T18:42:20Z

Sorry venting here but when the Discovery process starts, it is impossible to save the file.

Even Save As fails silently!

dokutoshi · 2024-06-04T19:10:31Z

For clarification, the Discovering Python Interpreters occurs from Python 2024.2.1 version up to and including the most recent version of VsCode Insiders.
@anthonykim1 - The answer is Yes!

Both local (Mac) and remote SSH (from Mac to Linux) exhibit the issues.

Version: 1.84.2
Commit: 1a5daa3a0231a0fbba4f14db7ec463cf99d7768e
Date: 2023-11-09T10:52:57.054Z
Electron: 25.9.2
ElectronBuildId: 24603566
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Darwin arm64 23.3.0
Python: Extension version: 2024.2.1

Ahh, ran extension profiler.

#23554

anthonykim1 · 2024-06-04T20:35:44Z

@dokutoshi Thanks for the requested logs.
Are you using Python2.7? I see

 [error] Support for Python 2.7 has been dropped by the Python extension so certain features may not work, upgrade to using Python 3.

from your log.

Support for 2.7 has been dropped by Python extension. Reference: https://github.com/microsoft/vscode-python/wiki/Limited-support-for-Python-2.7

dokutoshi · 2024-06-06T03:05:46Z

Python

No, I use python 3.10 - 3.12.

Conda environments established on the ec2. Why is vscode searching the ec2 for possible python directories outside of the designated environment?

Best,

Leo

dokutoshi · 2024-06-07T01:43:44Z

Btw, is there a method to suppress the discovery process e..g "Discovering Python Interpreters". The process appears event or timer based. is there a method to make this process interactive i.e. user controlled?

anthonykim1 · 2024-06-07T22:46:31Z

Conda environments established on the ec2. Why is vscode searching the ec2 for possible python directories outside of the designated environment?

It is not necessarily VS Code, it would be Python extension or other extension IN VS Code that is triggering interpreter discovery.

Btw, is there a method to suppress the discovery process e..g "Discovering Python Interpreters". The process appears event

I don't think so. The problem you are having can happen when some other extension triggers discovery, or there is some changes in the Python environment that you have set up. For now you can pin the extension to older version where everything used to work for you until we investigate this further.

I will keep this issue open just so I can get a census/reports of to the extent other users are experiencing the issue.

dokutoshi · 2024-06-09T05:36:30Z

i launched a new EC2 - Ubuntu 22.04 built on Python3 to avoid the interrupt encountered with Python 2.7.

Something is triggering the discovery and... the designated kernel is no longer stable.
I use only Microsoft extensions.

This is problematic.

dokutoshi · 2024-06-09T05:47:38Z

Maybe the Python Debugger extension @anthonykim1

dokutoshi · 2024-06-09T17:47:27Z

Figured out the bug:

Here is the latest @anthonykim1 ... it appears VScode's Python extension joins the local python environments and remote python environment directories, perhaps into a list of paths.
If the directories are the same on the local and remote, no issue.
If there are paths on the local, not present on the remote, the python extension crashes as the Discover Python Interpreters enters an infinite loop.

Remote device: Python Log
2024-06-09 17:41:28.668 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Remote device:
`ubuntu@ip-172-31-92-108:/mnt_data/home/ec2-user$ which -a python python2 python3

/usr/bin/python3
/bin/python3`

Local device:
`ec2-user@ip-172-31-0-196 homebrew % which -a python python2 python3
python not found

python2 not found

/opt/homebrew/bin/python3
/usr/bin/python3`

Setup:

An AWS Mac acts as a bastion device to remote ssh to a Linux device. [Not sure that matters]
The Python extension uses scandir on the remote Linux device and on the localhost device. It then searches for all the absolute paths in the list on the remote device irrespective if the path is only on the local machine. @anthonykim1

Temporary Solution:
`# Create the necessary directory if it doesn't exist
sudo mkdir -p /opt/homebrew/bin

Create the symbolic link

sudo ln -s /usr/bin/python3 /opt/homebrew/bin/python3

Verify the symlink

ls -l /opt/homebrew/bin/python3`

Result of Python discovery:
Pre-fix:
2024-06-09 17:50:36.354 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] {
errno: -2,
code: 'ENOENT',
syscall: 'scandir',
path: '/opt/homebrew/bin'
}

Post-fix:
2024-06-09 17:55:57.165 [info] Experiment 'pythonCreateEnvOnPipInstall' is active
2024-06-09 17:55:57.165 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-06-09 17:55:57.165 [info] Experiment 'pythonSurveyNotification' is active
2024-06-09 17:55:57.165 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-06-09 17:55:57.165 [info] Experiment 'pythonTestAdapter' is active
2024-06-09 17:55:57.165 [info] Default formatter is set to ms-python.black-formatter for workspace /mnt_data/home/ec2-user
2024-06-09 17:55:57.165 [info] Starting Environment refresh
2024-06-09 17:55:57.165 [info] Searching for interpreters in posix paths locator
2024-06-09 17:55:57.165 [info] Searching for pyenv environments
2024-06-09 17:55:57.165 [info] Searching for conda environments
2024-06-09 17:55:57.165 [info] Searching for global virtual environments
2024-06-09 17:55:57.165 [info] Searching for custom virtual environments
2024-06-09 17:55:57.165 [info] pyenv is not installed
2024-06-09 17:55:57.165 [info] Finished searching for pyenv environments: 339 milliseconds
2024-06-09 17:55:57.165 [info] Finished searching for custom virtual envs: 339 milliseconds
2024-06-09 17:55:57.165 [info] > conda info --json
2024-06-09 17:55:57.165 [info] > hatch env show --json
2024-06-09 17:55:57.165 [info] cwd: .
2024-06-09 17:55:57.165 [info] Finished searching for global virtual envs: 361 milliseconds
2024-06-09 17:55:57.287 [info] Found: /bin/python3 --> /bin/python3.10
2024-06-09 17:55:57.368 [info] Found: /bin/python3.10 --> /bin/python3.10
2024-06-09 17:55:57.444 [info] Found: /usr/bin/python3 --> /usr/bin/python3.10
2024-06-09 17:55:57.449 [info] Found: /usr/bin/python3.10 --> /usr/bin/python3.10
2024-06-09 17:55:57.457 [info] > /opt/homebrew/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py
2024-06-09 17:55:57.487 [info] Python interpreter path: /opt/homebrew/bin/python3
2024-06-09 17:55:57.503 [info] Found: /home/ubuntu/miniforge3/bin/python --> /home/ubuntu/miniforge3/bin/python3.10
2024-06-09 17:55:57.509 [info] Found: /home/ubuntu/miniforge3/bin/python3 --> /home/ubuntu/miniforge3/bin/python3.10
2024-06-09 17:55:57.510 [info] Found: /home/ubuntu/miniforge3/bin/python3.1 --> /home/ubuntu/miniforge3/bin/python3.10
2024-06-09 17:55:57.515 [info] Found: /home/ubuntu/miniforge3/bin/python3.10 --> /home/ubuntu/miniforge3/bin/python3.10
2024-06-09 17:55:57.549 [info] > /bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py
2024-06-09 17:55:57.598 [info] Finished searching for interpreters in posix paths locator: 793 milliseconds
2024-06-09 17:55:57.598 [info] > /usr/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py
2024-06-09 17:55:58.193 [info] Finished searching for conda environments: 1389 milliseconds

dokutoshi · 2024-06-09T18:32:58Z

There is still an issue... it is not possible to save the file while the Discovery process executes.

dokutoshi · 2024-06-10T23:59:52Z

Is there an approach to limit the Python environment to a specific path e.g. in a workspace environment file?

{ "python.venvPath": "/path/to/your/allowed/venvs", "python.autoComplete.extraPaths": [ "/path/to/your/allowed/venvs" ] }

dokutoshi · 2024-06-11T03:39:00Z

Really appreciate some help here @anthonykim1
Several articles of users experiencing similar issues:

https://stackoverflow.com/questions/78251441/visual-studio-code-keeps-discovering-python-interpreters-forever-and-vscode-serv
https://stackoverflow.com/questions/71153859/discovering-python-interpreters-taking-infinite-time-in-vs-code
Sitting at "Discovering Python Interpreters" every reload vscode-jupyter#12974 @DonJayamanne
Discovering python interpreters never finishes due to presence of symbolic link loop #21310 - Recall there is a symbolic link as VScode searches for the local path on the remote machine. ((
Stuck on Discovering Python Interpreters #20250

Issues were not solved. I suggest a look at the possible solution.

Figured out the bug:

Here is the latest @anthonykim1 ... it appears VScode's Python extension joins the local python environments and remote python environment directories, perhaps into a list of paths. If the directories are the same on the local and remote, no issue. If there are paths on the local, not present on the remote, the python extension crashes as the Discover Python Interpreters enters an infinite loop.

Remote device: Python Log 2024-06-09 17:41:28.668 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Remote device: `ubuntu@ip-172-31-92-108:/mnt_data/home/ec2-user$ which -a python python2 python3

/usr/bin/python3 /bin/python3`

Local device: `ec2-user@ip-172-31-0-196 homebrew % which -a python python2 python3 python not found

python2 not found

/opt/homebrew/bin/python3 /usr/bin/python3`

Setup:

An AWS Mac acts as a bastion device to remote ssh to a Linux device. [Not sure that matters]

The Python extension uses scandir on the remote Linux device and on the localhost device. It then searches for all the absolute paths in the list on the remote device irrespective if the path is only on the local machine. @anthonykim1

Temporary Solution: `# Create the necessary directory if it doesn't exist sudo mkdir -p /opt/homebrew/bin

Create the symbolic link

sudo ln -s /usr/bin/python3 /opt/homebrew/bin/python3

Verify the symlink

ls -l /opt/homebrew/bin/python3`

Result of Python discovery: Pre-fix: 2024-06-09 17:50:36.354 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Post-fix: 2024-06-09 17:55:57.165 [info] Experiment 'pythonCreateEnvOnPipInstall' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonRecommendTensorboardExt' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonSurveyNotification' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonTerminalEnvVarActivation' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonTestAdapter' is active 2024-06-09 17:55:57.165 [info] Default formatter is set to ms-python.black-formatter for workspace /mnt_data/home/ec2-user 2024-06-09 17:55:57.165 [info] Starting Environment refresh 2024-06-09 17:55:57.165 [info] Searching for interpreters in posix paths locator 2024-06-09 17:55:57.165 [info] Searching for pyenv environments 2024-06-09 17:55:57.165 [info] Searching for conda environments 2024-06-09 17:55:57.165 [info] Searching for global virtual environments 2024-06-09 17:55:57.165 [info] Searching for custom virtual environments 2024-06-09 17:55:57.165 [info] pyenv is not installed 2024-06-09 17:55:57.165 [info] Finished searching for pyenv environments: 339 milliseconds 2024-06-09 17:55:57.165 [info] Finished searching for custom virtual envs: 339 milliseconds 2024-06-09 17:55:57.165 [info] > conda info --json 2024-06-09 17:55:57.165 [info] > hatch env show --json 2024-06-09 17:55:57.165 [info] cwd: . 2024-06-09 17:55:57.165 [info] Finished searching for global virtual envs: 361 milliseconds 2024-06-09 17:55:57.287 [info] Found: /bin/python3 --> /bin/python3.10 2024-06-09 17:55:57.368 [info] Found: /bin/python3.10 --> /bin/python3.10 2024-06-09 17:55:57.444 [info] Found: /usr/bin/python3 --> /usr/bin/python3.10 2024-06-09 17:55:57.449 [info] Found: /usr/bin/python3.10 --> /usr/bin/python3.10 2024-06-09 17:55:57.457 [info] > /opt/homebrew/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:57.487 [info] Python interpreter path: /opt/homebrew/bin/python3 2024-06-09 17:55:57.503 [info] Found: /home/ubuntu/miniforge3/bin/python --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.509 [info] Found: /home/ubuntu/miniforge3/bin/python3 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.510 [info] Found: /home/ubuntu/miniforge3/bin/python3.1 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.515 [info] Found: /home/ubuntu/miniforge3/bin/python3.10 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.549 [info] > /bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:57.598 [info] Finished searching for interpreters in posix paths locator: 793 milliseconds 2024-06-09 17:55:57.598 [info] > /usr/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:58.193 [info] Finished searching for conda environments: 1389 milliseconds

dokutoshi · 2024-06-12T14:56:09Z

Turned on Trace log

Why is VS Code looking for node_modules under the home directory e.g. ~/node_modules?
What is setActiveFolders? Related open ticket: command 'remote-internal.setActiveFolders' not found vscode-remote-release#7178

Hypothesis: Some disconnect occurs during Remote SSH or Tunnels. I guess the reconnection triggers the Python Discovery process @anthonykim1

AWS support recommended this ticket is related: microsoft/vscode-remote-release#9832

These errors occur for both Remote SSH or Remote Tunnels.

Errors

**2024-06-12 05:31:40.234 [error] command 'remote-internal.setActiveFolders' not found: Error: command 'remote-internal.setActiveFolders' not found at v.n (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1637:23867)** at v.executeCommand (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1637:23796) 2024-06-12 05:31:41.198 [warning] [remote-ssh]: Couldn't find message for key setting.usseExecServer.description. 2024-06-12 05:31:41.248 [trace] lifecycle: phase changed (value: 4) 2024-06-12 05:31:41.274 [trace] ForwardedPorts: Doing first restore. 2024-06-12 05:31:41.288 [trace] ForwardedPorts: (TunnelModel) restoring ports 6006 **2024-06-12 05:31:41.310 [trace] Error: ENOENT: no such file or directory, stat '/mnt_data/home/ec2-user/node_modules':** EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, stat '/mnt_data/home/ec2-user/node_modules' at k.create (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:114:97841) at I (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:114:97955) at t.hb (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:119:35958) at t.stat (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:119:28306)

dokutoshi · 2024-06-13T13:30:44Z

Please take a look at this @karthiknadig . I believe it relates to concerns in this ticket. #21639
The process never terminates on a remote server.
Steps to recreate

Connect to a remote server
Command Palette - select Python interpreter
Observe Discover Python Interpreters start
Select interpreter or close it.
Observe Discover Python Interpreters does not terminate

dokutoshi · 2024-06-13T17:43:44Z

May hint at the a test scenario @karthiknadig

Visual Studio Code keeps discovering python interpreters forever and vscode-server on remote is busy 100%

Solution proposed on Mar 31, 2024
It is common for computer vision scientists to have directories with millions of images. I will consider moving the directory, this can take 1-2 days for 4 million images. This is not preferred for many reasons such as updating code base etc.

Eventually the remote SSH is dropped because the Discovery Crashes the IDE. Then the process starts over again. Extremely frustrating.

"Discovering Python Interpreters" taking Infinite time in VS Code
Unsolved
Stuck at discovering Python Interpreters in ssh connection
Ticket closed but unsolved

dokutoshi · 2024-06-13T18:49:15Z

Solved! Followed the answer to Number 1. I moved the notebook and other code source files to another project. The image directory with millions of files is outside of the project folder.

Why is the discovery process scanning all the subdirectories???? @karthiknadig @DonJayamanne @anthonykim1

karthiknadig · 2024-06-13T20:02:15Z

It looks for virtual environments created for monorepos this way. As I said we are re-doing this to be more efficient. But, basically some projects have several virtual environments for each sub folder, that we need to support. We are changing this to handle cases like yours better.

dokutoshi · 2024-06-14T06:48:12Z

@karthiknadig it is far simpler.
@DonJayamanne @anthonykim1

I was able to recreate the issue locally.

Set up a project with a data folder, src folder.
Data folder create 1,000,000 images, one can probably generate doc1.txt - doc999999.txt.
In the src folder, add python file and jupyter notebook, keep it simple.
Select interpreter at project level. Note the time it takes for the kernel to stop spinning. It can be at least 1 - 2 minutes. Sometimes i get a cup of tea.
Close folder and reopen at src folder. Vola, select interpreter. It is instant!

I was able to recreate this on a few machines - local and remote are both faster when using only the src folder and not including the data folder (1,000,000+ images, files, etc)

Best,

Leo

dokutoshi · 2024-06-18T16:09:16Z

One final question - @karthiknadig @anthonykim1. Why not allow the "folders to exclude" feature to limit the search path for the python interpreters or better yet, allow the user to customize? It is not advised for the sentinel to search across a device; there are corporate concerns with such activity that will prevent the use of the tool.

karthiknadig · 2024-06-18T17:11:55Z

We do have plans on adding customizable excludes in the new work that is being done for environment discovery. But the old discovery will likely not get update to add the new features to it.

dokutoshi · 2024-06-25T02:03:07Z

Is the current solution to limit the number of files inside a workspace e.g. remove large data i.e. image or corpus directories?

karthiknadig · 2024-06-25T02:05:14Z

Unfortunately yes.

dokutoshi · 2024-06-29T23:17:33Z

@karthiknadig - Does Visual Studio have the same limitations i.e. python interpreter scans all files in workspace?

ahh, not a long term solution anyhow... https://learn.microsoft.com/en-us/visualstudio/mac/what-happened-to-vs-for-mac?view=vsmac-2022

karthiknadig · 2024-07-01T17:01:40Z

@dokutoshi For long term we are going to switch to a different mechanism for discovery and one of the things we are seeing there is providing a way to exclude certain directories, in addition to improving the search performance.

dokutoshi · 2024-07-01T17:49:45Z

@karthiknadig that would be most excellent. From a corporate perspective, the current limitation created the following situation:

Migrate all AWS Linux 2 servers to Ubuntu as Linux 2 contains a python 2.7 env, albeit unused.
Load only the source directory. Exclude Directories such as TF folders, parquet and csv data source, and other meta libraries from the workspace. In addition, data folders were included.

These are significant changes at a corporate level . Ubuntu brings its own inherent limitations non-XFS etc that required further steps to support ML workflows.

dokutoshi · 2024-07-16T15:46:11Z

the bot closed the open ticket after 3 days - that is too short a window - please adjust the bot.

Python Uncaught Error 2024.2.1](#23554 (comment))

Clearing the cache with each reload is standard practice.

One important test scenario:

@karthiknadig @DonJayamanne

I extended the workspace using "Add Folder to Workspace" to include a functional folder for TensorFlow.

For clarification:

The first folder contains Python source files and a notebook (a static folder).

The second folder (the TensorFlow folder) contains logs and metadata files generated by TensorFlow during training, typically per epoch, to save model weights and biases (a dynamic folder). PyTorch has a similar on-training file creation.

Post-observations after including a dynamic folder show that the growth in file count (less than 10 files) during an active kernel leads to a kernel halt. This is different, in my opinion, from including a large data directory with millions of images. In the dynamic folder case, behavior suggests that files added/created in an active workspace trigger the Python discovery process, interrupting an active kernel.

Best,

Leo

anthonykim1 · 2024-12-09T20:18:15Z

Are you still able to repro this?
Also wondering if changing from native => js changes any behavior.

github-actions bot added the triage-needed Needs assignment to the proper sub-team label May 20, 2024

karthiknadig assigned anthonykim1 May 20, 2024

github-actions bot added the info-needed Issue requires more information from poster label May 20, 2024

github-actions bot removed the info-needed Issue requires more information from poster label Jun 1, 2024

github-actions bot added the info-needed Issue requires more information from poster label Jun 4, 2024

github-actions bot removed the info-needed Issue requires more information from poster label Jun 4, 2024

github-actions bot added the info-needed Issue requires more information from poster label Jun 4, 2024

github-actions bot removed the info-needed Issue requires more information from poster label Jun 6, 2024

github-actions bot added the info-needed Issue requires more information from poster label Jun 7, 2024

github-actions bot removed the info-needed Issue requires more information from poster label Jun 9, 2024

dokutoshi mentioned this issue Jun 29, 2024

Discovering Python Interpreter blocks UI and Kernel on Remote Hosts #23669

Open

dokutoshi mentioned this issue Jul 29, 2024

command 'remote-internal.setActiveFolders' not found microsoft/vscode-remote-release#7178

Closed

anthonykim1 assigned karthiknadig and DonJayamanne and unassigned anthonykim1 Dec 9, 2024

github-actions bot added the info-needed Issue requires more information from poster label Dec 9, 2024

anthonykim1 added area-environments Features relating to handling interpreter environments and removed info-needed Issue requires more information from poster labels Dec 9, 2024

DonJayamanne removed their assignment Dec 9, 2024

karthiknadig added the bug Issue identified by VS Code Team member as probable bug label Dec 19, 2024

Extension v2024.4.0 and v2024.4.1 create indefinite "Discover Python Interpreters" #23452

Extension v2024.4.0 and v2024.4.1 create indefinite "Discover Python Interpreters" #23452

Comments

dokutoshi commented May 20, 2024

Behaviour

Expected vs. Actual

Steps to reproduce:

Diagnostic data

anthonykim1 commented May 20, 2024

dcx commented May 21, 2024

anthonykim1 commented May 22, 2024

anthonykim1 commented May 22, 2024

dokutoshi commented Jun 1, 2024 via email

dokutoshi commented Jun 4, 2024

anthonykim1 commented Jun 4, 2024

dokutoshi commented Jun 4, 2024

dokutoshi commented Jun 4, 2024 • edited Loading

dokutoshi commented Jun 4, 2024

anthonykim1 commented Jun 4, 2024 • edited Loading

dokutoshi commented Jun 6, 2024 • edited Loading

dokutoshi commented Jun 7, 2024

anthonykim1 commented Jun 7, 2024

dokutoshi commented Jun 9, 2024 • edited Loading

dokutoshi commented Jun 9, 2024

dokutoshi commented Jun 9, 2024 • edited Loading

Create the symbolic link

Verify the symlink

dokutoshi commented Jun 9, 2024

dokutoshi commented Jun 10, 2024

dokutoshi commented Jun 11, 2024

Create the symbolic link

Verify the symlink

dokutoshi commented Jun 12, 2024 • edited Loading

dokutoshi commented Jun 13, 2024

dokutoshi commented Jun 13, 2024

dokutoshi commented Jun 13, 2024 • edited Loading

karthiknadig commented Jun 13, 2024

dokutoshi commented Jun 14, 2024

dokutoshi commented Jun 18, 2024

karthiknadig commented Jun 18, 2024

dokutoshi commented Jun 25, 2024

karthiknadig commented Jun 25, 2024

dokutoshi commented Jun 29, 2024 • edited Loading

karthiknadig commented Jul 1, 2024

dokutoshi commented Jul 1, 2024

dokutoshi commented Jul 16, 2024

anthonykim1 commented Dec 9, 2024

dokutoshi commented Jun 4, 2024 •

edited

Loading

anthonykim1 commented Jun 4, 2024 •

edited

Loading

dokutoshi commented Jun 6, 2024 •

edited

Loading

dokutoshi commented Jun 9, 2024 •

edited

Loading

dokutoshi commented Jun 9, 2024 •

edited

Loading

dokutoshi commented Jun 12, 2024 •

edited

Loading

dokutoshi commented Jun 13, 2024 •

edited

Loading

dokutoshi commented Jun 29, 2024 •

edited

Loading