Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pythonTestAdapter fails if sockets are disabled for tests #22382

Closed
johnchristopherjones opened this issue Oct 30, 2023 · 2 comments
Closed

pythonTestAdapter fails if sockets are disabled for tests #22382

johnchristopherjones opened this issue Oct 30, 2023 · 2 comments
Labels
triage-needed Needs assignment to the proper sub-team

Comments

@johnchristopherjones
Copy link

johnchristopherjones commented Oct 30, 2023 •
edited
Loading

Type: Bug

Behaviour

Expected vs. Actual

Actual

When using pytest with pytest_socket to disable network connections in test, the test suite fails. Test Results displays the following error:

vscode_pytest.VSCodePytestError: Error attempting to connect to extension communication socket[vscode-pytest]: A test tried to use socket.socket.connect() with host "localhost" (allowed: "localhost").

Meanwhile, the test suite passes with on the terminal with:

pytest --disable-socket

The test suite can be made to pass by opting out of the pythonTestAdapter experiment in Settings:

"python.experiments.optOutFrom": ["pythonTestAdapter"],

Expected

The test runner succeeds like the CLI.

Steps to reproduce:

I've created a minimal reproduction repo:
https://github.com/johnchristopherjones/vscode-python-test-adapter-socket

The really short version is:

  1. Install pytest_socket
  2. Decorate a test with @pytest.mark.disable_socket:
@pytest.mark.disable_socket
def test_example():
    assert True
  1. Try to run the test with the test runner. (⌘; C)

Or, alternatively:

  1. Install pytest_socket

  2. Add this to your pyproject.toml:

    [tool.pytest.ini_options]
addopts = [
    "--disable-socket",    # Disable network in test with pytest-socket

  3. Try to run any tests. (⌘; A)

Diagnostic data

  • Python version (& distribution if applicable, e.g. Anaconda): 3.11.6
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
  • Value of the python.languageServer setting: Pylance
Output for Python in the Python Test Log panel 

Starting now, all test run output will be sent to the Test Result panel, while test discovery output will be sent to the "Python" output channel instead of the "Python Test Log" channel. The "Python Test Log" channel will be deprecated within the next month. See https://github.com/microsoft/vscode-python/wiki/New-Method-for-Output-Handling-in-Python-Testing for details.CLIENT: Server listening on port 50821...
Received JSON data in run script
============================= test session starts ==============================
platform darwin -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/jcj/src/vscode-python-test-adapter-socket
configfile: pyproject.toml
plugins: socket-0.6.0
collected 1 item

tests/test_always_fail.py Error attempting to connect to extension communication socket[vscode-pytest]: A test tried to use socket.socket.
If you are on a Windows machine, this error may be occurring if any of your tests clear environment variables as they are required to communicate with the extension. Please reference https://docs.pytest.org/en/stable/how-to/monkeypatch.html#monkeypatching-environment-variablesfor the correct way to clear environment variables during testing.


INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 721, in send_post_request
INTERNALERROR>     __socket.connect()
INTERNALERROR>   File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/testing_tools/socket_manager.py", line 32, in connect
INTERNALERROR>     self.socket = socket.socket(
INTERNALERROR>                   ^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pytest_socket.py", line 80, in __new__
INTERNALERROR>     raise SocketBlockedError()
INTERNALERROR> pytest_socket.SocketBlockedError: A test tried to use socket.socket.
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/main.py", line 325, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/main.py", line 350, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 114, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 133, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 226, in call_and_report
INTERNALERROR>     hook.pytest_runtest_logreport(report=report)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/terminal.py", line 574, in pytest_runtest_logreport
INTERNALERROR>     *self.config.hook.pytest_report_teststatus(report=rep, config=self.config)
INTERNALERROR>      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 62, in _multicall
INTERNALERROR>     next(wrapper_gen)  # first yield
INTERNALERROR>     ^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 233, in pytest_report_teststatus
INTERNALERROR>     execution_post(
INTERNALERROR>   File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 660, in execution_post
INTERNALERROR>     send_post_request(payload)
INTERNALERROR>   File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 732, in send_post_request
INTERNALERROR>     raise VSCodePytestError(error_msg)
INTERNALERROR> vscode_pytest.VSCodePytestError: Error attempting to connect to extension communication socket[vscode-pytest]: A test tried to use socket.socket.
Error attempting to connect to extension communication socket[vscode-pytest]: A test tried to use socket.socket.
If you are on a Windows machine, this error may be occurring if any of your tests clear environment variables as they are required to communicate with the extension. Please reference https://docs.pytest.org/en/stable/how-to/monkeypatch.html#monkeypatching-environment-variablesfor the correct way to clear environment variables during testing.

Traceback (most recent call last):
  File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 721, in send_post_request
    __socket.connect()
  File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/testing_tools/socket_manager.py", line 32, in connect
    self.socket = socket.socket(
                  ^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pytest_socket.py", line 80, in __new__
    raise SocketBlockedError()
pytest_socket.SocketBlockedError: A test tried to use socket.socket.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/run_pytest_script.py", line 67, in <module>
    pytest.main(arg_array)
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 169, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/main.py", line 318, in pytest_cmdline_main
    return wrap_session(config, _main)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/main.py", line 306, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 130, in _multicall
    teardown[0].send(outcome)
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/_pytest/terminal.py", line 857, in pytest_sessionfinish
    outcome.get_result()
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/Users/jcj/src/vscode-python-test-adapter-socket/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 367, in pytest_sessionfinish
    execution_post(
  File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 660, in execution_post
    send_post_request(payload)
  File "/Users/jcj/.vscode/extensions/ms-python.python-2023.19.13031019/pythonFiles/vscode_pytest/__init__.py", line 732, in send_post_request
    raise VSCodePytestError(error_msg)
vscode_pytest.VSCodePytestError: Error attempting to connect to extension communication socket[vscode-pytest]: A test tried to use socket.socket.
Starting now, all test run output will be sent to the Test Result panel, while test discovery output will be sent to the "Python" output channel instead of the "Python Test Log" channel. The "Python Test Log" channel will be deprecated within the next month. See https://github.com/microsoft/vscode-python/wiki/New-Method-for-Output-Handling-in-Python-Testing for details.

User Settings 


languageServer: "Pylance"

testing
• pytestEnabled: true

terminal
• launchArgs: "<placeholder>"

Extension version: 2023.19.13031019
VS Code version: Code 1.83.1 (f1b07bd25dfad64b0167beb15359ae573aecd2cc, 2023-10-10T23:57:32.750Z)
OS version: Darwin arm64 22.6.0
Modes:
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Oct 30, 2023
@karthiknadig
Copy link
Member

duplicate of #22383

@johnchristopherjones
Copy link
Author

duplicate of #22383

Apologies! I was completely unaware of the dupe. I followed the directions to open the PR through VSCode. As far as I knew, I opened a single PR, but 🤷 this is apparently what happened.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
triage-needed Needs assignment to the proper sub-team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@johnchristopherjones @karthiknadig