Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify what counts as 'running' #39

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python: ['3.7', '3.8', '3.9', '3.10']
python: ['3.7', '3.8', '3.9', '3.10', '3.11']

steps:
- name: Checkout
Expand All @@ -34,7 +34,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python: ['3.7', '3.8', '3.9', '3.10', '3.11-dev']
python: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12-dev']
check_formatting: ['0']
extra_name: ['']
include:
Expand Down Expand Up @@ -70,7 +70,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python: ['3.7', '3.8', '3.9', '3.10']
python: ['3.7', '3.8', '3.9', '3.10', '3.11']
steps:
- name: Checkout
uses: actions/checkout@v2
Expand Down
83 changes: 48 additions & 35 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,52 +54,65 @@ easy.
**Step 1:** Pick the magic string that will identify your library. To
avoid collisions, this should match your library's PEP 503 normalized name on PyPI.

**Step 2:** There's a special :class:`threading.local` object:
**Step 2:** There's a special :class:`threading.local` object attribute that
sniffio consults to determine the currently running library:

.. data:: thread_local.name
.. data:: sniffio.thread_local.name

Make sure that whenever your library is calling a coroutine ``throw()``, ``send()``, or ``close()``
that this is set to your identifier string. In most cases, this will be as simple as:
Make sure that whenever your library is potentially executing user-provided code,
this is set to your identifier string. In many cases, you can set it once when
your library starts up and restore it on shutdown:

.. code-block:: python3

from sniffio import thread_local

# Your library's step function
def step(...):
old_name, thread_local.name = thread_local.name, "my-library's-PyPI-name"
try:
result = coro.send(None)
finally:
thread_local.name = old_name
from sniffio import thread_local as sniffio_library

# Your library's run function (like trio.run() or asyncio.run())
def run(...):
old_name, sniffio_library.name = sniffio_library.name, "my-library's-PyPI-name"
try:
# actual event loop implementation left as an exercise to the reader
finally:
sniffio_library.name = old_name

In unusual situations you may need to be more fine-grained about it:

* If you're using something akin to Trio `guest mode
<https://trio.readthedocs.io/en/stable/reference-lowlevel.html#using-guest-mode-to-run-trio-on-top-of-other-event-loops>`__
to permit running your library on top of another event loop, then
you'll want to make sure that :func:`current_async_library` can
correctly identify which library (host or guest) is running at any
given moment. To achieve this, you should set and restore
:data:`thread_local.name` around each "tick" of your library's logic
(the part that is invoked as a callback from the host loop), rather
than around an entire ``run()`` function.

* If you're using something akin to `trio-asyncio
<https://trio-asyncio.readthedocs.io/en/latest/>`__ to implement one
async library on top of another, then you can set and restore
:data:`thread_local.name` around each synchronous call that might
execute user code on behalf of the 'inner' library.
For example, trio-asyncio does something like:

.. code-block:: python3

from sniffio import thread_local as sniffio_library

# Your library's compatibility loop
async def main_loop(self, ...) -> None:
...
handle: asyncio.Handle = await self.get_next_handle()
old_name, sniffio_library.name = sniffio_library.name, "asyncio"
try:
result = handle._callback(obj._args)
finally:
sniffio_library.name = old_name

**Step 3:** Send us a PR to add your library to the list of supported
libraries above.

That's it!

There are libraries that directly drive a sniffio-naive coroutine from another,
outer sniffio-aware coroutine such as `trio_asyncio`.
These libraries should make sure to set the correct value
while calling a synchronous function that will go on to drive the
sniffio-naive coroutine.


.. code-block:: python3

from sniffio import thread_local

# Your library's compatibility loop
async def main_loop(self, ...) -> None:
...
handle: asyncio.Handle = await self.get_next_handle()
old_name, thread_local.name = thread_local.name, "asyncio"
try:
result = handle._callback(obj._args)
finally:
thread_local.name = old_name


.. toctree::
:maxdepth: 1

Expand Down
13 changes: 13 additions & 0 deletions newsfragments/39.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
sniffio now attempts to return the expected library name when
:func:`sniffio.current_async_library` is called from code that is
associated with an async library but is not part of an async task.
This includes asyncio ``call_soon()`` and ``call_later()`` callbacks, and
Trio instrumentation and ``abort_fn`` handlers. (Previously, sniffio's
behavior in these situations was inconsistent.) The sniffio
documentation now explains more precisely which async library counts
as "currently running" in ambiguous cases. Libraries other than
asyncio may need updates to their sniffio integration in order to
fully conform to the new semantics; the documentation includes an
updated recipe. The new semantics also reduce the number of situations
where updates to sniffio's internals are required, which should modestly
improve the performance of libraries that interoperate with sniffio.
30 changes: 20 additions & 10 deletions sniffio/_impl.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from contextvars import ContextVar
from typing import Optional
from typing import Callable, Optional
import sys
import threading

Expand Down Expand Up @@ -37,6 +37,23 @@ def current_async_library() -> str:
depending on current mode
================ =========== ============================

If :func:`current_async_library` returns ``"someio"``, then that
generally means you can ``await someio.sleep(0)`` if you're in an
async function, and you can access ``someio``\\'s global state (to
start background tasks, determine the current time, etc) even if you're
not in an async function.

.. note:: Situations such as `guest mode
<https://trio.readthedocs.io/en/stable/reference-lowlevel.html#using-guest-mode-to-run-trio-on-top-of-other-event-loops>`__
and `trio-asyncio <https://trio-asyncio.readthedocs.io/en/latest/>`__
can result in more than one async library being "running" in the same
thread at the same time. In such ambiguous cases, `sniffio`
returns the name of the library that has most directly invoked its
caller. Within an async task, if :func:`current_async_library`
returns ``"someio"`` then that means you can ``await someio.sleep(0)``.
Outside of a task, you will get ``"asyncio"`` in asyncio callbacks,
``"trio"`` in trio instruments and abort handlers, etc.

Returns:
A string like ``"trio"``.

Expand Down Expand Up @@ -74,15 +91,8 @@ async def generic_sleep(seconds):
# Need to sniff for asyncio
if "asyncio" in sys.modules:
import asyncio
try:
current_task = asyncio.current_task # type: ignore[attr-defined]
except AttributeError:
current_task = asyncio.Task.current_task # type: ignore[attr-defined]
try:
if current_task() is not None:
return "asyncio"
except RuntimeError:
pass
if asyncio._get_running_loop() is not None:
return "asyncio"

# Sniff for curio (for now)
if 'curio' in sys.modules:
Expand Down
22 changes: 13 additions & 9 deletions sniffio/_tests/test_sniffio.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,23 +45,29 @@ def test_asyncio():

ran = []

async def this_is_asyncio():
def test_from_callback():
assert current_async_library() == "asyncio"
# Call it a second time to exercise the caching logic
ran.append(2)

async def this_is_asyncio():
asyncio.get_running_loop().call_soon(test_from_callback)
assert current_async_library() == "asyncio"
ran.append(True)
ran.append(1)

asyncio.run(this_is_asyncio())
assert ran == [True]
assert ran == [1, 2]

with pytest.raises(AsyncLibraryNotFoundError):
current_async_library()


# https://github.com/dabeaz/curio/pull/354
# https://github.com/dabeaz/curio/pull/354 has the Windows/3.9 fix.
# 3.12 error is from importing a private name that no longer exists in the
# multiprocessing module; unclear if it's going to be fixed or not.
@pytest.mark.skipif(
os.name == "nt" and sys.version_info >= (3, 9),
reason="Curio breaks on Python 3.9+ on Windows. Fix was not released yet",
(os.name == "nt" and sys.version_info >= (3, 9))
or sys.version_info >= (3, 12),
reason="Curio breaks on Python 3.9+ on Windows and 3.12+ everywhere",
)
def test_curio():
import curio
Expand All @@ -72,8 +78,6 @@ def test_curio():
ran = []

async def this_is_curio():
assert current_async_library() == "curio"
# Call it a second time to exercise the caching logic
assert current_async_library() == "curio"
ran.append(True)

Expand Down