Python 3.13: KeyError: 'name' on request_meta in ResponseContextManager.__exit__ after GC

[ktosiek]

Prerequisites

• I am using the latest version of Locust
• I am reporting a bug, not asking a question

Description

After upgrade to Python 3.13 I've started seeing errors when running locust (I've added the traceback at the end). It seems if a GC happens within with self.client.get(..., catch_response=True) then somehow ResponseContextManager.request_meta gets cleared.

Changing ResponseContextManager.__init__ to copy the response.__dict__, instead of just grabbing it, seems to help. I'm not sure how that change affects the rest of the project.

The Traceback on failure:

[2025-02-19 07:59:11,514] SDUPC07/ERROR/locust.user.task: 'name'
Traceback (most recent call last):
  File "/home/wsl/.cache/uv/archive-v0/s81GZnyMdf_txEZnLJlMJ/lib/python3.13/site-packages/locust/user/task.py", line 340, in run
    self.execute_next_task()
    ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/wsl/.cache/uv/archive-v0/s81GZnyMdf_txEZnLJlMJ/lib/python3.13/site-packages/locust/user/task.py", line 373, in execute_next_task
    self.execute_task(self._task_queue.popleft())
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wsl/.cache/uv/archive-v0/s81GZnyMdf_txEZnLJlMJ/lib/python3.13/site-packages/locust/user/task.py", line 490, in execute_task
    task(self.user)
    ~~~~^^^^^^^^^^^
  File "/home/wsl/locust-gc-bug-demo/locustfile.py", line 10, in get_json
    with self.client.get("/json", catch_response=True) as get_json:
         ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wsl/.cache/uv/archive-v0/s81GZnyMdf_txEZnLJlMJ/lib/python3.13/site-packages/locust/clients.py", line 368, in __exit__
    self.url = self.request_meta["name"]
               ~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'name'

Command line

uv run --with=locust==2.32.10 -p 3.13 locust --headless -u 1 -t 1s

Locustfile contents

from locust import run_single_user, HttpUser, task
import gc


class Tests(HttpUser):
    host = "https://httpbin.org"

    @task
    def get_json(self):
        with self.client.get("/json", catch_response=True) as get_json:
            gc.collect()


if __name__ == "__main__":
    run_single_user(Tests)

Python version

3.13.2

Locust version

2.32.10

Operating system

Ubuntu 20.04, but also happens on Windows

[cyberw]

Huh, that's strange, maybe there has been some change/optimization in Python's GC. I've confirmed that the issue is present in all 3.13 versions.

I'll have a look at it but I'm kinda busy atm, so any research you can do on your end would be most appreciated :)

[ktosiek]

I've found a reproduction without Locust, I'll try bisecting Python with it:

import gc
import sys

print(sys.version_info)

class A:
    def __init__(self, v):
        self.test = v

class ContextA:
    def __init__(self, a, other):
        self.__dict__ = a.__dict__
        self.other = other

    def __enter__(self):
        print("enter", self.__dict__)
        return self

    def __exit__(self, *args):
        print("exit", self.__dict__)
        assert "other_key" in self.other


def get_context(v):
    a = A(v)
    return ContextA(a, {"other_key": a})


with get_context("asdf") as c:
    gc.collect()

print("OK!")

Edit: bisected this problem to python/cpython@c32dc47. I wonder if I can make a smaller repro.