*: use ruff linting/formatting #178
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tych0
force-pushed
the
move-to-ruff
branch
11 times, most recently
from
c41f967 to
2f0ac7e
Compare
this is fast enough that we can *always* use it on the generated output now, vs autopep8 which was quite slow and I always forgot to invoke during releases. this means that the generated bindings will be pep8 formatted, as people asked me to do many years ago but I have constantly forgotten to do :) My intent here was really to fix some whitespace noise that had been floating around for a while and I had manually pruned from other commits when peoples' editors auto fixed it, but if we're going to do that, we might as well move to standard formatting too... Signed-off-by: Tycho Andersen <[email protected]>
We just got a CI failure:
2025-01-12T16:37:45.5888665Z Fatal Python error: Segmentation fault
2025-01-12T16:37:45.5889018Z
2025-01-12T16:37:45.5889270Z Thread 0x00007f3e0ae006c0 (most recent call first):
2025-01-12T16:37:45.5890898Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 534 in read
2025-01-12T16:37:45.5893372Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 567 in from_io
2025-01-12T16:37:45.5895355Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 1160 in _thread_receiver
2025-01-12T16:37:45.5897310Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 341 in run
2025-01-12T16:37:45.5899179Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
2025-01-12T16:37:45.5900143Z
2025-01-12T16:37:45.5900589Z Current thread 0x00007f3e0c3dfb80 (most recent call first):
2025-01-12T16:37:45.5901534Z File "/home/runner/work/xcffib/xcffib/xcffib/__init__.py", line 554 in _setup_extensions
2025-01-12T16:37:45.5902549Z File "/home/runner/work/xcffib/xcffib/xcffib/__init__.py", line 546 in _init_x
2025-01-12T16:37:45.5903523Z File "/home/runner/work/xcffib/xcffib/xcffib/__init__.py", line 533 in __init__
2025-01-12T16:37:45.5904555Z File "/home/runner/work/xcffib/xcffib/xcffib/testing.py", line 133 in _connect_to_xvfb
2025-01-12T16:37:45.5905560Z File "/home/runner/work/xcffib/xcffib/xcffib/testing.py", line 83 in setUp
2025-01-12T16:37:45.5906470Z File "/home/runner/work/xcffib/xcffib/test/conftest.py", line 39 in setUp
2025-01-12T16:37:45.5907373Z File "/home/runner/work/xcffib/xcffib/xcffib/testing.py", line 116 in __enter__
2025-01-12T16:37:45.5908315Z File "/home/runner/work/xcffib/xcffib/test/conftest.py", line 25 in xcffib_test
2025-01-12T16:37:45.5909756Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/fixtures.py", line 891 in call_fixture_func
2025-01-12T16:37:45.5911860Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/fixtures.py", line 1140 in pytest_fixture_setup
2025-01-12T16:37:45.5913712Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
2025-01-12T16:37:45.5915445Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
2025-01-12T16:37:45.5916992Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
2025-01-12T16:37:45.5918405Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/fixtures.py", line 1091 in execute
2025-01-12T16:37:45.5919707Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/fixtures.py", line 617 in _get_active_fixturedef
2025-01-12T16:37:45.5921851Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/fixtures.py", line 532 in getfixturevalue
2025-01-12T16:37:45.5923730Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/fixtures.py", line 697 in _fillfixtures
2025-01-12T16:37:45.5925572Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/python.py", line 1630 in setup
2025-01-12T16:37:45.5927343Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 514 in setup
2025-01-12T16:37:45.5929435Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 160 in pytest_runtest_setup
2025-01-12T16:37:45.5931936Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
2025-01-12T16:37:45.5933720Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
2025-01-12T16:37:45.5935753Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
2025-01-12T16:37:45.5937515Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
2025-01-12T16:37:45.5939298Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
2025-01-12T16:37:45.5941233Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
2025-01-12T16:37:45.5943079Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 126 in runtestprotocol
2025-01-12T16:37:45.5944982Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
2025-01-12T16:37:45.5946847Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
2025-01-12T16:37:45.5948594Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
2025-01-12T16:37:45.5950400Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
2025-01-12T16:37:45.5952062Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/xdist/remote.py", line 195 in run_one_test
2025-01-12T16:37:45.5953854Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/xdist/remote.py", line 174 in pytest_runtestloop
2025-01-12T16:37:45.5955648Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
2025-01-12T16:37:45.5957196Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
2025-01-12T16:37:45.5958246Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
2025-01-12T16:37:45.5959947Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
2025-01-12T16:37:45.5961876Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
2025-01-12T16:37:45.5963765Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
2025-01-12T16:37:45.5965619Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
2025-01-12T16:37:45.5967366Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
2025-01-12T16:37:45.5969132Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
2025-01-12T16:37:45.5971008Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/xdist/remote.py", line 393 in <module>
2025-01-12T16:37:45.5972833Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 1291 in executetask
2025-01-12T16:37:45.5974827Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 341 in run
2025-01-12T16:37:45.5976255Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
2025-01-12T16:37:45.5977369Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 389 in integrate_as_primary_thread
2025-01-12T16:37:45.5978630Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 1273 in serve
2025-01-12T16:37:45.5979632Z File "/home/runner/work/xcffib/xcffib/xcffib-test-abi.teoYalBIO9/lib/python3.10/site-packages/execnet/gateway_base.py", line 1806 in serve
2025-01-12T16:37:45.5980450Z File "<string>", line 8 in <module>
2025-01-12T16:37:45.5980812Z File "<string>", line 1 in <module>
https://github.com/tych0/xcffib/actions/runs/12735265435/job/35493822666
which represents *another* race that has probably been around forever that
we are only now finding.
I'm not exactly sure what is going on here, but it seems likely that some
CFFI lifetime thing has gone awry, and the memory was re-used and caused
some kind of seg fault.
But we don't need all the fancy cffi lifetime gunk here: we have a 1:1
mapping for an owner of the xcb_extension_t: the xcffib.ExtensionKey
object. So let's stash the key there as a one-time allocation when the
extension object is created at module import. This way the object will be
pinned, and we hopefully won't get any of these strange seg faults.
Signed-off-by: Tycho Andersen <[email protected]>
We are seeing a failure in CI:
2025-01-12T17:04:29.4036037Z ============================= test session starts ==============================
2025-01-12T17:04:29.4041867Z platform linux -- Python 3.10.16, pytest-8.3.4, pluggy-1.5.0
2025-01-12T17:04:29.4057331Z rootdir: /home/runner/work/xcffib/xcffib
2025-01-12T17:04:29.4058439Z configfile: pyproject.toml
2025-01-12T17:04:29.4058973Z plugins: xdist-3.6.1
2025-01-12T17:04:29.4060410Z created: 4/4 workers
2025-01-12T17:04:29.4060890Z 4 workers [41 items]
2025-01-12T17:04:29.4061222Z
2025-01-12T17:04:30.9148442Z ................................F........ [100%]
2025-01-12T17:04:30.9149097Z =================================== FAILURES ===================================
2025-01-12T17:04:30.9149686Z ______________ TestXcffibTestGenerator.test_XcffibTest_generator _______________
2025-01-12T17:04:30.9150278Z [gw2] linux -- Python 3.10.16 /home/runner/work/xcffib/xcffib/xcffib-test-api.9CS94gIlPW/bin/python
2025-01-12T17:04:30.9150624Z
2025-01-12T17:04:30.9150858Z self = <test.test_python_code.TestXcffibTestGenerator object at 0x7f0503d80a60>
2025-01-12T17:04:30.9151193Z
2025-01-12T17:04:30.9151307Z def test_XcffibTest_generator(self):
2025-01-12T17:04:30.9151562Z try:
2025-01-12T17:04:30.9151807Z old_display = os.environ["DISPLAY"]
2025-01-12T17:04:30.9152073Z except KeyError:
2025-01-12T17:04:30.9152289Z old_display = ""
2025-01-12T17:04:30.9152543Z # use some non-default width/height
2025-01-12T17:04:30.9152844Z > with XcffibTest(width=1001, height=502) as test:
2025-01-12T17:04:30.9153042Z
2025-01-12T17:04:30.9153144Z test/test_python_code.py:205:
2025-01-12T17:04:30.9153420Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2025-01-12T17:04:30.9153717Z xcffib/testing.py:116: in __enter__
2025-01-12T17:04:30.9153974Z self.setUp()
2025-01-12T17:04:30.9154181Z test/conftest.py:39: in setUp
2025-01-12T17:04:30.9154417Z XvfbTest.setUp(self)
2025-01-12T17:04:30.9154642Z xcffib/testing.py:83: in setUp
2025-01-12T17:04:30.9155083Z self.conn = self._connect_to_xvfb()
2025-01-12T17:04:30.9155406Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2025-01-12T17:04:30.9155604Z
2025-01-12T17:04:30.9155763Z self = <test.conftest.XcffibTest object at 0x7f0503db7d90>
2025-01-12T17:04:30.9155990Z
2025-01-12T17:04:30.9156084Z def _connect_to_xvfb(self):
2025-01-12T17:04:30.9156357Z # sometimes it takes a while for Xvfb to start
2025-01-12T17:04:30.9156628Z for _ in range(100):
2025-01-12T17:04:30.9156841Z try:
2025-01-12T17:04:30.9157066Z conn = Connection(os.environ["DISPLAY"])
2025-01-12T17:04:30.9157330Z conn.invalid()
2025-01-12T17:04:30.9157534Z
2025-01-12T17:04:30.9157804Z # xvfb creates a screen with a default width, and then resizes it.
2025-01-12T17:04:30.9158215Z # we wait here for the resize event, so that the actual test only
2025-01-12T17:04:30.9158571Z # ever sees a screen that is the right size.
2025-01-12T17:04:30.9158863Z setup = conn.get_setup()
2025-01-12T17:04:30.9159122Z screen = setup.roots[0]
2025-01-12T17:04:30.9159745Z if screen.width_in_pixels == self.width and screen.height_in_pixels == self.height:
2025-01-12T17:04:30.9160150Z return conn
2025-01-12T17:04:30.9160390Z conn.disconnect()
2025-01-12T17:04:30.9160639Z except ConnectionException:
2025-01-12T17:04:30.9160896Z time.sleep(0.2)
2025-01-12T17:04:30.9161110Z continue
2025-01-12T17:04:30.9161344Z > assert False, "couldn't connect to xvfb"
2025-01-12T17:04:30.9161650Z E AssertionError: couldn't connect to xvfb
2025-01-12T17:04:30.9161841Z
2025-01-12T17:04:30.9161956Z xcffib/testing.py:147: AssertionError
2025-01-12T17:04:30.9162294Z ----------------------------- Captured stderr call -----------------------------
2025-01-12T17:04:30.9162744Z _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
2025-01-12T17:04:30.9163203Z _XSERVTransMakeAllCOTSServerListeners: server already running
2025-01-12T17:04:30.9163509Z (EE)
2025-01-12T17:04:30.9163680Z Fatal server error:
2025-01-12T17:04:30.9164065Z (EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE)
In principle this Should Not Happen: we are using flock's LOCK_EX to get
exclusive locks on these files. But it's very possible that with xdist we
are forking, losing the lock, etc. etc.
I played around with open(O_CREAT | O_EXCL), and that didn't fix it either,
so it doesn't seem to be a problem with the locking itself. Maybe we are
starting two copies of xvfb or xephyr or something somehow. In any case,
let's drop some parallelism for now so we can land this other stuff
reliably, since this doesn't seem to be an issue with the binding itself,
and just the test harness.
Signed-off-by: Tycho Andersen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this is fast enough that we can always use it on the generated output now, vs autopep8 which was quite slow and I always forgot to invoke during releases. this means that the generated bindings will be pep8 formatted, as people asked me to do many years ago but I have constantly forgotten to do :)
My intent here was really to fix some whitespace noise that had been floating around for a while and I had manually pruned from other commits when peoples' editors auto fixed it, but if we're going to do that, we might as well move to standard formatting too...