release: 0.14.3 #1796
Merged
release: 0.14.3 #1796
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The field wasn't defined previously. And the enum wasn't defined as `[repr(C)]`. This missing field could result in memory corruption if a Rust-allocated `PyStatus` was passed to a Python API, which could perform an out-of-bounds write. In my code, the out-of-bounds write corrupted a variable on the stack, leading to a segfault due to illegal memory access. However, this crash only occurred on Rust 1.54! So I initially mis-attribted it as a compiler bug / regression. It appears that a low-level Rust change in 1.54.0 changed the LLVM IR in such a way to cause LLVM optimization passes to produce sufficiently different assembly code, tickling the crash. See rust-lang/rust#87947 if you want to see the wild goose chase I went on in Rust / LLVM land to potentially pin this on a compiler bug. Lessen learned: Rust crashes are almost certainly due to use of `unsafe`.
The setter function will receive a NULL value on deletion requests. This wasn't properly handled before, leading to a panic. The new code raises AttributeError in this scenario instead. A test for the behavior has been added. Documentation has also been updated to reflect the behavior.
pyo3 doesn't currently define various Unicode bindings that allow the retrieval of raw data from Python strings. Said bindings are a prerequisite to possibly exposing this data in the Rust API (#1776). Even if those high-level APIs never materialize, the FFI bindings are necessary to enable consumers of the raw C API to utilize them. This commit partially defines the FFI bindings as defined in CPython's Include/cpython/unicodeobject.h file. I used the latest CPython 3.9 Git commit for defining the order of the symbols and the implementation of various inline preprocessor macros. I tried to be as faithful as possible to the original implementation, preserving intermediate `#define`s as inline functions. Missing symbols have been annotated with `skipped` and symbols currently defined in `src/ffi/unicodeobject.rs` have been annotated with `move`. The `state` field of `PyASCIIObject` is a bitfield, which Rust doesn't support. So we've provided accessor functions for retrieving these fields' values. No accessor functions are present because you shouldn't be touching these values from Rust code. Tests of the bitfield APIs and macro implementations have been added.
All symbols which are canonically defined in cpython/unicodeobject.h and had been defined in our unicodeobject.rs have been moved to our corresponding cpython/unicodeobject.rs file. This module is only present in non-limited build configurations, so we were able to drop the cfg annotations as part of moving the definitions.
This API is available in all supported Python versions and isn't deprecated.
|
It's worked well for us with Pants to create a release branch as soon as rc0 goes out for a particular release. It allows us to immediately land breaking changes to |
When building an extension with rust-numpy and ndarray on the MSRV of 1.41 with complex numbers. The num-complex crate version needs to be 0.2 which was the pinned version as of ndarray 0.13.1 which was the last release of ndarray that supported building with rust 1.41. However, the pyo3 pinned version of 0.4 is incompatible with this and will cause an error when building because of the version mismatch. To fix this This commit expands the supported versions for num-complex to match what rust-numpy uses [1] so that we can build pyo3, numpy, ndarray, and num-complex in an extension with rust 1.41. Fixes #1798 [1] https://github.com/PyO3/rust-numpy/blob/v0.14.1/Cargo.toml#L19
Co-authored-by: David Hewitt <[email protected]>
davidhewitt
force-pushed
the
release-0.14.3
branch
from
bd50724 to
0c54749
Compare
|
Thanks, that's definitely interesting food for thought. I also realised this morning that in our CHANGELOG we use git ranges between the tags to produce comparison links between each PyO3 version. It might get more fiddly to create those if we have a slightly nonlinear history. But maybe that's fine. |
|
CI seems to be borked so I'm going to close and open the PR... ? |
|
Hmm still no CI. Very confusing. |
Because of the conflicting file? See https://github.community/t/run-actions-on-pull-requests-with-merge-conflicts/17104 Edit: the github.community link seems down, here is a Google snapshot: https://webcache.googleusercontent.com/search?q=cache:lj47EKvJecEJ:https://github.community/t/run-actions-on-pull-requests-with-merge-conflicts/17104+&cd=1&hl=en-US&ct=clnk |
|
Ah interesting, somehow I'd never been aware of that. I think I'll have to make a 0.14 release branch in that case, and change this PR to merge into that. |
|
Aha looks like that's working now. |
|
Could you please cherry-pick #1812 into 0.14.3, too? Thanks! |
* update changelog * fix memory leak in PyModule::from_code * add PR link to changelog * Add Py_DECREF also when PyImport_ExecCodeModuleEx fails * Remove duplicated calls, simplify logic Co-authored-by: messense <[email protected]> Co-authored-by: messense <[email protected]>
Fixes: #1811 Signed-off-by: Christian Heimes <[email protected]>
davidhewitt
force-pushed
the
release-0.14.3
branch
from
0c54749 to
cfd4d71
Compare
|
Thanks, cherry-picked the new fixes. I'll put this live tonight unless I hear a reason to wait! |
davidhewitt
force-pushed
the
release-0.14.3
branch
from
cfd4d71 to
2d23f4c
Compare
With the recent implementation of non-limited unicode APIs, we're able to query Python's low-level state to access the raw bytes that Python is using to store string objects. This commit implements a safe Rust API for obtaining a view into Python's internals and representing the raw bytes Python is using to store strings. Not only do we allow accessing what Python has stored internally, but we also support coercing this data to a `Cow<str>`. Closes #1776.
I'm building functionality on top of this config and figured I'd take the time to document the fields to make things easier to understand.
davidhewitt
force-pushed
the
release-0.14.3
branch
from
2d23f4c to
5873d7b
Compare
davidhewitt
force-pushed
the
release-0.14.3
branch
from
5873d7b to
5e53c4d
Compare
|
Release is live! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since there have been a few FFI fixes, in particular #1791 which causes a nasty crash, I felt it would be worth cherry-picking these on top of 0.14.2 and creating a 0.14.3 release.
As these commits all exist in
mainI don't think it's worth actually merging this branch, I'll create a separate PR to update the CHANGELOG to separate out the contents released in 0.14.3 when this goes live. Perhaps there's a lesson for the future to consider having an 0.14.x branch, 0.15.x branch etc., although that in itself also creates extra work.I propose I'll put this live next weekend, I was thinking it could be nice to also cherry-pick #1794 (and maybe non-breaking parts of #1793) into this release, so that PyOxidizer is able to move ahead with its PyO3 port without waiting for us to release 0.15 (which might be a couple months off).
cc @indygreg