Tracking Issue for ByteStr/ByteString #134915

joshtriplett · 2024-12-30T10:33:35Z

Feature gate: #![feature(bstr)]

This is a tracking issue for the ByteStr/ByteString types, which represent human-readable strings that are usually, but not always, UTF-8. Unlike &str/String, these types permit non-UTF-8 contents, making them suitable for user input, non-native filenames (as Path only supports native filenames), and other applications that need to round-trip whatever data the user provides.

This was approved in ACP rust-lang/libs-team#502 .

Public API

// In core::bstr
#[repr(transparent)]
pub struct ByteStr(pub [u8]);

impl ByteStr {
    pub fn new<B: ?Sized + AsRef<[u8]>>(bytes: &B) -> &Self { ... }
}

impl Debug for ByteStr { ... }
impl Display for ByteStr { ... }
impl Deref for ByteStr { type Target = [u8]; ... }
impl DerefMut for ByteStr { ... }
// Other trait impls from bstr, including From impls

// In alloc::bstr
#[repr(transparent)]
pub struct ByteString(pub Vec<u8>);

impl Debug for ByteString { ... }
impl Display for ByteString { ... }
impl Deref for ByteString { type Target = Vec<u8>; ... }
impl DerefMut for ByteString { ... }
// Other trait impls from bstr, including From impls

Steps / History

Implementation: Implement ByteStr and ByteString types #135073
Final comment period (FCP)¹
Stabilization PR

Unresolved Questions

Should we call this BStr/BString, or ByteStr/ByteString? The former will be more familiar to users of the bstr crate in the ecosystem. The latter is more explicit, and avoids potential naming conflicts (making it easier to, for instance, add it to the prelude).
Should the Display impl use the Unicode replacement character, or do escaping like the Debug impl?

https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

The text was updated successfully, but these errors were encountered:

joshtriplett · 2025-01-03T04:25:05Z

In the course of implementing this, I'm addressing BurntSushi/bstr#190 : both the ByteStr and ByteString types will implement Index and IndexMut.

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, and `From`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, and `From`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, and `From`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, and `From`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, and `From`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, `From`, and `PartialOrd`, when those would be the second implementation for a type (counting the `T` impl) or otherwise may cause inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

Implement `ByteStr` and `ByteString` types Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (`@BurntSushi).` r? `@BurntSushi`

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, `From`, and `PartialOrd`, when those would be the second implementation for a type (counting the `T` impl) or otherwise may cause inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

Implement `ByteStr` and `ByteString` types Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (`@BurntSushi).` r? `@BurntSushi`

@BurntSushi

Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, `Borrow`, `From`, and `PartialOrd`, when those would be the second implementation for a type (counting the `T` impl) or otherwise may cause inference failures. These impls are important, but we can attempt to add them later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (@BurntSushi).

Implement `ByteStr` and `ByteString` types Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (`@BurntSushi).` r? `@BurntSushi`

There was a lobste.rs thread about this that was pretty disappointing, given that the purpose of this project is the big ideas and this is a pretty irrelevant detail. Moving this here means it's not the first thing you see clicking around on the project, and also puts the strings discussion in context of the larger question of encoding handling.

Implement `ByteStr` and `ByteString` types Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang/rust#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (`@BurntSushi).` r? `@BurntSushi`

clarfonthey · 2025-01-24T12:23:15Z

This is mostly just a nit, but it feels appropriate that the Debug impl would emit a b"..." string rather than a "..." string, to more closely resemble Rust syntax. It still wouldn't include the ByteStr(...) wrapper but it makes at least a little bit of sense and shouldn't cause too many issues.

Thinking about it, we should probably do the same for CStr as well, prefixing the string with c.

I'd be fine making a PR if this seems reasonable, or not if it just feels too pedantic.

Implement `ByteStr` and `ByteString` types Approved ACP: rust-lang/libs-team#502 Tracking issue: rust-lang/rust#134915 These types represent human-readable strings that are conventionally, but not always, UTF-8. The `Debug` impl prints non-UTF-8 bytes using escape sequences, and the `Display` impl uses the Unicode replacement character. This is a minimal implementation of these types and associated trait impls. It does not add any helper methods to other types such as `[u8]` or `Vec<u8>`. I've omitted a few implementations of `AsRef`, `AsMut`, and `Borrow`, when those would be the second implementation for a type (counting the `T` impl), to avoid potential inference failures. We can attempt to add more impls later in standalone commits, and run them through crater. In addition to the `bstr` feature, I've added a `bstr_internals` feature for APIs provided by `core` for use by `alloc` but not currently intended for stabilization. This API and its implementation are based *heavily* on the `bstr` crate by Andrew Gallant (`@BurntSushi).` r? `@BurntSushi`

joshtriplett added C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Dec 30, 2024

joshtriplett changed the title ~~Tracking Issue for BStr/BString~~ Tracking Issue for ByteStr/ByteString Dec 30, 2024

joshtriplett mentioned this issue Jan 2, 2025

Fix formatting of control characters \x1a through \x1f BurntSushi/bstr#201

Merged

joshtriplett mentioned this issue Jan 3, 2025

Implement ByteStr and ByteString types #135073

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking Issue for ByteStr/ByteString #134915

Tracking Issue for ByteStr/ByteString #134915

joshtriplett commented Dec 30, 2024 •

edited

Loading

joshtriplett commented Jan 3, 2025

clarfonthey commented Jan 24, 2025 •

edited

Loading

Tracking Issue for ByteStr/ByteString #134915

Tracking Issue for ByteStr/ByteString #134915

Comments

joshtriplett commented Dec 30, 2024 • edited Loading

Public API

Steps / History

Unresolved Questions

Footnotes

joshtriplett commented Jan 3, 2025

clarfonthey commented Jan 24, 2025 • edited Loading

joshtriplett commented Dec 30, 2024 •

edited

Loading

clarfonthey commented Jan 24, 2025 •

edited

Loading