Skip to content

Commit

Permalink
update original
Browse files Browse the repository at this point in the history
funkill committed Dec 6, 2024
1 parent 269031f commit 4fa9bda
Showing 30 changed files with 1,549 additions and 2,022 deletions.
14 changes: 4 additions & 10 deletions rustbook-en/.github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -52,12 +52,8 @@ jobs:
- name: Run `tools` package tests
run: |
cargo test
- name: Run `mdbook-trpl-note` package tests
working-directory: packages/mdbook-trpl-note
run: |
cargo test
- name: Run `mdbook-trpl-listing` package tests
working-directory: packages/mdbook-trpl-listing
- name: Run `mdbook-trpl` package tests
working-directory: packages/mdbook-trpl
run: |
cargo test
lint:
@@ -77,10 +73,8 @@ jobs:
mkdir bin
curl -sSL https://github.com/rust-lang/mdBook/releases/download/v0.4.21/mdbook-v0.4.21-x86_64-unknown-linux-gnu.tar.gz | tar -xz --directory=bin
echo "$(pwd)/bin" >> "${GITHUB_PATH}"
- name: Install mdbook-trpl-note
run: cargo install --path packages/mdbook-trpl-note
- name: Install mdbook-trpl-listing
run: cargo install --path packages/mdbook-trpl-listing
- name: Install mdbook-trpl binaries
run: cargo install --path packages/mdbook-trpl
- name: Install aspell
run: sudo apt-get install aspell
- name: Install shellcheck
29 changes: 29 additions & 0 deletions rustbook-en/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -11,6 +11,35 @@ of the print version. The snapshot files reflect what has been sent or not, so
they only get updated when edits are sent to No Starch. **Do not submit pull
requests changing files in the `nostarch` directory, they will be closed.**

We use [`rustfmt`][rustfmt] to apply standard formatting to Rust code in the
repo and [`dprint`][dprint] to apply standing formatting to the Markdown source
and the non-Rust code in the project.

[rustfmt]: https://github.com/rust-lang/rustfmt
[dprint]: https://dprint.dev

You will normally have `rustfmt` installed if you have a Rust toolchain
installed; if for some reason you do not have a copy of `rustfmt`, you can add
it by running the following command:

```sh
rustup component add rustfmt
```

To install `dprint`, you can run the following command:

```sh
cargo install dprint
```

Or follow the [instructions][install-dprint] on the `dprint` website.

[install-dprint]: https://dprint.dev/install/

To format Rust code, you can run `rustfmt <path to file>`, and to format other
files, you can pass `dprint <path to file>`. Many text editors also have native
support or extensions for both `rustfmt` and `dprint`.

## Checking for Fixes

The book rides the Rust release trains. Therefore, if you see a problem on
3 changes: 1 addition & 2 deletions rustbook-en/README.md
Original file line number Diff line number Diff line change
@@ -39,8 +39,7 @@ look right, but you _will_ still be able to build the book. To use the plugins,
you should run:

```bash
$ cargo install --locked --path packages/mdbook-trpl-listing
$ cargo install --locked --path packages/mdbook-trpl-note
$ cargo install --locked --path packages/mdbook_trpl
```

## Building
Original file line number Diff line number Diff line change
@@ -9,6 +9,6 @@ fn main() {
// ANCHOR: here
fn calculate_length(s: &String) -> usize { // s is a reference to a String
s.len()
} // Here, s goes out of scope. But because it does not have ownership of what
// it refers to, it is not dropped.
} // Here, s goes out of scope. But because s does not have ownership of what
// it refers to, the value is not dropped.
// ANCHOR_END: here
5 changes: 5 additions & 0 deletions rustbook-en/nostarch/book.toml
Original file line number Diff line number Diff line change
@@ -14,5 +14,10 @@ build-dir = "../tmp"
[preprocessor.trpl-listing]
output-mode = "simple"

# Only used in this version, *not* in the root `book.toml`, because its job is
# to remove `<figure>` and `<figcaption>` markup from the version we send them.
[preprocessor.trpl-figure]
output-mode = "simple"

[rust]
edition = "2021"
1,087 changes: 0 additions & 1,087 deletions rustbook-en/packages/mdbook-trpl-note/Cargo.lock

This file was deleted.

21 changes: 0 additions & 21 deletions rustbook-en/packages/mdbook-trpl-note/Cargo.toml

This file was deleted.

346 changes: 0 additions & 346 deletions rustbook-en/packages/mdbook-trpl-note/src/lib.rs

This file was deleted.

22 changes: 0 additions & 22 deletions rustbook-en/packages/mdbook-trpl-note/tests/integration/main.rs

This file was deleted.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,16 +1,27 @@
[package]
name = "mdbook-trpl-listing"
name = "mdbook-trpl"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[[bin]]
name = "mdbook-trpl-note"
path = "src/bin/note.rs"

[[bin]]
name = "mdbook-trpl-listing"
path = "src/bin/listing.rs"

[[bin]]
name = "mdbook-trpl-figure"
path = "src/bin/figure.rs"

[dependencies]
anyhow = "1"
clap = { version = "4", features = ["derive"] }
html_parser = "0.7.0"
mdbook = { version = "0.4", default-features = false } # only need the library
pulldown-cmark = { version = "0.10", features = ["simd"] }
pulldown-cmark-to-cmark = "13"
pulldown-cmark = { version = "0.12", features = ["simd"] }
pulldown-cmark-to-cmark = "19"
serde_json = "1"
thiserror = "1.0.60"
toml = "0.8.12"
13 changes: 13 additions & 0 deletions rustbook-en/packages/mdbook-trpl/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# mdbook_trpl

A shared package for [mdbook][mdbook] [preprocessors][pre] used in [_The Rust
Programming Language_][trpl].

Supplies the following preprocessor binaries:

- [mdbook-trpl-note](./src/bin/note)
- [mdbook-trpl-listing](./src/bin/listing)

[mdbook]: https://crates.io/crates/mdbook
[pre]: https://rust-lang.github.io/mdBook/format/configuration/preprocessors.html
[trpl]: https://doc.rust-lang.org/book/
42 changes: 42 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/bin/figure.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
use std::io;

use clap::{self, Parser, Subcommand};

use mdbook::preprocess::{CmdPreprocessor, Preprocessor};
use mdbook_trpl::Figure;

fn main() -> Result<(), String> {
match Cli::parse().command {
Some(Command::Supports { renderer }) => {
if Figure.supports_renderer(&renderer) {
Ok(())
} else {
Err(format!("Renderer '{renderer}' is unsupported"))
}
}
None => {
let (ctx, book) = CmdPreprocessor::parse_input(io::stdin())
.map_err(|e| format!("{e}"))?;
let processed =
Figure.run(&ctx, book).map_err(|e| format!("{e}"))?;
serde_json::to_writer(io::stdout(), &processed)
.map_err(|e| format!("{e}"))
}
}
}

/// A simple preprocessor for handling figures with images in _The Rust
/// Programming Language_ book.
#[derive(Parser, Debug)]
struct Cli {
#[command(subcommand)]
command: Option<Command>,
}

#[derive(Subcommand, Debug)]
enum Command {
/// Is the renderer supported?
///
/// Supported renderers are `'html'`, `'markdown'`, and `'test'`.
Supports { renderer: String },
}
Original file line number Diff line number Diff line change
@@ -3,20 +3,21 @@ use std::io;
use clap::{self, Parser, Subcommand};
use mdbook::preprocess::{CmdPreprocessor, Preprocessor};

use mdbook_trpl_listing::TrplListing;
use mdbook_trpl::Listing;

fn main() -> Result<(), String> {
let cli = Cli::parse();
if let Some(Command::Supports { renderer }) = cli.command {
return if TrplListing.supports_renderer(&renderer) {
return if Listing.supports_renderer(&renderer) {
Ok(())
} else {
Err(format!("Renderer '{renderer}' is unsupported"))
};
}

let (ctx, book) = CmdPreprocessor::parse_input(io::stdin()).map_err(|e| format!("{e}"))?;
let processed = TrplListing.run(&ctx, book).map_err(|e| format!("{e}"))?;
let (ctx, book) = CmdPreprocessor::parse_input(io::stdin())
.map_err(|e| format!("{e}"))?;
let processed = Listing.run(&ctx, book).map_err(|e| format!("{e}"))?;
serde_json::to_writer(io::stdout(), &processed).map_err(|e| format!("{e}"))
}

Original file line number Diff line number Diff line change
@@ -3,11 +3,11 @@ use std::io;
use clap::{self, Parser, Subcommand};
use mdbook::preprocess::{CmdPreprocessor, Preprocessor};

use mdbook_trpl_note::TrplNote;
use mdbook_trpl::Note;

fn main() -> Result<(), String> {
let cli = Cli::parse();
let simple_note = TrplNote;
let simple_note = Note;
if let Some(Command::Supports { renderer }) = cli.command {
return if simple_note.supports_renderer(&renderer) {
Ok(())
@@ -16,8 +16,8 @@ fn main() -> Result<(), String> {
};
}

let (ctx, book) =
CmdPreprocessor::parse_input(io::stdin()).map_err(|e| format!("blah: {e}"))?;
let (ctx, book) = CmdPreprocessor::parse_input(io::stdin())
.map_err(|e| format!("blah: {e}"))?;
let processed = simple_note.run(&ctx, book).map_err(|e| format!("{e}"))?;
serde_json::to_writer(io::stdout(), &processed).map_err(|e| format!("{e}"))
}
71 changes: 71 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/config/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
//! Get any `preprocessor.trpl-*` config.
use mdbook::preprocess::PreprocessorContext;

#[derive(Debug, Clone, Copy)]
pub enum Mode {
Default,
Simple,
}

impl Mode {
pub fn from_context(
ctx: &PreprocessorContext,
preprocessor_name: &str,
) -> Result<Mode, Error> {
let config = ctx
.config
.get_preprocessor(preprocessor_name)
.ok_or_else(|| Error::NoConfig(preprocessor_name.into()))?;

let key = String::from("output-mode");
let mode = config
.get(&key)
.map(|value| match value.as_str() {
Some(s) => Mode::try_from(s).map_err(|_| Error::BadValue {
key,
value: value.to_string(),
}),
None => Err(Error::BadValue {
key,
value: value.to_string(),
}),
})
.transpose()?
.unwrap_or(Mode::Default);

Ok(mode)
}
}

/// Trivial marker struct to indicate an internal error.
///
/// The caller has enough info to do what it needs without passing data around.
pub struct ParseErr;

impl TryFrom<&str> for Mode {
type Error = ParseErr;

fn try_from(value: &str) -> Result<Self, Self::Error> {
match value {
"default" => Ok(Mode::Default),
"simple" => Ok(Mode::Simple),
_ => Err(ParseErr),
}
}
}

#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error(transparent)]
Mdbook(#[from] mdbook::errors::Error),

#[error("No config for '{0}'")]
NoConfig(String),

#[error("Bad config value '{value}' for key '{key}'")]
BadValue { key: String, value: String },
}

#[cfg(test)]
mod tests;
Original file line number Diff line number Diff line change
@@ -6,96 +6,118 @@
//! more complex in the future, it would be good to revisit and integrate
//! the same kinds of tests as the unit tests above here.
use super::*;
use mdbook::{
book::Book,
errors::Result,
preprocess::{Preprocessor, PreprocessorContext},
BookItem,
};

use crate::config::Mode;

/// Dummy preprocessor for testing purposes to exercise config.
struct TestPreprocessor;

impl Preprocessor for TestPreprocessor {
fn name(&self) -> &str {
"test-preprocessor"
}

fn run(&self, ctx: &PreprocessorContext, mut book: Book) -> Result<Book> {
let mode = Mode::from_context(ctx, self.name())?;
book.push_item(BookItem::PartTitle(format!("{mode:?}")));
Ok(book)
}
}

// TODO: what *should* the behavior here be? I *think* it should error,
// in that there is a problem if it is invoked without that info.
#[test]
fn no_config() {
let input_json = r##"[
{
"root": "/path/to/book",
"config": {
"book": {
"authors": ["AUTHOR"],
"language": "en",
"multilingual": false,
"src": "src",
"title": "TITLE"
},
"preprocessor": {}
},
"renderer": "html",
"mdbook_version": "0.4.21"
{
"root": "/path/to/book",
"config": {
"book": {
"authors": ["AUTHOR"],
"language": "en",
"multilingual": false,
"src": "src",
"title": "TITLE"
},
"preprocessor": {}
},
"renderer": "html",
"mdbook_version": "0.4.21"
},
{
"sections": [
{
"sections": [
{
"Chapter": {
"name": "Chapter 1",
"content": "# Chapter 1\n",
"number": [1],
"sub_items": [],
"path": "chapter_1.md",
"source_path": "chapter_1.md",
"parent_names": []
}
}
],
"__non_exhaustive": null
"Chapter": {
"name": "Chapter 1",
"content": "# Chapter 1\n",
"number": [1],
"sub_items": [],
"path": "chapter_1.md",
"source_path": "chapter_1.md",
"parent_names": []
}
}
]"##;
],
"__non_exhaustive": null
}
]"##;
let input_json = input_json.as_bytes();
let (ctx, book) =
mdbook::preprocess::CmdPreprocessor::parse_input(input_json).unwrap();
let result = TrplListing.run(&ctx, book);
let result = TestPreprocessor.run(&ctx, book);
assert!(result.is_err());
let err = result.unwrap_err();
assert_eq!(format!("{err}"), "No config for trpl-listing");
assert_eq!(format!("{err}"), "No config for 'test-preprocessor'");
}

#[test]
fn empty_config() {
let input_json = r##"[
{
"root": "/path/to/book",
"config": {
"book": {
"authors": ["AUTHOR"],
"language": "en",
"multilingual": false,
"src": "src",
"title": "TITLE"
},
"preprocessor": {
"trpl-listing": {}
}
},
"renderer": "html",
"mdbook_version": "0.4.21"
{
"root": "/path/to/book",
"config": {
"book": {
"authors": ["AUTHOR"],
"language": "en",
"multilingual": false,
"src": "src",
"title": "TITLE"
},
"preprocessor": {
"test-preprocessor": {}
}
},
"renderer": "html",
"mdbook_version": "0.4.21"
},
{
"sections": [
{
"sections": [
{
"Chapter": {
"name": "Chapter 1",
"content": "# Chapter 1\n",
"number": [1],
"sub_items": [],
"path": "chapter_1.md",
"source_path": "chapter_1.md",
"parent_names": []
}
}
],
"__non_exhaustive": null
"Chapter": {
"name": "Chapter 1",
"content": "# Chapter 1\n",
"number": [1],
"sub_items": [],
"path": "chapter_1.md",
"source_path": "chapter_1.md",
"parent_names": []
}
}
]"##;
],
"__non_exhaustive": null
}
]"##;
let input_json = input_json.as_bytes();
let (ctx, book) =
mdbook::preprocess::CmdPreprocessor::parse_input(input_json).unwrap();
let result = TrplListing.run(&ctx, book);
assert!(result.is_ok());
let book = TestPreprocessor.run(&ctx, book).unwrap();
assert!(book.iter().any(
|item| matches!(item, BookItem::PartTitle(title) if title == &format!("{:?}", Mode::Default))
))
}

#[test]
@@ -112,7 +134,7 @@ fn specify_default() {
"title": "TITLE"
},
"preprocessor": {
"trpl-listing": {
"test-preprocessor": {
"output-mode": "default"
}
}
@@ -140,8 +162,10 @@ fn specify_default() {
let input_json = input_json.as_bytes();
let (ctx, book) =
mdbook::preprocess::CmdPreprocessor::parse_input(input_json).unwrap();
let result = TrplListing.run(&ctx, book);
assert!(result.is_ok());
let book = TestPreprocessor.run(&ctx, book).unwrap();
assert!(book.iter().any(
|item| matches!(item, BookItem::PartTitle(title) if title == &format!("{:?}", Mode::Default))
));
}

#[test]
@@ -158,7 +182,7 @@ fn specify_simple() {
"title": "TITLE"
},
"preprocessor": {
"trpl-listing": {
"test-preprocessor": {
"output-mode": "simple"
}
}
@@ -186,8 +210,10 @@ fn specify_simple() {
let input_json = input_json.as_bytes();
let (ctx, book) =
mdbook::preprocess::CmdPreprocessor::parse_input(input_json).unwrap();
let result = TrplListing.run(&ctx, book);
assert!(result.is_ok());
let book = TestPreprocessor.run(&ctx, book).unwrap();
assert!(book.iter().any(
|item| matches!(item, BookItem::PartTitle(title) if title == &format!("{:?}", Mode::Simple))
))
}

#[test]
@@ -204,7 +230,7 @@ fn specify_invalid() {
"title": "TITLE"
},
"preprocessor": {
"trpl-listing": {
"test-preprocessor": {
"output-mode": "nonsense"
}
}
@@ -232,11 +258,9 @@ fn specify_invalid() {
let input_json = input_json.as_bytes();
let (ctx, book) =
mdbook::preprocess::CmdPreprocessor::parse_input(input_json).unwrap();
let result = TrplListing.run(&ctx, book);
assert!(result.is_err());
let err = result.unwrap_err();
let result = TestPreprocessor.run(&ctx, book).unwrap_err();
assert_eq!(
format!("{err}"),
format!("{result}"),
"Bad config value '\"nonsense\"' for key 'output-mode'"
);
}
252 changes: 252 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/figure/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
use anyhow::{anyhow, Result};
use html_parser::{Dom, Node};
use mdbook::{book::Book, preprocess::Preprocessor, BookItem};

use pulldown_cmark::Event;
use pulldown_cmark_to_cmark::cmark;

use crate::config::Mode;

/// A simple preprocessor to rewrite `<figure>`s with `<img>`s.
///
/// This is a no-op by default; it only operates on the book chapters when the
/// `[preprocessor.trpl-figure]` has `output-mode = "simple"`.
///
/// Takes in Markdown containing like this:
///
/// ```markdown
/// <figure>
///
/// <img src="http://www.example.com/some-cool-image.jpg">
///
/// <figcaption>Figure 1-2: A description of the image</figcaption>
///
/// </figure>
/// ```
///
/// Spits out Markdown like this:
///
/// ```markdown
///
/// <img src="http://www.example.com/some-cool-image.jpg">
///
/// Figure 1-2: A description of the image
///
/// ```
pub struct TrplFigure;

impl TrplFigure {
pub fn supports_renderer(&self, renderer: &str) -> bool {
renderer == "html" || renderer == "markdown" || renderer == "test"
}
}

impl Preprocessor for TrplFigure {
fn name(&self) -> &str {
"trpl-figure"
}

fn run(
&self,
ctx: &mdbook::preprocess::PreprocessorContext,
mut book: Book,
) -> Result<Book> {
// The `<figure>`-based output is only replaced in the `Simple` mode.
let Mode::Simple = Mode::from_context(ctx, self.name())? else {
return Ok(book);
};

let mut errors = vec![];
book.for_each_mut(|item| {
if let BookItem::Chapter(ref mut chapter) = item {
match rewrite_figure(&chapter.content) {
Ok(rewritten) => chapter.content = rewritten,
Err(reason) => errors.push(reason),
}
}
});

if errors.is_empty() {
Ok(book)
} else {
Err(CompositeError(errors).into())
}
}
}

#[derive(Debug, thiserror::Error)]
struct CompositeError(Vec<anyhow::Error>);

impl std::fmt::Display for CompositeError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"Error(s) rewriting input: {}",
self.0.iter().map(|e| format!("{e:?}")).collect::<String>()
)
}
}

const OPEN_FIGURE: &'static str = "<figure>";
const CLOSE_FIGURE: &'static str = "</figure>";

const OPEN_CAPTION: &'static str = "<figcaption>";
const CLOSE_CAPTION: &'static str = "</figcaption>";

fn rewrite_figure(text: &str) -> Result<String> {
let final_state = crate::parser(text).try_fold(
State {
current: None,
events: Vec::new(),
},
|mut state, event| {
match (event, &mut state.current) {
// -- Open figure
(Event::Html(tag), None) if tag.starts_with(OPEN_FIGURE) => {
let mut figure = Figure::new();
figure.events.push(Event::Text("\n".into()));
state.current.replace(figure);
}

(Event::Html(tag), Some(_)) if tag.starts_with(OPEN_FIGURE) => {
return Err(anyhow!(
"Opening `<figure>` when already in a `<figure>`"
))
}

// -- Close figure
(Event::Html(tag), Some(figure))
if tag.starts_with(CLOSE_FIGURE) =>
{
if figure.in_caption {
return Err(anyhow!("Unclosed `<figcaption>`"));
}

state.events.append(&mut figure.events);
state.events.push(Event::Text("\n".into()));
let _ = state.current.take();
}

(Event::Html(tag), None) if tag.trim() == CLOSE_FIGURE => {
return Err(anyhow!(bad_close(CLOSE_FIGURE, OPEN_CAPTION)));
}

// -- Start captions
// We do not allow nested captions, but if we have not yet
// started a caption, it is legal to start one, and we
// intentionally ignore that event entirely other than tracking
// that we have started a caption. We will push the body of the
// caption into the figure’s events when we hit them.
//
// Note: this does not support `<figcaption class="...">`.
(Event::Html(tag), Some(fig))
if tag.starts_with(OPEN_CAPTION) =>
{
if fig.in_caption {
return Err(anyhow!(bad_open(OPEN_CAPTION)));
} else {
if tag.trim().ends_with(CLOSE_CAPTION) {
let text = Dom::parse(tag.as_ref())?
.children
.into_iter()
.filter_map(text_of)
.collect::<String>();

if text.is_empty() {
return Err(anyhow!(
"Missing caption in `<figcaption>`"
));
}

fig.events.push(Event::Text(text.into()));
} else {
fig.events.push(Event::Text("\n".into()));
fig.in_caption = true;
}
}
}

(Event::Html(tag), None) if tag.starts_with(OPEN_CAPTION) => {
return Err(anyhow!(bad_open(OPEN_CAPTION)))
}

// -- Close captions
(Event::Html(tag), Some(fig))
if tag.trim() == CLOSE_CAPTION =>
{
if fig.in_caption {
fig.events.push(Event::Text("\n".into()));
fig.in_caption = false;
} else {
return Err(anyhow!(bad_close(
CLOSE_CAPTION,
OPEN_CAPTION
)));
}
}

(Event::Html(tag), None) if tag.trim() == CLOSE_CAPTION => {
return Err(anyhow!(bad_close(CLOSE_CAPTION, OPEN_FIGURE)));
}

// Otherwise, if in the body of a figure, push whatever other
// events without modification into the figure state.
(ev, Some(ref mut figure)) => figure.events.push(ev),

// And if not in a figure, no modifications whatsoever.
(ev, None) => state.events.push(ev),
}
Ok(state)
},
)?;

if final_state.current.is_some() {
return Err(anyhow!("Unclosed `<figure>`"));
}

let mut rewritten = String::new();
cmark(final_state.events.into_iter(), &mut rewritten)?;
Ok(rewritten)
}

fn text_of(node: Node) -> Option<String> {
match node {
Node::Text(text) => Some(text),
Node::Element(element) => {
Some(element.children.into_iter().filter_map(text_of).collect())
}
Node::Comment(_) => None,
}
}

fn bad_open(tag: &str) -> String {
format!("Opening `<{tag}>` while not in a `<figure>`.")
}

fn bad_close(close: &str, required_open: &str) -> String {
format!("Closing `<{close}>` while not in a `<{required_open}>`.")
}

#[derive(Debug)]
struct State<'e> {
current: Option<Figure<'e>>,
events: Vec<Event<'e>>,
}

#[derive(Debug)]
struct Figure<'e> {
events: Vec<Event<'e>>,
in_caption: bool,
}

impl<'e> Figure<'e> {
fn new() -> Figure<'e> {
Figure {
events: vec![],
in_caption: false,
}
}
}

#[cfg(test)]
mod tests;
60 changes: 60 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/figure/tests.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
use super::*;

#[test]
fn text_without_figures_is_ignored() {
let actual = rewrite_figure("This is some basic text.").unwrap();
assert_eq!(actual, "This is some basic text.");
}

#[test]
fn text_with_figure_replaces_it_with_simple_text() {
let actual = rewrite_figure(
r#"<figure>
<img src="http://www.example.com/some-image.jpg">
<figcaption>Figure 12-34: Look at this cool picture!</figcaption>
</figure>"#,
)
.unwrap();

let expected = r#"
<img src="http://www.example.com/some-image.jpg">
Figure 12-34: Look at this cool picture!
"#;

assert_eq!(actual, expected);
}

#[test]
fn unclosed_figure() {
let result = rewrite_figure("<figure>");
let actual = format!("{:?}", result.unwrap_err());
assert_eq!(actual, "Unclosed `<figure>`");
}

#[test]
fn empty_caption() {
let result = rewrite_figure(
"<figure>
<figcaption></figcaption>
</figure>",
);
let actual = format!("{:?}", result.unwrap_err());
assert_eq!(actual, "Missing caption in `<figcaption>`");
}

#[test]
fn unclosed_caption() {
let result = rewrite_figure(
"<figure>
<figcaption>
</figure>",
);
let actual = format!("{:?}", result.unwrap_err());
assert_eq!(actual, "Unclosed `<figcaption>`");
}
35 changes: 35 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
mod config;
mod figure;
mod listing;
mod note;

pub use config::Mode;
pub use figure::TrplFigure as Figure;
pub use listing::TrplListing as Listing;
pub use note::TrplNote as Note;
use pulldown_cmark::{Options, Parser};

/// Convenience function to get a parser matching `mdbook::new_cmark_parser`.
///
/// This is implemented separately so we are decoupled from mdbook's dependency
/// versions and can update at will (albeit with care to stay aligned with what
/// mdbook does!) to later versions of `pulldown-cmark` and related tools.
///
/// Notes:
///
/// - `mdbook::new_cmark_parser` has an additional parameter which allows smart
/// punctuation to be enabled or disabled; we always enable it.
/// - We do not use footnotes in the text at present, but this goes out of its
/// way to match this up to the old footnotes behavior just to make sure the
/// parsing etc. is all the same.
pub fn parser(text: &str) -> Parser<'_> {
let mut opts = Options::empty();
opts.insert(Options::ENABLE_TABLES);
opts.insert(Options::ENABLE_FOOTNOTES);
opts.insert(Options::ENABLE_OLD_FOOTNOTES);
opts.insert(Options::ENABLE_STRIKETHROUGH);
opts.insert(Options::ENABLE_TASKLISTS);
opts.insert(Options::ENABLE_HEADING_ATTRIBUTES);
opts.insert(Options::ENABLE_SMART_PUNCTUATION);
Parser::new_ext(text, opts)
}

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -33,26 +33,33 @@ fn main() {}
#[test]
fn simple_mode_works() {
let result = rewrite_listing(
r#"<Listing number="1-2" caption="A write-up which *might* include inline Markdown like `code` etc." file-name="src/main.rs">
r#"Leading text.
<Listing number="1-2" caption="A write-up which *might* include inline Markdown like `code` etc." file-name="src/main.rs">
```rust
fn main() {}
```
</Listing>"#,
</Listing>
Trailing text."#,
Mode::Simple,
);

assert_eq!(
&result.unwrap(),
r#"
r#"Leading text.
Filename: src/main.rs
````rust
```rust
fn main() {}
````
```
Listing 1-2: A write-up which <em>might</em> include inline Markdown like <code>code</code> etc."#
Listing 1-2: A write-up which *might* include inline Markdown like `code` etc.
Trailing text."#
);
}

@@ -287,9 +294,3 @@ fn main() {}
)
}
}

#[test]
fn missing_value() {}

#[cfg(test)]
mod config;
145 changes: 145 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/note/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
use mdbook::{
book::Book,
errors::Result,
preprocess::{Preprocessor, PreprocessorContext},
BookItem,
};
use pulldown_cmark::{
Event::{self, *},
Tag, TagEnd,
};
use pulldown_cmark_to_cmark::cmark;

/// A simple preprocessor for semantic notes in _The Rust Programming Language_.
///
/// Takes in Markdown like this:
///
/// ```markdown
/// > Note: This is a note.
/// ```
///
/// Spits out Markdown like this:
///
/// ```markdown
/// <section class="note" aria-role="note">
///
/// This is a note.
///
/// </section>
/// ```
pub struct TrplNote;

impl Preprocessor for TrplNote {
fn name(&self) -> &str {
"simple-note-preprocessor"
}

fn run(&self, _ctx: &PreprocessorContext, mut book: Book) -> Result<Book> {
book.for_each_mut(|item| {
if let BookItem::Chapter(ref mut chapter) = item {
chapter.content = rewrite(&chapter.content);
}
});
Ok(book)
}

fn supports_renderer(&self, renderer: &str) -> bool {
renderer == "html" || renderer == "markdown" || renderer == "test"
}
}

pub fn rewrite(text: &str) -> String {
let parser = crate::parser(text);

let mut events = Vec::new();
let mut state = Default;

for event in parser {
match (&mut state, event) {
(Default, Start(Tag::BlockQuote(_))) => {
state = StartingBlockquote(vec![Start(Tag::BlockQuote(None))]);
}

(StartingBlockquote(blockquote_events), Text(content)) => {
if content.starts_with("Note: ") {
// This needs the "extra" `SoftBreak`s so that when the final rendering pass
// happens, it does not end up treating the internal content as inline *or*
// treating the HTML tags as inline tags:
//
// - Content inside HTML blocks is only rendered as Markdown when it is
// separated from the block HTML elements: otherwise it gets treated as inline
// HTML and *not* rendered.
// - Along the same lines, an HTML tag that happens to be directly adjacent to
// the end of a previous Markdown block will end up being rendered as part of
// that block.
events.extend([
SoftBreak,
SoftBreak,
Html(
r#"<section class="note" aria-role="note">"#.into(),
),
SoftBreak,
SoftBreak,
Start(Tag::Paragraph),
Text(content),
]);
state = InNote;
} else {
events.append(blockquote_events);
events.push(Text(content));
state = Default;
}
}

(
StartingBlockquote(_blockquote_events),
heading @ Start(Tag::Heading { .. }),
) => {
events.extend([
SoftBreak,
SoftBreak,
Html(r#"<section class="note" aria-role="note">"#.into()),
SoftBreak,
SoftBreak,
heading,
]);
state = InNote;
}

(StartingBlockquote(ref mut events), Start(tag)) => {
events.push(Start(tag));
}

(InNote, End(TagEnd::BlockQuote(_))) => {
// As with the start of the block HTML, the closing HTML must be
// separated from the Markdown text by two newlines.
events.extend([
SoftBreak,
SoftBreak,
Html("</section>".into()),
]);
state = Default;
}

(_, event) => {
events.push(event);
}
}
}

let mut buf = String::new();
cmark(events.into_iter(), &mut buf).unwrap();
buf
}

use State::*;

#[derive(Debug)]
enum State<'e> {
Default,
StartingBlockquote(Vec<Event<'e>>),
InNote,
}

#[cfg(test)]
mod tests;
195 changes: 195 additions & 0 deletions rustbook-en/packages/mdbook-trpl/src/note/tests.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
use super::*;

#[test]
fn no_note() {
let text = "Hello, world.\n\nThis is some text.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<p>Hello, world.</p>\n<p>This is some text.</p>\n"
);
}

#[test]
fn with_note() {
let text = "> Note: This is some text.\n> It keeps going.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<p>Note: This is some text.\nIt keeps going.</p>\n</section>"
);
}

#[test]
fn regular_blockquote() {
let text = "> This is some text.\n> It keeps going.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<blockquote>\n<p>This is some text.\nIt keeps going.</p>\n</blockquote>\n"
);
}

#[test]
fn combined() {
let text = "> Note: This is some text.\n> It keeps going.\n\nThis is regular text.\n\n> This is a blockquote.\n";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<p>Note: This is some text.\nIt keeps going.</p>\n</section>\n<p>This is regular text.</p>\n<blockquote>\n<p>This is a blockquote.</p>\n</blockquote>\n"
);
}

#[test]
fn blockquote_then_note() {
let text = "> This is quoted.\n\n> Note: This is noted.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<blockquote>\n<p>This is quoted.</p>\n</blockquote>\n<section class=\"note\" aria-role=\"note\">\n<p>Note: This is noted.</p>\n</section>"
);
}

#[test]
fn note_then_blockquote() {
let text = "> Note: This is noted.\n\n> This is quoted.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<p>Note: This is noted.</p>\n</section>\n<blockquote>\n<p>This is quoted.</p>\n</blockquote>\n"
);
}

#[test]
fn with_h1_note() {
let text = "> # Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h1>Header</h1>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn with_h2_note() {
let text = "> ## Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h2>Header</h2>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn with_h3_note() {
let text = "> ### Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h3>Header</h3>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn with_h4_note() {
let text = "> #### Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h4>Header</h4>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn with_h5_note() {
let text = "> ##### Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h5>Header</h5>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn with_h6_note() {
let text = "> ###### Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h6>Header</h6>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn h1_then_blockquote() {
let text =
"> # Header\n > And then some note content.\n\n> This is quoted.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<section class=\"note\" aria-role=\"note\">\n<h1>Header</h1>\n<p>And then some note content.</p>\n</section>\n<blockquote>\n<p>This is quoted.</p>\n</blockquote>\n"
);
}

#[test]
fn blockquote_then_h1_note() {
let text =
"> This is quoted.\n\n> # Header\n > And then some note content.";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<blockquote>\n<p>This is quoted.</p>\n</blockquote>\n<section class=\"note\" aria-role=\"note\">\n<h1>Header</h1>\n<p>And then some note content.</p>\n</section>"
);
}

#[test]
fn blockquote_with_strong() {
let text = "> **Bold text in a paragraph.**";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<blockquote>\n<p><strong>Bold text in a paragraph.</strong></p>\n</blockquote>\n"
);
}

#[test]
fn normal_table() {
let text = "| Header 1 | Header 2 |\n| -------- | -------- |\n| Text 123 | More 456 |";
let processed = rewrite(text);

assert_eq!(
processed,
"|Header 1|Header 2|\n|--------|--------|\n|Text 123|More 456|",
"It strips some whitespace but otherwise leaves the table intact."
);
}

#[test]
fn table_in_note() {
let text = "> Note: table stuff.\n\n| Header 1 | Header 2 |\n| -------- | -------- |\n| Text 123 | More 456 |";
let processed = rewrite(text);

assert_eq!(
processed,
"\n\n<section class=\"note\" aria-role=\"note\">\n\nNote: table stuff.\n\n</section>\n\n|Header 1|Header 2|\n|--------|--------|\n|Text 123|More 456|",
"It adds the note markup but leaves the table untouched, to be rendered as Markdown."
);
}

#[test]
fn table_in_quote() {
let text = "> A table.\n\n| Header 1 | Header 2 |\n| -------- | -------- |\n| Text 123 | More 456 |";
let processed = rewrite(text);
assert_eq!(
render_markdown(&processed),
"<blockquote>\n<p>A table.</p>\n</blockquote>\n<table><thead><tr><th>Header 1</th><th>Header 2</th></tr></thead><tbody>\n<tr><td>Text 123</td><td>More 456</td></tr>\n</tbody></table>\n",
"It renders blockquotes with nested tables as expected."
);
}

fn render_markdown(text: &str) -> String {
let parser = crate::parser(text);
let mut buf = String::new();
pulldown_cmark::html::push_html(&mut buf, parser);
buf
}
20 changes: 20 additions & 0 deletions rustbook-en/packages/mdbook-trpl/tests/integration/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
mod note {
use assert_cmd::Command;
#[test]
fn supports_html_renderer() {
let cmd = Command::cargo_bin("mdbook-trpl-note")
.unwrap()
.args(["supports", "html"])
.ok();
assert!(cmd.is_ok());
}

#[test]
fn errors_for_other_renderers() {
let cmd = Command::cargo_bin("mdbook-trpl-note")
.unwrap()
.args(["supports", "total-nonsense"])
.ok();
assert!(cmd.is_err());
}
}
5 changes: 5 additions & 0 deletions rustbook-en/packages/tools/Cargo.toml
Original file line number Diff line number Diff line change
@@ -36,6 +36,11 @@ path = "src/bin/remove_links.rs"
name = "remove_markup"
path = "src/bin/remove_markup.rs"

[[bin]]
name = "cleanup_blockquotes"
path = "src/bin/cleanup_blockquotes.rs"


[dependencies]
walkdir = { workspace = true }
docopt = { workspace = true }
147 changes: 147 additions & 0 deletions rustbook-en/packages/tools/src/bin/cleanup_blockquotes.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
//! Fix incorrect round-tripping of block quotes in `pulldown-cmark-to-cmark`:
//!
//! - Eliminate extraneous leading `>`
//! - Eliminate extraneous indent.
//!
//! Note: later versions of `pulldown-cmark-to-cmark` will likely fix this, so
//! check when upgrading it if it is still necessary!
use std::io::{self, Read};

use lazy_static::lazy_static;
use regex::Regex;

fn main() {
let input = {
let mut buffer = String::new();
io::stdin()
.read_to_string(&mut buffer)
.unwrap_or_else(|e| panic!("{e}"));
buffer
};

let fixed = cleanup_blockquotes(input);
print!("{fixed}");
}

fn cleanup_blockquotes(input: String) -> String {
let normal_start = EXTRA_SPACE.replace_all(&input, ">");
let sans_empty_leading = EMPTY_LEADING.replace_all(&normal_start, "\n\n");
sans_empty_leading.to_string()
}

lazy_static! {
static ref EXTRA_SPACE: Regex = Regex::new("(?m)^ >").unwrap();
static ref EMPTY_LEADING: Regex = Regex::new("\n\n> ?\n").unwrap();
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn extra_space() {
let input = " > Hello".to_string();
let actual = cleanup_blockquotes(input);
assert_eq!(actual, "> Hello");
}

#[test]
fn empty_leading() {
let input = "\n\n>\n> Hello".into();
let actual = cleanup_blockquotes(input);
assert_eq!(actual, "\n\n> Hello");
}

#[test]
fn leading_after_extra_space_cleaned_up() {
let input = r#"Start
>
> Note: Hey.
Wrap."#
.into();

let actual = cleanup_blockquotes(input);
assert_eq!(
actual,
r#"Start
> Note: Hey.
Wrap."#
);
}

/// This particular input was the result of running any of the mdbook
/// preprocessors which use `pulldown-cmark-to-cmark@<=18.0.0`.
#[test]
fn regression_ch17_example() {
// This is an example of the original motivating input which we are fixing.
let input = r#"
We have to explicitly await both of these futures, because futures in Rust are
*lazy*: they don’t do anything until you ask them to with `await`. (In fact,
Rust will show a compiler warning if you don’t use a future.) This should
remind you of our discussion of iterators [back in Chapter 13][iterators-lazy].
Iterators do nothing unless you call their `next` method—whether directly, or
using `for` loops or methods such as `map` which use `next` under the hood. With
futures, the same basic idea applies: they do nothing unless you explicitly ask
them to. This laziness allows Rust to avoid running async code until it’s
actually needed.
>
> Note: This is different from the behavior we saw when using `thread::spawn` in
> the previous chapter, where the closure we passed to another thread started
> running immediately. It’s also different from how many other languages
> approach async! But it’s important for Rust. We’ll see why that is later.
Once we have `response_text`, we can then parse it into an instance of the
`Html` type using `Html::parse`. Instead of a raw string, we now have a data
type we can use to work with the HTML as a richer data structure. In particular,
we can use the `select_first` method to find the first instance of a given CSS
selector. By passing the string `"title"`, we’ll get the first `<title>`
element in the document, if there is one. Because there may not be any matching
element, `select_first` returns an `Option<ElementRef>`. Finally, we use the
`Option::map` method, which lets us work with the item in the `Option` if it’s
present, and do nothing if it isn’t. (We could also use a `match` expression
here, but `map` is more idiomatic.) In the body of the function we supply to
`map`, we call `inner_html` on the `title_element` to get its content, which is
a `String`. When all is said and done, we have an `Option<String>`.
"#.to_string();

let actual = cleanup_blockquotes(input);
assert_eq!(
actual,
r#"
We have to explicitly await both of these futures, because futures in Rust are
*lazy*: they don’t do anything until you ask them to with `await`. (In fact,
Rust will show a compiler warning if you don’t use a future.) This should
remind you of our discussion of iterators [back in Chapter 13][iterators-lazy].
Iterators do nothing unless you call their `next` method—whether directly, or
using `for` loops or methods such as `map` which use `next` under the hood. With
futures, the same basic idea applies: they do nothing unless you explicitly ask
them to. This laziness allows Rust to avoid running async code until it’s
actually needed.
> Note: This is different from the behavior we saw when using `thread::spawn` in
> the previous chapter, where the closure we passed to another thread started
> running immediately. It’s also different from how many other languages
> approach async! But it’s important for Rust. We’ll see why that is later.
Once we have `response_text`, we can then parse it into an instance of the
`Html` type using `Html::parse`. Instead of a raw string, we now have a data
type we can use to work with the HTML as a richer data structure. In particular,
we can use the `select_first` method to find the first instance of a given CSS
selector. By passing the string `"title"`, we’ll get the first `<title>`
element in the document, if there is one. Because there may not be any matching
element, `select_first` returns an `Option<ElementRef>`. Finally, we use the
`Option::map` method, which lets us work with the item in the `Option` if it’s
present, and do nothing if it isn’t. (We could also use a `match` expression
here, but `map` is more idiomatic.) In the body of the function we supply to
`map`, we call `inner_html` on the `title_element` to get its content, which is
a `String`. When all is said and done, we have an `Option<String>`.
"#
);
}
}
4 changes: 2 additions & 2 deletions rustbook-en/src/ch04-02-references-and-borrowing.md
Original file line number Diff line number Diff line change
@@ -45,8 +45,8 @@ Let’s take a closer look at the function call here:
```

The `&s1` syntax lets us create a reference that _refers_ to the value of `s1`
but does not own it. Because it does not own it, the value it points to will
not be dropped when the reference stops being used.
but does not own it. Because the reference does not own it, the value it points
to will not be dropped when the reference stops being used.

Likewise, the signature of the function uses `&` to indicate that the type of
the parameter `s` is a reference. Let’s add some explanatory annotations:
15 changes: 6 additions & 9 deletions rustbook-en/tools/nostarch.sh
Original file line number Diff line number Diff line change
@@ -4,13 +4,7 @@ set -eu

cargo build --release

cd packages/mdbook-trpl-listing
cargo install --locked --path .

cd ../mdbook-trpl-note
cargo install --locked --path .

cd ../..
cargo install --locked --path ./packages/mdbook-trpl --offline

mkdir -p tmp
rm -rf tmp/*.md
@@ -20,7 +14,9 @@ rm -rf tmp/markdown
MDBOOK_OUTPUT__MARKDOWN=1 mdbook build nostarch

# Get all the Markdown files
find tmp/markdown -name "${1:-\"\"}*.md" -print0 | \
# TODO: what was this doing and why?!?
# find tmp/markdown -name "${1:-\"\"}*.md" -print0 | \
find tmp/markdown -name "*.md" -print0 | \
# Extract just the filename so we can reuse it easily.
xargs -0 basename | \
# Remove all links followed by `<!-- ignore -->``, then
@@ -29,7 +25,8 @@ while IFS= read -r filename; do
< "tmp/markdown/$filename" ./target/release/remove_links \
| ./target/release/link2print \
| ./target/release/remove_markup \
| ./target/release/remove_hidden_lines > "tmp/$filename"
| ./target/release/remove_hidden_lines \
| ./target/release/cleanup_blockquotes > "tmp/$filename"
done
# Concatenate the files into the `nostarch` dir.
./target/release/concat_chapters tmp nostarch

0 comments on commit 4fa9bda

Please sign in to comment.