Skip to content

Commit

Permalink
Dry first pass at the docs
Browse files Browse the repository at this point in the history
  • Loading branch information
bglw committed Jun 12, 2024
1 parent afb2ad4 commit bf76342
Show file tree
Hide file tree
Showing 3 changed files with 229 additions and 4 deletions.
156 changes: 153 additions & 3 deletions docs/content/docs/_index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,158 @@
---
title: "Getting Started with Toolproof"
nav_title: "Quick start"
title: "Toolproof Overview"
nav_title: "Overview"
nav_section: Root
weight: 1
---

Coming soon.
Toolproof evaluates all `*.toolproof.yml` files it finds. Each file contains exactly one test.

Here's a simple file that tests the `mv` command:

```yml
name: mv moves files

steps:
- I have a "start.txt" file with the content "hello world"
- I run "mv start.txt end.txt"
- The file "end.txt" should contain "hello"
- snapshot: The file "end.txt"
snapshot_content: |-
╎hello world
```
## Syntax
Toolproof files contain a `steps` array, where each item is a test step. The steps you write are matched to Toolproof functions.

Toolproof's syntax uses plaintext sentences with placeholders,
for example the first step in our example file above matches the Toolproof function:
```
I have a {name} file with the content {text}
```

When writing steps, you can specify the values inline using either single or double quotes:
```yaml
steps:
- I have a "start.txt" file with the content 'hello world'
```

Alternatively, you can specify the values using keys, which can be preferred for long instructions or multiline values.
When doing this, place the step inside an object under a `step` key, and use curly braces to match the key names:
```yaml
steps:
- step: I have a {filename} file with the content {text}
filename: start.txt
text: |-
hello
world
```

By convention, these steps are being written in yaml without quotes. They can also be wrapped in quotes when needed:
```yaml
steps:
- "I have a 'start.txt' file with the content 'hello world'"
```

## Terminology

Steps are comprised of the following elements:

### Instructions

Instructions generally _do_ things. For example, creating a file:
```
I have a {file} with the content {text}
```

or running a command in a terminal:
```
I run {command}
```

Instructions can be a step on their own:
```yaml
steps:
- I run "echo 'hi'"
```

### Retrievals

Retrievals get values, and are used either in assertions or snapshots. For example, getting the contents of a file:
```
The file {name}
```

or getting the contents of stdout:
```
stdout
```

Retrievals can be used for snapshots, or can be paired with an Assertion to make up a step.

### Assertions

Assertions test values. For example, testing exact matches:
```
be exactly {expected}
```

or having a value:
```
not be empty
```

To make up a step, join an Assertion to a Retrieval with `should`:
```yaml
steps:
- The file "index.html" should not be empty
- stdout should contain "hello"
```

### Snapshots

Any Retrieval can also drive a snapshot. To do so, place the step inside an object under a `snapshot` key:
```yaml
steps:
- snapshot: stdout
- snapshot: The file "index.html"
```

After running Toolproof in interactive mode and accepting changes, the file will be updated to:
```yaml
steps:
- snapshot: stdout
snapshot_content: |-
╎contents of
╎stdout go here
- snapshot: The file "index.html"
snapshot_content: |-
╎<body>
╎ <h1>Hello World</h1>
╎</body>
```

In future runs, Toolproof will ensure the retrieved value matches the `snapshot_content` key. Running Toolproof in
interactive mode (`-i`) will also allow you to accept the changes and update the file automatically.

## Test environment

Toolproof automatically runs tests in a temporary directory that is discarded at the end of a run.

Any steps that interact with files will act in this directory, and commands will run relative to this directory.

## Placeholders

Placeholders can be supplied for tests that require dynamic values. These can be supplied in a config file or on the command line:
```bash
npx toolproof --placeholders project_dir="$(pwd)" -i
```

These can be accessed inside any value:
```yaml
steps:
- I run "%project_dir%/index.js"
- step: stdout should contain {text}
text: |-
Hello world from %project_dir%
```
2 changes: 1 addition & 1 deletion docs/content/docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: "Installing and running Toolproof"
nav_title: "Installing Toolproof"
nav_section: References
weight: 49
weight: 5
---

Toolproof is a static binary with no dynamic dependencies, so in most cases will be simple to install and run. Toolproof is currently supported on Windows, macOS, and Linux distributions.
Expand Down
75 changes: 75 additions & 0 deletions docs/content/docs/reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
title: "Functions Reference"
nav_title: "Functions Reference"
nav_section: References
weight: 6
---

Toolproof provides the following Instructions:

## Filesystem

Instructions:
- `I have a {filename} file with the content {contents}`

Retrievals:
- `The file {filename}`
- Returns a string value

## Process

Instructions:
- `I have the environment variable {name} set to {value}`
- `I run {command}`
- `I run {command} and expect it to fail`

Retrievals:
- `stdout`
- Returns a string value
- `stderr`
- Returns a string value

## Hosting

Instructions:
- `I serve the directory {dir}`

## Browser

Instructions:
- `In my browser, I load {url}`
- `In my browser, I evaluate {js}`

Retrievals:
- `In my browser, the result of {js}`
- Returns a a value of the returned type
- `In my browser, the console`
- Returns a string value

## Assertions

### Exact assertions
- `be exactly {expected}`
- `not be exactly {expected}`

Exact assertions can compare complex objects. For example:
```yaml
steps:
- step: In my browser, the result of {js} should be exactly {result}
js: |-
return { hello: "world", numbers: [1, 2, 3] };
result:
hello: world
numbers:
- 1
- 2
- 3
```
### Contain assertions
- `contain {expected}`
- `not contain {expected}`

### Presence assertions
- `be empty`
- `not be empty`

0 comments on commit bf76342

Please sign in to comment.