Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary guide #26

Merged
merged 9 commits into from
Dec 4, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Will work on actual guide next.
ninjeeter committed Nov 28, 2024
commit b4499d1022a081e960830060281d58d0d9818278
4 changes: 4 additions & 0 deletions .vitepress/sidebars/guides.ts
Original file line number Diff line number Diff line change
@@ -46,6 +46,10 @@ export const guidesSidebar: DefaultTheme.SidebarItem[] = [
text: "Spawning a Process",
link: "/guides/components/spawning_process",
},
{
text: "Using Invalid UTF-8",
link: "/guides/components/utf",
},
],
},
{
Binary file added src/_images/raw_byte.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/_images/replaced_character.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/_images/rust_conversion.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
82 changes: 58 additions & 24 deletions src/concepts/essentials/binary.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,74 @@
# Dealing with Binary Data
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would create a new concept section Backend runtime

Copy link
Member

@Corb3nik Corb3nik Dec 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sytten "Dealing with Binary Data" is a lot more meaningful than "Backend runtime" for a user

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Section is the parent of the concept

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah gotcha, makes sense now 👌


Caido plugins are written in JavaScript. When creating a plugin that must handle raw bytes, there are some caveats to be aware of as the backend language used by Caido is Rust.

## UTF-8 Encoding

For Unicode code points outside of the ASCII range <span style="color: #EBEBF599; font-style: italic">(U+0000 to U+007F)</span> multiple bytes are used.

When encoding code points into multibyte UTF-8 characters, the bytes are prefixed with specific bit patterns. The first byte prefix indicates how many bytes are used, and any subsequent bytes begin with 10.

<img alt="UTF-8 encoding chart." src="/_images/utf_chart.png" center/>

This can be an issue if you want to create a Caido plugin that sends binary to a target server to see how it is processed.

## Example

Consider this `/packages/backend/src/index.ts` file:

``` ts
import { RequestSpecRaw } from "caido:utils";
import { SDK, DefineAPI } from "caido:plugin";

// Function to convert a string into an array of bytes.
function stringToUint8Array(str: string): Uint8Array {
const arr = new Uint8Array(str.length);
for (let i = 0; i < str.length; i++) {
arr[i] = str.charCodeAt(i);
}
return arr;
}

async function testSendRequest(sdk: SDK): Promise<void> {
console.log("Testing send request");
// Create a new request using bytes.
const req = new RequestSpecRaw("http://localhost:5555");
req.setRaw("GET /admin\x85 HTTP/1.1\r\nHost: HOST\r\n\r\n");

// Convert the request line into a byte array.
const rawRequest = "GET /admin\x85 HTTP/1.1\r\nHost: localhost:5555\r\n\r\n";
req.setRaw(stringToUint8Array(rawRequest));

// Send the request.
const res = await sdk.requests.send(req);
console.log(res?.response.getRaw().toText());
}
```

// Define the API for the plugin.
export type API = DefineAPI<{
testSendRequest: typeof testSendRequest;
}>;
The intention is to send the byte `\x85` with a binary value of `[1000 0101]`. However, JavaScript is interpreting it as the Unicode code point.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code point C2 85.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the goal was to send just the byte 85?


// Initialize the plugin.
export function init(sdk: SDK<API>) {
sdk.api.register("testSendRequest", testSendRequest);
}
```
::: info This happens because:

1. JavaScript strings are UTF-16 encoded.
2. `\x85` in a string becomes `U+0085` in UTF-16 (`[0000 0000 1000 0101]`).
3. When passed to Rust, `U+0085` is encoded as UTF-8:
- First byte `C2`: `[11000010]` <span style="color: #EBEBF599; font-style: italic">(prefix marks start of 2-byte sequence)</span>
- Second byte `85`: `[10000101]` <span style="color: #EBEBF599; font-style: italic">(continuation byte)</span>
:::

This will result in it being interpreted as the unprintable `NEL` <span style="color: #EBEBF599; font-style: italic">(Next Line)</span> character, which will be replaced with `�`.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence plus the tip make no sense in context, it was probably moved around or something before it was removed.
You just said that this will send C2 85 and NOT just 85. This exemple is when the actual character 85 is sent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to show what happens if you dont deal with bytes correctly, then you would should xxd capture with C2 85


::: tip
You can view this visually by listening for the request with Netcat:

`ncat -lvnp 5555`

<img alt="Unprintable character." src="/_images/replaced_character.png" center/>

You can view the bytes by piping `xxd` to display the hex dump:

`ncat -lvnp 5555 | xxd -g1`

<img alt="C2 85" src="/_images/rust_conversion.png" center/>
:::

To preserve the byte, the raw byte must be used instead. By sending it raw, it is invalid UTF-8. When the target receives invalid UTF-8 it may fallback to a different encoding standard to try and make sense of it:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a guide, but we can give at least one example like:

// path is /admin
let path = [...req.getPath({ raw: true }), 0x85];
req.setPath(path)


Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move here the NEL explanation + screens.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So keep everything but just move it underneath this? Starting at the "Example" header?

::: info Byte value: `0x85`

Different interpretations:

1. In CP437 (IBM PC): `å` <span style="color: #EBEBF599; font-style: italic">(lowercase a with ring)</span>
2. In Windows-1252: `…` <span style="color: #EBEBF599; font-style: italic">(horizontal ellipsis)</span>
3. As Unicode code point U+0085: `NEL` <span style="color: #EBEBF599; font-style: italic">(Next Line)</span>
4. As raw byte: `133` <span style="color: #EBEBF599; font-style: italic">(in decimal)</span>
:::

This can result in security bypasses. For example, if a validation filter is matching against the `/admin` path, if it receives `/admin…` it may allow the request to pass through.

To learn how you can use raw bytes in Caido plugins, click [here](/guides/components/utf.md).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We dont care about that in this concept

97 changes: 97 additions & 0 deletions src/guides/components/utf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Using Invalid UTF-8

``` ts
import { RequestSpecRaw } from "caido:utils";
import { SDK, DefineAPI } from "caido:plugin";

// Function to convert a string into an array of bytes.
function stringToUint8Array(str: string): Uint8Array {
const arr = new Uint8Array(str.length);
for (let i = 0; i < str.length; i++) {
arr[i] = str.charCodeAt(i);
}
return arr;
}

async function testSendRequest(sdk: SDK): Promise<void> {
console.log("Testing send request");
// Create a new request using bytes.
const req = new RequestSpecRaw("http://localhost:5555");

// Convert the request line into a byte array.
const rawRequest = "GET /admin\x85 HTTP/1.1\r\nHost: localhost:5555\r\n\r\n";
req.setRaw(stringToUint8Array(rawRequest));

// Send the request.
const res = await sdk.requests.send(req);
console.log(res?.response.getRaw().toText());
}

// Define the API for the plugin.
export type API = DefineAPI<{
testSendRequest: typeof testSendRequest;
}>;

// Initialize the plugin.
export function init(sdk: SDK<API>) {
sdk.api.register("testSendRequest", testSendRequest);
}
```

::: tip
<details>
<summary>To view the entire frontend script, expand the following:</summary>

``` ts
import type { Caido } from "@caido/sdk-frontend";
import type { API } from "../../backend/src/index";

import "./styles/index.css";

export type CaidoSDK = Caido<API>;

const Commands = {
sending: "my-plugin-page.req",
} as const;

const sending = async (sdk: CaidoSDK) => {
await sdk.backend.testSendRequest();
};

const createPage = (sdk: CaidoSDK) => {
const requestButton = sdk.ui.button({
variant: "primary",
label: "Send Request",
});

requestButton.addEventListener("click", async () => {
await sending(sdk);
});

const bodyContainer = document.createElement("div");
bodyContainer.appendChild(requestButton);

const card = sdk.ui.card({
body: bodyContainer,

});

sdk.navigation.addPage("/my-plugin-page", {
body: card,
});
};

export const init = (sdk: CaidoSDK) => {
createPage(sdk);
sdk.sidebar.registerItem("My Plugin", "/my-plugin-page", {
icon: "fas fa-rocket",
});

sdk.commands.register(Commands.sending, {
name: "Send Request",
run: () => sending(sdk),
});
};
```

</details>