-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Binary guide #26
Binary guide #26
Changes from 1 commit
07f90c2
3013f3f
d4bb846
b4499d1
4bbdc50
d73f4fb
166fabb
15e1c44
a1963b1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,40 +1,74 @@ | ||
# Dealing with Binary Data | ||
|
||
Caido plugins are written in JavaScript. When creating a plugin that must handle raw bytes, there are some caveats to be aware of as the backend language used by Caido is Rust. | ||
|
||
## UTF-8 Encoding | ||
|
||
For Unicode code points outside of the ASCII range <span style="color: #EBEBF599; font-style: italic">(U+0000 to U+007F)</span> multiple bytes are used. | ||
|
||
When encoding code points into multibyte UTF-8 characters, the bytes are prefixed with specific bit patterns. The first byte prefix indicates how many bytes are used, and any subsequent bytes begin with 10. | ||
|
||
<img alt="UTF-8 encoding chart." src="/_images/utf_chart.png" center/> | ||
|
||
This can be an issue if you want to create a Caido plugin that sends binary to a target server to see how it is processed. | ||
|
||
## Example | ||
|
||
Consider this `/packages/backend/src/index.ts` file: | ||
|
||
``` ts | ||
import { RequestSpecRaw } from "caido:utils"; | ||
import { SDK, DefineAPI } from "caido:plugin"; | ||
|
||
// Function to convert a string into an array of bytes. | ||
function stringToUint8Array(str: string): Uint8Array { | ||
const arr = new Uint8Array(str.length); | ||
for (let i = 0; i < str.length; i++) { | ||
arr[i] = str.charCodeAt(i); | ||
} | ||
return arr; | ||
} | ||
|
||
async function testSendRequest(sdk: SDK): Promise<void> { | ||
console.log("Testing send request"); | ||
// Create a new request using bytes. | ||
const req = new RequestSpecRaw("http://localhost:5555"); | ||
req.setRaw("GET /admin\x85 HTTP/1.1\r\nHost: HOST\r\n\r\n"); | ||
|
||
// Convert the request line into a byte array. | ||
const rawRequest = "GET /admin\x85 HTTP/1.1\r\nHost: localhost:5555\r\n\r\n"; | ||
req.setRaw(stringToUint8Array(rawRequest)); | ||
|
||
// Send the request. | ||
const res = await sdk.requests.send(req); | ||
console.log(res?.response.getRaw().toText()); | ||
} | ||
``` | ||
|
||
// Define the API for the plugin. | ||
export type API = DefineAPI<{ | ||
testSendRequest: typeof testSendRequest; | ||
}>; | ||
The intention is to send the byte `\x85` with a binary value of `[1000 0101]`. However, JavaScript is interpreting it as the Unicode code point. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. code point There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought the goal was to send just the byte 85? |
||
|
||
// Initialize the plugin. | ||
export function init(sdk: SDK<API>) { | ||
sdk.api.register("testSendRequest", testSendRequest); | ||
} | ||
``` | ||
::: info This happens because: | ||
|
||
1. JavaScript strings are UTF-16 encoded. | ||
2. `\x85` in a string becomes `U+0085` in UTF-16 (`[0000 0000 1000 0101]`). | ||
3. When passed to Rust, `U+0085` is encoded as UTF-8: | ||
- First byte `C2`: `[11000010]` <span style="color: #EBEBF599; font-style: italic">(prefix marks start of 2-byte sequence)</span> | ||
- Second byte `85`: `[10000101]` <span style="color: #EBEBF599; font-style: italic">(continuation byte)</span> | ||
::: | ||
|
||
This will result in it being interpreted as the unprintable `NEL` <span style="color: #EBEBF599; font-style: italic">(Next Line)</span> character, which will be replaced with `�`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This sentence plus the tip make no sense in context, it was probably moved around or something before it was removed. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you want to show what happens if you dont deal with bytes correctly, then you would should xxd capture with C2 85 |
||
|
||
::: tip | ||
You can view this visually by listening for the request with Netcat: | ||
|
||
`ncat -lvnp 5555` | ||
|
||
<img alt="Unprintable character." src="/_images/replaced_character.png" center/> | ||
|
||
You can view the bytes by piping `xxd` to display the hex dump: | ||
|
||
`ncat -lvnp 5555 | xxd -g1` | ||
|
||
<img alt="C2 85" src="/_images/rust_conversion.png" center/> | ||
::: | ||
|
||
To preserve the byte, the raw byte must be used instead. By sending it raw, it is invalid UTF-8. When the target receives invalid UTF-8 it may fallback to a different encoding standard to try and make sense of it: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not a guide, but we can give at least one example like: // path is /admin
let path = [...req.getPath({ raw: true }), 0x85];
req.setPath(path) |
||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Move here the NEL explanation + screens. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So keep everything but just move it underneath this? Starting at the "Example" header? |
||
::: info Byte value: `0x85` | ||
|
||
Different interpretations: | ||
|
||
1. In CP437 (IBM PC): `å` <span style="color: #EBEBF599; font-style: italic">(lowercase a with ring)</span> | ||
2. In Windows-1252: `…` <span style="color: #EBEBF599; font-style: italic">(horizontal ellipsis)</span> | ||
3. As Unicode code point U+0085: `NEL` <span style="color: #EBEBF599; font-style: italic">(Next Line)</span> | ||
4. As raw byte: `133` <span style="color: #EBEBF599; font-style: italic">(in decimal)</span> | ||
::: | ||
|
||
This can result in security bypasses. For example, if a validation filter is matching against the `/admin` path, if it receives `/admin…` it may allow the request to pass through. | ||
|
||
To learn how you can use raw bytes in Caido plugins, click [here](/guides/components/utf.md). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We dont care about that in this concept |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
# Using Invalid UTF-8 | ||
|
||
``` ts | ||
import { RequestSpecRaw } from "caido:utils"; | ||
import { SDK, DefineAPI } from "caido:plugin"; | ||
|
||
// Function to convert a string into an array of bytes. | ||
function stringToUint8Array(str: string): Uint8Array { | ||
const arr = new Uint8Array(str.length); | ||
for (let i = 0; i < str.length; i++) { | ||
arr[i] = str.charCodeAt(i); | ||
} | ||
return arr; | ||
} | ||
|
||
async function testSendRequest(sdk: SDK): Promise<void> { | ||
console.log("Testing send request"); | ||
// Create a new request using bytes. | ||
const req = new RequestSpecRaw("http://localhost:5555"); | ||
|
||
// Convert the request line into a byte array. | ||
const rawRequest = "GET /admin\x85 HTTP/1.1\r\nHost: localhost:5555\r\n\r\n"; | ||
req.setRaw(stringToUint8Array(rawRequest)); | ||
|
||
// Send the request. | ||
const res = await sdk.requests.send(req); | ||
console.log(res?.response.getRaw().toText()); | ||
} | ||
|
||
// Define the API for the plugin. | ||
export type API = DefineAPI<{ | ||
testSendRequest: typeof testSendRequest; | ||
}>; | ||
|
||
// Initialize the plugin. | ||
export function init(sdk: SDK<API>) { | ||
sdk.api.register("testSendRequest", testSendRequest); | ||
} | ||
``` | ||
|
||
::: tip | ||
<details> | ||
<summary>To view the entire frontend script, expand the following:</summary> | ||
|
||
``` ts | ||
import type { Caido } from "@caido/sdk-frontend"; | ||
import type { API } from "../../backend/src/index"; | ||
|
||
import "./styles/index.css"; | ||
|
||
export type CaidoSDK = Caido<API>; | ||
|
||
const Commands = { | ||
sending: "my-plugin-page.req", | ||
} as const; | ||
|
||
const sending = async (sdk: CaidoSDK) => { | ||
await sdk.backend.testSendRequest(); | ||
}; | ||
|
||
const createPage = (sdk: CaidoSDK) => { | ||
const requestButton = sdk.ui.button({ | ||
variant: "primary", | ||
label: "Send Request", | ||
}); | ||
|
||
requestButton.addEventListener("click", async () => { | ||
await sending(sdk); | ||
}); | ||
|
||
const bodyContainer = document.createElement("div"); | ||
bodyContainer.appendChild(requestButton); | ||
|
||
const card = sdk.ui.card({ | ||
body: bodyContainer, | ||
|
||
}); | ||
|
||
sdk.navigation.addPage("/my-plugin-page", { | ||
body: card, | ||
}); | ||
}; | ||
|
||
export const init = (sdk: CaidoSDK) => { | ||
createPage(sdk); | ||
sdk.sidebar.registerItem("My Plugin", "/my-plugin-page", { | ||
icon: "fas fa-rocket", | ||
}); | ||
|
||
sdk.commands.register(Commands.sending, { | ||
name: "Send Request", | ||
run: () => sending(sdk), | ||
}); | ||
}; | ||
``` | ||
|
||
</details> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would create a new concept section
Backend runtime
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sytten "Dealing with Binary Data" is a lot more meaningful than "Backend runtime" for a user
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Section is the parent of the concept
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah gotcha, makes sense now 👌