Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Binary guide #26

Merged
merged 9 commits into from
Dec 4, 2024
Merged

Binary guide #26

merged 9 commits into from
Dec 4, 2024

Conversation

Sytten
Copy link
Member

@Sytten Sytten commented Dec 1, 2024

No description provided.

@caidobot caidobot temporarily deployed to AP-Guides - doc-developer PR #26 December 1, 2024 01:46 — with Render Destroyed
Copy link
Contributor

github-actions bot commented Dec 1, 2024

Images automagically compressed by Calibre's image-actions

Compression reduced images by 32.2%, saving 32.37 KB.

Filename Before After Improvement Visual comparison
src/_images/raw_byte.png 22.12 KB 13.45 KB -39.2% View diff
src/_images/replaced_character.png 11.89 KB 7.91 KB -33.5% View diff
src/_images/rust_conversion.png 18.71 KB 11.64 KB -37.8% View diff
src/_images/utf_chart.png 47.96 KB 35.31 KB -26.4% View diff

15 images did not require optimisation.

@@ -0,0 +1,74 @@
# Dealing with Binary Data
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would create a new concept section Backend runtime

Copy link
Member

@Corb3nik Corb3nik Dec 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sytten "Dealing with Binary Data" is a lot more meaningful than "Backend runtime" for a user

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Section is the parent of the concept

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah gotcha, makes sense now 👌

}
```

The intention is to send the byte `\x85` with a binary value of `[1000 0101]`. However, JavaScript is interpreting it as the Unicode code point.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code point C2 85.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the goal was to send just the byte 85?

- Second byte `85`: `[10000101]` <span style="color: #EBEBF599; font-style: italic">(continuation byte)</span>
:::

This will result in it being interpreted as the unprintable `NEL` <span style="color: #EBEBF599; font-style: italic">(Next Line)</span> character, which will be replaced with `�`.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence plus the tip make no sense in context, it was probably moved around or something before it was removed.
You just said that this will send C2 85 and NOT just 85. This exemple is when the actual character 85 is sent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to show what happens if you dont deal with bytes correctly, then you would should xxd capture with C2 85

Comment on lines 72 to 74
This can result in security bypasses. For example, if a validation filter is matching against the `/admin` path, if it receives `/admin…` it may allow the request to pass through.

To learn how you can use raw bytes in Caido plugins, click [here](/guides/components/utf.md).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We dont care about that in this concept

<img alt="C2 85" src="/_images/rust_conversion.png" center/>
:::

To preserve the byte, the raw byte must be used instead. By sending it raw, it is invalid UTF-8. When the target receives invalid UTF-8 it may fallback to a different encoding standard to try and make sense of it:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a guide, but we can give at least one example like:

// path is /admin
let path = [...req.getPath({ raw: true }), 0x85];
req.setPath(path)

:::

To preserve the byte, the raw byte must be used instead. By sending it raw, it is invalid UTF-8. When the target receives invalid UTF-8 it may fallback to a different encoding standard to try and make sense of it:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move here the NEL explanation + screens.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So keep everything but just move it underneath this? Starting at the "Example" header?

Copy link
Contributor

github-actions bot commented Dec 3, 2024

Images automagically compressed by Calibre's image-actions

Compression reduced images by 38.7%, saving 61.97 KB.

Filename Before After Improvement Visual comparison
src/_images/requestspec_raw_byte.png 160.14 KB 98.17 KB -38.7% View diff

19 images did not require optimisation.

@caidobot caidobot temporarily deployed to AP-Guides - doc-developer PR #26 December 3, 2024 22:28 — with Render Destroyed
Copy link
Contributor

github-actions bot commented Dec 3, 2024

Images automagically compressed by Calibre's image-actions

Compression reduced images by 46.2%, saving 55.01 KB.

Filename Before After Improvement Visual comparison
src/_images/raw_byte_example.png 119.02 KB 64.01 KB -46.2% View diff

18 images did not require optimisation.

@caidobot caidobot temporarily deployed to AP-Guides - doc-developer PR #26 December 3, 2024 22:28 — with Render Destroyed
@Sytten Sytten merged commit cf57780 into main Dec 4, 2024
1 check passed
@Sytten Sytten deleted the AP-Guides branch December 4, 2024 23:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants