-
Notifications
You must be signed in to change notification settings - Fork 360
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle non-utf8 strings in protobuf structures #1282
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BPF/kernel strings are C strings which are a sequence of null-terminated bytes. They may or may not be valid utf-8 strings. For example, execve() can accept invalid utf-8 strings. Similarly, filenames are not required to be utf-8 encoded. This becomes an issue because we define fields like Binary, Arguments, and CWD as strings in the proto descriptions. According to the protobuf spec: "A string must always contain UTF-8 encoded or 7-bit ASCII text, and cannot be longer than 2^32." As a result, when passing non-utf8 data as strings, protobuf clients (e.g., the JSON writer and the gRPC client) cannot handle the event. For example, running `tetra getevents` and executing a program with invalid utf8 arguments leads the following error: msg="Failed to receive events" error="rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8" There are a number of different approaches we can use to address this problem. One is that we can try to encode arbitrary bytes in the string. In the past, we have used base64 to do that (specifically, for bytes arguments). An alternative (better?) solution would be to quote the string, using strconv.Quote() or similar. Quoting, however, means that we need to always parse the data and quote them, which has a performance cost. To avoid this cost this patch uses strings.ToValidUTF8 instead, which has small overhead (minimal in case where the data are actually valid utf8), to replace invalid runes with "�". This means that we loose information (what the actual bytes were) but we also keep backwards compatibility and remain close to the existing behavior. In the future, we can modify this behavior (e.g., via a command line switch) to quote the arguments instead, so that all bytes are preserved for non-utf8 data encoded as strings. A more radical change (which seems like the right thing to do) is to change the gRPC fields that are not actually strings to bytes, and let the clients do the decoding. The patch also adds a unit test to check for this case. Reported-by: Гаврилов Иван Сергеевич <[email protected]> Signed-off-by: Kornilios Kourtis <[email protected]>
handleGenericKprobeString returns "/" in case of an error. This happens to accommodate issues when traversing paths. This patch modifies handleGenericKprobeString so that it accepts a default value to return as an error. The intention is to use it for other types, which be done in the next patch. No functional changes. Signed-off-by: Kornilios Kourtis <[email protected]>
Use handleGenericKprobeString for GenericFilenameType and GenericStringType as well. The code is the same, so no functional changes. Signed-off-by: Kornilios Kourtis <[email protected]>
Kprobe arguments that include strings are generated using handleGenericKprobeString. Because protobufs only support strings that are valid utf-8, we need to ensure that the strings we get from bpf are valid utf-8. Also, ensure that Path in loader is valid utf-8. Signed-off-by: Kornilios Kourtis <[email protected]>
This code is seemingly not used. Remove it. Signed-off-by: Kornilios Kourtis <[email protected]> Suggested-by: Anastasios Papagiannis <[email protected]>
kkourt
changed the title
Pr/kkourt/nonutf8
handle non-utf8 strings in protobuf structures
Jul 28, 2023
kkourt
changed the title
handle non-utf8 strings in protobuf structures
Handle non-utf8 strings in protobuf structures
Jul 28, 2023
tpapagian
approved these changes
Jul 28, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks! (assuming tests are green)
kkourt
added
release-note/bug
This PR fixes an issue in a previous release of Tetragon.
needs-backport/0.9
labels
Jul 28, 2023
This was referenced Jul 28, 2023
Closed
Merged
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
BPF/kernel strings are C strings which are a sequence of null-terminated
bytes. They may or may not be valid utf-8 strings.
For example, execve() can accept invalid utf-8 strings. Similarly,
filenames are not required to be utf-8 encoded.
This becomes an issue because we define fields like Binary, Arguments,
and CWD as strings in the proto descriptions.
According to the protobuf spec:
"A string must always contain UTF-8 encoded or 7-bit ASCII text, and
cannot be longer than 2^32."
As a result, when passing non-utf8 data as strings, protobuf clients
(e.g., the JSON writer and the gRPC client) cannot handle the event. For
example, running
tetra getevents
and executing a program with invalidutf8 arguments leads the following error:
msg="Failed to receive events" error="rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8"
There are a number of different approaches we can use to address this
problem. One is that we can try to encode arbitrary bytes in the
string. In the past, we have used base64 to do that (specifically, for
bytes arguments). An alternative (better?) solution would be to quote
the string, using strconv.Quote() or similar.
Quoting, however, means that we need to always parse the data and quote
them, which has a performance cost. To avoid this cost this patch
uses strings.ToValidUTF8 instead, which has small overhead (minimal in
case where the data are actually valid utf8), to replace invalid runes
with "�". This means that we loose information (what the actual bytes
were) but we also keep backwards compatibility and remain close to the
existing behavior.
In the future, we can modify this behavior (e.g., via a command line
switch) to quote the arguments instead, so that all bytes are preserved
for non-utf8 data encoded as strings.
A more radical change (which seems like the right thing to do) is to
change the gRPC fields that are not actually strings to bytes, and let
the clients do the decoding.
Reported-by: Гаврилов Иван Сергеевич [email protected]
Signed-off-by: Kornilios Kourtis [email protected]