Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle non-utf8 strings in protobuf structures #1282

Merged
merged 5 commits into from
Jul 28, 2023
Merged

Conversation

kkourt
Copy link
Contributor

@kkourt kkourt commented Jul 28, 2023

BPF/kernel strings are C strings which are a sequence of null-terminated
bytes. They may or may not be valid utf-8 strings.

For example, execve() can accept invalid utf-8 strings. Similarly,
filenames are not required to be utf-8 encoded.

This becomes an issue because we define fields like Binary, Arguments,
and CWD as strings in the proto descriptions.

According to the protobuf spec:
"A string must always contain UTF-8 encoded or 7-bit ASCII text, and
cannot be longer than 2^32."

As a result, when passing non-utf8 data as strings, protobuf clients
(e.g., the JSON writer and the gRPC client) cannot handle the event. For
example, running tetra getevents and executing a program with invalid
utf8 arguments leads the following error:
msg="Failed to receive events" error="rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8"

There are a number of different approaches we can use to address this
problem. One is that we can try to encode arbitrary bytes in the
string. In the past, we have used base64 to do that (specifically, for
bytes arguments). An alternative (better?) solution would be to quote
the string, using strconv.Quote() or similar.

Quoting, however, means that we need to always parse the data and quote
them, which has a performance cost. To avoid this cost this patch
uses strings.ToValidUTF8 instead, which has small overhead (minimal in
case where the data are actually valid utf8), to replace invalid runes
with "�". This means that we loose information (what the actual bytes
were) but we also keep backwards compatibility and remain close to the
existing behavior.

In the future, we can modify this behavior (e.g., via a command line
switch) to quote the arguments instead, so that all bytes are preserved
for non-utf8 data encoded as strings.

A more radical change (which seems like the right thing to do) is to
change the gRPC fields that are not actually strings to bytes, and let
the clients do the decoding.

Reported-by: Гаврилов Иван Сергеевич [email protected]
Signed-off-by: Kornilios Kourtis [email protected]

Ensure that protobuf strings are valid utf-8

BPF/kernel strings are C strings which are a sequence of null-terminated
bytes. They may or may not be valid utf-8 strings.

For example, execve() can accept invalid utf-8 strings. Similarly,
filenames are not required to be utf-8 encoded.

This becomes an issue because we define fields like Binary, Arguments,
and CWD as strings in the proto descriptions.

According to the protobuf spec:
 "A string must always contain UTF-8 encoded or 7-bit ASCII text, and
  cannot be longer than 2^32."

As a result, when passing non-utf8 data as strings, protobuf clients
(e.g., the JSON writer and the gRPC client) cannot handle the event. For
example, running `tetra getevents` and executing a program with invalid
utf8 arguments leads the following error:
msg="Failed to receive events" error="rpc error: code = Internal desc = grpc: error while marshaling: string field contains invalid UTF-8"

There are a number of different approaches we can use to address this
problem. One is that we can try to encode arbitrary bytes in the
string. In the past, we have used base64 to do that (specifically, for
bytes arguments). An alternative (better?) solution would be to quote
the string, using strconv.Quote() or similar.

Quoting, however, means that we need to always parse the data and quote
them, which has a performance cost. To avoid this cost this patch
uses strings.ToValidUTF8 instead, which has small overhead (minimal in
case where the data are actually valid utf8), to replace invalid runes
with "�". This means that we loose information (what the actual bytes
were) but we also keep backwards compatibility and remain close to the
existing behavior.

In the future, we can modify this behavior (e.g., via a command line
switch) to quote the arguments instead, so that all bytes are preserved
for non-utf8 data encoded as strings.

A more radical change (which seems like the right thing to do) is to
change the gRPC fields that are not actually strings to bytes, and let
the clients do the decoding.

The patch also adds a unit test to check for this case.

Reported-by: Гаврилов Иван Сергеевич <[email protected]>
Signed-off-by: Kornilios Kourtis <[email protected]>
handleGenericKprobeString returns "/" in case of an error. This happens
to accommodate issues when traversing paths. This patch modifies
handleGenericKprobeString so that it accepts a default value to return
as an error. The intention is to use it for other types, which be done
in the next patch.

No functional changes.

Signed-off-by: Kornilios Kourtis <[email protected]>
Use handleGenericKprobeString for GenericFilenameType and
GenericStringType as well. The code is the same, so no functional changes.

Signed-off-by: Kornilios Kourtis <[email protected]>
Kprobe arguments that include strings are generated using
handleGenericKprobeString. Because protobufs only support strings that
are valid utf-8, we need to ensure that the strings we get from bpf are
valid utf-8.

Also, ensure that Path in loader is valid utf-8.

Signed-off-by: Kornilios Kourtis <[email protected]>
This code is seemingly not used. Remove it.

Signed-off-by: Kornilios Kourtis <[email protected]>
Suggested-by: Anastasios Papagiannis <[email protected]>
@kkourt kkourt requested a review from a team as a code owner July 28, 2023 06:27
@kkourt kkourt requested a review from tpapagian July 28, 2023 06:27
@kkourt kkourt changed the title Pr/kkourt/nonutf8 handle non-utf8 strings in protobuf structures Jul 28, 2023
@kkourt kkourt changed the title handle non-utf8 strings in protobuf structures Handle non-utf8 strings in protobuf structures Jul 28, 2023
Copy link
Member

@tpapagian tpapagian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! (assuming tests are green)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note/bug This PR fixes an issue in a previous release of Tetragon.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants