Merge remote-tracking branch 'upstream/master' into 6.0/stage #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The basic support for LoongArch has been merged into the upstream Linux kernel since 5.19-rc1 on June 5, 2022, the kernel ABI definitions have settled down. In order to run the bcc scripts on LoongArch, add basic and usdt support. Here is the LoongArch documention: https://www.kernel.org/doc/html/latest/loongarch/index.html Co-developed-by: Youling Tang <[email protected]> Signed-off-by: Youling Tang <[email protected]> Signed-off-by: Tiezhu Yang <[email protected]>
Originally, the tcpconnect utility didn't display the source port when tracing events and ignored it (intentionally) when counting new connections. Add a new option -s (or --source-port) to display the source port when tracing, or to use the source port as part of the key when counting connections. This option is unset by default to provide the original output. Signed-off-by: Anton Protopopov <[email protected]>
“blk_start_request” and “blk_mq_start_request” should be chosen between the two.
When working on fixing individual tests, it is useful to be able to tell the test suite to only run that individual test. The catch framework allows to do this. See https://github.com/catchorg/Catch2/blob/devel/docs/command-line.md#specifying-which-tests-to-run When the wrapper was originally introduced, it used to pass the list of arguments to the tests as an array. For some reasons, it got changed in iovisor/bcc@7009b55#diff-29e66e11b6682a5b66d214c108dd3c351a557bc884b64946af004e0e3195d209 This diff, re-introduces passing the list of arguments as an array (`$@`). Also, it needs to be double-quoted in order to be able to handle space in test names. Before: ``` $ docker run -ti \ --privileged \ --network=host \ --pid=host \ -v $(pwd):/bcc \ -v /sys/kernel/debug:/sys/kernel/debug:rw \ -v /lib/modules:/lib/modules:ro \ -v /usr/src:/usr/src:ro \ -e CTEST_OUTPUT_ON_FAILURE=1 \ bcc-docker \ /bin/bash -c \ '/bcc/build/tests/wrapper.sh \ c_test_all sudo /bcc/build/tests/cc/test_libbcc "test prob*"' =============================================================================== No tests ran ``` After: ``` $ docker run -ti \ --privileged \ --network=host \ --pid=host \ -v $(pwd):/bcc \ -v /sys/kernel/debug:/sys/kernel/debug:rw \ -v /lib/modules:/lib/modules:ro \ -v /usr/src:/usr/src:ro \ -e CTEST_OUTPUT_ON_FAILURE=1 \ bcc-docker \ /bin/bash -c \ '/bcc/build/tests/wrapper.sh \ c_test_all sudo /bcc/build/tests/cc/test_libbcc "test prob*"' =============================================================================== All tests passed (15 assertions in 2 test cases) ``` Also tested `namespace` and `simple` kinds: ``` $ docker run -ti \ --privileged \ --network=host \ --pid=host \ -v $(pwd):/bcc \ -v /sys/kernel/debug:/sys/kernel/debug:rw \ -v /lib/modules:/lib/modules:ro \ -v /usr/src:/usr/src:ro \ -e CTEST_OUTPUT_ON_FAILURE=1 \ bcc-docker \ /bin/bash -c \ '/bcc/build/tests/wrapper.sh \ c_test_all simple /bcc/build/tests/cc/test_libbcc "test prob*"' =============================================================================== All tests passed (15 assertions in 2 test cases) [16:08:12] chantra@focal:bcc git:(test_wrapper_fix) $ docker run -ti \ --privileged \ --network=host \ --pid=host \ -v $(pwd):/bcc \ -v /sys/kernel/debug:/sys/kernel/debug:rw \ -v /lib/modules:/lib/modules:ro \ -v /usr/src:/usr/src:ro \ -e CTEST_OUTPUT_ON_FAILURE=1 \ bcc-docker \ /bin/bash -c \ '/bcc/build/tests/wrapper.sh \ c_test_all namespace /bcc/build/tests/cc/test_libbcc "test prob*"' Actual changes: tx-checksumming: off tx-checksum-ip-generic: off tx-checksum-sctp: off tcp-segmentation-offload: off tx-tcp-segmentation: off [requested on] tx-tcp-ecn-segmentation: off [requested on] tx-tcp-mangleid-segmentation: off [requested on] tx-tcp6-segmentation: off [requested on] open(/sys/kernel/debug/tracing/uprobe_events): No such file or directory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ test_libbcc is a Catch v1.4.0 host application. Run with -? for options ------------------------------------------------------------------------------- test probing running Ruby process in namespaces in separate mount namespace ------------------------------------------------------------------------------- /bcc/tests/cc/test_usdt_probes.cc:351 ............................................................................... /bcc/tests/cc/test_usdt_probes.cc:374: FAILED: REQUIRE( res.ok() ) with expansion: false ioctl(PERF_EVENT_IOC_DISABLE) failed: Bad file descriptor close perf event FD failed: Bad file descriptor open(/sys/kernel/debug/tracing/uprobe_events): No such file or directory Failed to detach all probes on destruction: Failed to detach uprobe event p__proc_64889_root_usr_local_bin_ruby_0x453b0_64889: Unable to detach uprobe p__proc_64889_root_usr_local_bin_ruby_0x453b0_64889 open(/sys/kernel/debug/tracing/uprobe_events): No such file or directory ------------------------------------------------------------------------------- test probing running Ruby process in namespaces in separate mount namespace and separate PID namespace ------------------------------------------------------------------------------- /bcc/tests/cc/test_usdt_probes.cc:351 ............................................................................... /bcc/tests/cc/test_usdt_probes.cc:400: FAILED: REQUIRE( res.ok() ) with expansion: false ioctl(PERF_EVENT_IOC_DISABLE) failed: Bad file descriptor close perf event FD failed: Bad file descriptor open(/sys/kernel/debug/tracing/uprobe_events): No such file or directory Failed to detach all probes on destruction: Failed to detach uprobe event p__proc_64891_root_usr_local_bin_ruby_0x453b0_64891: Unable to detach uprobe p__proc_64891_root_usr_local_bin_ruby_0x453b0_64891 =============================================================================== test cases: 2 | 1 passed | 1 failed as expected assertions: 13 | 11 passed | 2 failed as expected ```
Sometimes, I want to known total on-CPU or off-CPU time and count (same as context switch times) at a fixed interval (for example: 1s).
Like #3384, This patch try to add an option -e to show extension summary (average/total/count).
$ ./cpudist.py -p $(pgrep -nx mysqld) -e 1
usecs : count distribution
0 -> 1 : 4123 |************** |
2 -> 3 : 11690 |****************************************|
4 -> 7 : 1668 |***** |
8 -> 15 : 859 |** |
16 -> 31 : 618 |** |
32 -> 63 : 290 | |
64 -> 127 : 247 | |
128 -> 255 : 198 | |
256 -> 511 : 161 | |
512 -> 1023 : 370 |* |
1024 -> 2047 : 98 | |
2048 -> 4095 : 6 | |
4096 -> 8191 : 16 | |
avg = 33 usecs, total: 682091 usecs, count: 20383
Sometimes, I'd only care about a single syscall rather than all syscalls. Use the --syscall option for this. # syscount -i 1 -p $(pgrep -nx mysqld) --syscall fsync -L Tracing syscall 'fsync'... Ctrl+C to quit. [13:02:24] SYSCALL COUNT TIME (us) fsync 956 2448760.979 [13:02:25] SYSCALL COUNT TIME (us) fsync 979 2387591.025 [13:02:26] SYSCALL COUNT TIME (us) fsync 845 2488404.454
There were some leaks detected when running the test suite. But for `bcc_elf_get_buildid` which did not free the elf object, the rest of the leaks were isolated in the tests themselves which did not free some resources here and there.
This diff clears those leaks. This will allow running the tests suite in the future with LSAN enabled, helping in catching possible future leaks earlier.
Ran the sanitizer using:
```
docker run --privileged \
--pid=host \
-v $(pwd):/bcc \
-v /sys/kernel/debug:/sys/kernel/debug:rw \
-v /lib/modules:/lib/modules:ro \
-v /usr/src:/usr/src:ro \
-v /usr/include/linux:/usr/include/linux:ro \
bcc-docker \
/bin/bash -c \
'mkdir -p /bcc/build && cd /bcc/build && \
cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_LLVM_NATIVECODEGEN=OFF -DCMAKE_SANITIZE_TYPE=leak .. && make -j9'
```
followed by tests.
Before:
```
docker run -ti \
--privileged \
--network=host \
--pid=host \
-v $(pwd):/bcc \
-v /sys/kernel/debug:/sys/kernel/debug:rw \
-v /lib/modules:/lib/modules:ro \
-v /usr/src:/usr/src:ro \
-e CTEST_OUTPUT_ON_FAILURE=1 \
bcc-docker \
/bin/bash -c \
'/bcc/build/tests/wrapper.sh \
c_test_all sudo /bcc/build/tests/cc/test_libbcc' > /tmp/out
grep 'Indirect leak' /tmp/out | wc -l
99
grep 'Direct leak' /tmp/out | wc -l
4
```
Full out file available in https://gist.github.com/chantra/caa3c6f6a274895d8743fe9e48a7c528
After:
```
docker run -ti \
--privileged \
--network=host \
--pid=host \
-v $(pwd):/bcc \
-v /sys/kernel/debug:/sys/kernel/debug:rw \
-v /lib/modules:/lib/modules:ro \
-v /usr/src:/usr/src:ro \
-e CTEST_OUTPUT_ON_FAILURE=1 \
bcc-docker \
/bin/bash -c \
'/bcc/build/tests/wrapper.sh \
c_test_all sudo /bcc/build/tests/cc/test_libbcc'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_libbcc is a Catch v1.4.0 host application.
Run with -? for options
-------------------------------------------------------------------------------
searching for modules in /proc/[pid]/maps
-------------------------------------------------------------------------------
/bcc/tests/cc/test_c_api.cc:497
...............................................................................
/bcc/tests/cc/test_c_api.cc:499: FAILED:
REQUIRE( dummy_maps != __null )
with expansion:
NULL != 0
-------------------------------------------------------------------------------
test bpf table
-------------------------------------------------------------------------------
/bcc/tests/cc/test_bpf_table.cc:24
...............................................................................
/bcc/tests/cc/test_bpf_table.cc:24: FAILED:
{Unknown expression after the reported line}
due to unexpected exception with message:
bad_function_call
-------------------------------------------------------------------------------
test bpf percpu tables
-------------------------------------------------------------------------------
/bcc/tests/cc/test_bpf_table.cc:94
...............................................................................
/bcc/tests/cc/test_bpf_table.cc:94: FAILED:
{Unknown expression after the reported line}
due to unexpected exception with message:
bad_function_call
-------------------------------------------------------------------------------
test bpf stack_id table
-------------------------------------------------------------------------------
/bcc/tests/cc/test_bpf_table.cc:227
...............................................................................
/bcc/tests/cc/test_bpf_table.cc:268: FAILED:
REQUIRE( addrs.size() > 0 )
with expansion:
0 > 0
Parse error:
4@i%ra+1r
-------^
===============================================================================
test cases: 51 | 47 passed | 1 failed | 3 failed as expected
assertions: 984 | 980 passed | 1 failed | 3 failed as expected
Failed
```
SIGIOT is an alias of SIGABRT so it's assigned to the same number.
However it caused an error in my build setup like below:
libbpf-tools/sigsnoop.c:40:8: error: initializer overrides prior
initialization of this subobject
[-Werror,-Winitializer-overrides]
[6] = "SIGIOT",
^~~~~~~~
libbpf-tools/sigsnoop.c:39:8: note: previous initialization is here
[6] = "SIGABRT",
^~~~~~~~~
1 error generated.
Anyway, it's gonna show only single entry. So let's remove the other.
In recent kernels, i.e. since commit 322cbb50de71 ("block: remove
genhd.h"), genhd.h header has been removed and its content moved to
blkdev.h. Since genhd.h has been included in blkdev.h since forever,
including blkdev instead of genhd in the mdflush tool works for both
older and newer kernel.
We need bpftool for skeleton generation only, let's build and use bootstrap
bpftool like libbpf-bootstrap does ([0]). This avoids the following errors
on old kernels:
skeleton/pid_iter.bpf.c:35:10: error: incomplete definition of type 'struct bpf_link'
return BPF_CORE_READ((struct bpf_link *)ent, id);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0]: libbpf/libbpf-bootstrap#92
Signed-off-by: Hengqi Chen <[email protected]>
libbpf-tools: Build and use lightweight bootstrap version of bpftool
tools/mdflush: include blkdev.h instead of genhd.h
libbpf-tools: tcpconnect: take source port into consideration
Extension summary logic seems a bit redundant, try to simplify it (total already be calculated by FACTOR replacement).
Signed-off-by: Hengqi Chen <[email protected]>
With #4131 included, running the tool with -L reports the following warning:
/virtual/main.c:57:28: warning: multi-character character constant [-Wmultichar]
pid_key_t key = {.id = ' + pid + ', .slot = bpf_log2l(delta)};
^
/virtual/main.c:57:28: warning: character constant too long for its type
2 warnings generated.
The `pid` part should not be treated as string literal. Fix it.
Signed-off-by: Hengqi Chen <[email protected]>
Now usdt tracing has been supported in libbpf, so we add the javagc monitor as an example with this feature. Normally, you can use the command "readelf -n binary" to find the usdt in the binary. The javagc tracing result like this: Tracing javagc time... Hit Ctrl-C to end. TIME CPU PID GC TIME 21:33:42 0 90984 1662 21:33:52 0 90984 1303 21:33:59 0 90984 1101 21:33:59 0 90984 1425 21:34:11 0 90984 1015 Signed-off-by: chentao.ct <[email protected]>
tools/cpudist: Fix warning introduced by recent change
oomkill: Remove trailing newline from output
With llvm15, bcc failed the compilation with the following errors:
[100%] Building CXX object tests/cc/CMakeFiles/test_libbcc.dir/test_shared_table.cc.o
/home/yhs/work/llvm-project/llvm/build/install/lib/libclangSema.a(SemaRISCVVectorLookup.cpp.o):
In function `(anonymous namespace)::RISCVIntrinsicManagerImpl::InitIntrinsicList()':
SemaRISCVVectorLookup.cpp:
(.text._ZN12_GLOBAL__N_125RISCVIntrinsicManagerImpl17InitIntrinsicListEv+0x14b):
undefined reference to `clang::RISCV::RVVIntrinsic::computeBuiltinTypes(
llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>, bool, bool, bool, unsigned int)'
SemaRISCVVectorLookup.cpp:(.text._ZN12_GLOBAL__N_125RISCVIntrinsicManagerImpl17InitIntrinsicListEv+0x182):
undefined reference to `clang::RISCV::RVVIntrinsic::computeBuiltinTypes(
llvm::ArrayRef<clang::RISCV::PrototypeDescriptor>, bool, bool, bool, unsigned int)'
...
make[1]: *** [CMakeFiles/Makefile2:1110: examples/cpp/CMakeFiles/CGroupTest.dir/all] Error 2
...
The failure is due to llvm upstream patch https://reviews.llvm.org/D111617
which introduced another dependency on libclangSupport.a for bcc.
To fix the issue, I added libclangSupport in cmake file.
Signed-off-by: Yonghong Song <[email protected]>
LLVM16 patch llvm/llvm-project@b4e9977 caused bcc build failure like below: from /.../bcc/src/cc/frontends/clang/b_frontend_action.cc:23: /.../llvm-project/llvm/build/install/include/llvm/ADT/StringRef.h:96:54: error: expected ‘)’ before ‘Str’ /*implicit*/ constexpr StringRef(std::string_view Str) ~ ^~~~ ) /.../llvm-project/llvm/build/install/include/llvm/ADT/StringRef.h:239:14: error: expected type-specifier operator std::string_view() const { ^~~ LLVM build itself now is done with c++17. Let us also compile with c++17 if bcc is built with llvm16. Signed-off-by: Yonghong Song <[email protected]>
Sync with latest libbpf repo with top commit: 0667206913b3 Use checkout action in version v3 Signed-off-by: Yonghong Song <[email protected]>
* Support for kernel up to 5.19 * bcc tool updates for oomkill.py, biolatpcts.py, sslsniff.py, tcpaccept.py, etc. * libbpf tool updates for klockstat, opensnoop, tcpconnect, etc. * new bcc tools: tcpcong * new libbpf tools: tcpsynbl, mdflush, oomkill, sigsnoop * usdt: support xmm registers as args for x64 * bpftool as a submodule now * remove uses of libbpf deprecated APIs * use new llvm pass manager * support cgroup filtering libbpf tools * fix shared lib module offset <-> global addr conversion * riscv support * LoongArch support * doc update, bug fixes and other tools improvement Signed-off-by: Yonghong Song <[email protected]>
In #4126 I solved one problem but essentially pushed it somewhere else.
Now the problem is that when multiple arguments are passed, they all end up
being within the same argument because they get wrapped within the double-quotes.
This is pretty much an escape-hell as we keep on passing the same arguments over
and over through functions and then within `bash -c`.
The reason for the `bash -c` bit is that it allows us to set some envirtonment variable used in the tests.
Those env var are set to the local environment values.
Instead of going an extra layer of indirection, we can tell `sudo` to preserve those
specific env var by using the `--preserve-env` argument this way, we do not have to
re-escape the arguments that are passed within the `bash -c` quoted arg.
Tests:
Confirm that this was not working before:
```
$ docker run -ti \
--privileged \
--network=host \
--pid=host \
-v $(pwd):/bcc \
-v /sys/kernel/debug:/sys/kernel/debug:rw \
-v /lib/modules:/lib/modules:ro \
-v /usr/src:/usr/src:ro \
-e CTEST_OUTPUT_ON_FAILURE=1 \
bcc-docker-focal \
/bin/bash -c \
'/bcc/build/tests/wrapper.sh \
c_test_all sudo /bcc/build/tests/cc/test_libbcc -s "test probing running Ruby*"'
test probing running Ruby*": -c: line 0: unexpected EOF while looking for matching `"'
test probing running Ruby*": -c: line 1: syntax error: unexpected end of file
Failed
```
and is fixed after the patch:
```
$ docker run -ti \
--privileged \
--network=host \
--pid=host \
-v $(pwd):/bcc \
-v /sys/kernel/debug:/sys/kernel/debug:rw \
-v /lib/modules:/lib/modules:ro \
-v /usr/src:/usr/src:ro \
-e CTEST_OUTPUT_ON_FAILURE=1 \
bcc-docker-focal \
/bin/bash -c \
'/bcc/build/tests/wrapper.sh \
c_test_all sudo /bcc/build/tests/cc/test_libbcc -s "test probing running Ruby*"'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_libbcc is a Catch v1.4.0 host application.
Run with -? for options
-------------------------------------------------------------------------------
test probing running Ruby process in namespaces
in separate mount namespace
-------------------------------------------------------------------------------
/bcc/tests/cc/test_usdt_probes.cc:351
...............................................................................
/bcc/tests/cc/test_usdt_probes.cc:367:
PASSED:
REQUIRE( res.msg() == "" )
with expansion:
"" == ""
/bcc/tests/cc/test_usdt_probes.cc:368:
PASSED:
REQUIRE( res.ok() )
with expansion:
true
/bcc/tests/cc/test_usdt_probes.cc:371:
PASSED:
REQUIRE( res.ok() )
with expansion:
true
/bcc/tests/cc/test_usdt_probes.cc:374:
PASSED:
REQUIRE( res.ok() )
with expansion:
true
-------------------------------------------------------------------------------
test probing running Ruby process in namespaces
in separate mount namespace and separate PID namespace
-------------------------------------------------------------------------------
/bcc/tests/cc/test_usdt_probes.cc:351
...............................................................................
/bcc/tests/cc/test_usdt_probes.cc:393:
PASSED:
REQUIRE( res.msg() == "" )
with expansion:
"" == ""
/bcc/tests/cc/test_usdt_probes.cc:394:
PASSED:
REQUIRE( res.ok() )
with expansion:
true
/bcc/tests/cc/test_usdt_probes.cc:397:
PASSED:
REQUIRE( res.ok() )
with expansion:
true
/bcc/tests/cc/test_usdt_probes.cc:400:
PASSED:
REQUIRE( res.ok() )
with expansion:
true
/bcc/tests/cc/test_usdt_probes.cc:405:
PASSED:
REQUIRE( bcc_resolve_symname(module.c_str(), "rb_gc_mark", 0x0, ruby_pid, nullptr, &sym) == 0 )
with expansion:
0 == 0
/bcc/tests/cc/test_usdt_probes.cc:406:
PASSED:
REQUIRE( std::string(sym.module).find(pid_root, 1) == std::string::npos )
with expansion:
18446744073709551615 (0xffffffffffffffff)
==
18446744073709551615 (0xffffffffffffffff)
===============================================================================
All tests passed (10 assertions in 1 test case)
```
Just compile ruby manually similarly to the Ubuntu test image. Systemtap devel header is needed for sdt.h, so that ruby can be compiled with USDTs. Signed-off-by: Dave Marchevsky <[email protected]>
Add systemtap header and ruby to fedora docker
Those tests have started to fail since kernel 5.15. The restriction was lifted in https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=0c48eefae712c2fd91480346a07a1a9cd0f9470b This diff makes the expected returned value to the call to `update_value` conditional on the kernel version. Tested on 5.15 (using a Ubuntu 22.04 host), which is representative of the kernel running in GH CI. Also tested on Ubuntu 20.04 stock kernel: ``` $ docker run -ti \ --privileged \ --network=host \ --pid=host \ -v $(pwd):/bcc \ -v /sys/kernel/debug:/sys/kernel/debug:rw \ -v /lib/modules:/lib/modules:ro \ -v /usr/src:/usr/src:ro \ -e CTEST_OUTPUT_ON_FAILURE=1 \ bcc-docker-focal \ /bin/bash -c \ '/bcc/build/tests/wrapper.sh \ c_test_all sudo /bcc/build/tests/cc/test_libbcc "test sock*"' =============================================================================== All tests passed (8 assertions in 2 test cases) [22:40:55] chantra@focal:bcc git:(fix_sock_map_tests*) $ uname -a Linux focal 5.4.0-122-generic #138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux ```
We run tests within a container that may run on a kernel which is different than the headers we have in /usr/include/linux . This causes problem when we check for kernel version at compile as we use a version.h from another kernel. This change attempts to discover linux/version.h from installed linux-headers.
Currently softirq only report the time but not the event counts. If we have both time and counts it will be better understand the frequency and duration of events. Signed-off-by: Hailong Liu <[email protected]>
Since kernel commit fcb14cb1bdac("new iov_iter flavour - ITER_UBUF"),
tty_write() will use ITER_UBUF. And introduce iov_iter->ubuf field, so
we should use CO-RE way.
Signed-off-by: Rong Tao <[email protected]>
List all the kernel configs required for eBPF with its functionalities and description.
…4442) Truncate long Uprobe name and append hash.
add the config for kprobe multi in docs/kernel_config.md
ENV: LLVM 13.0.1, Kernel 5.15.67, aarch64
Overview of the error:
The verifier is unhappy, if '(r10 -32)' is not initialized, see also [0].
$ sudo ./ttysnoop.py 10
bpf: Failed to load program: Permission denied
3: (7b) *(u64 *)(r10 -8) = r6
4: (7b) *(u64 *)(r10 -16) = r6
5: (7b) *(u64 *)(r10 -24) = r6
6: (bf) r1 = r10
...
91: (67) r0 <<= 32
92: (77) r0 >>= 32
; if (bpf_probe_read_user(&data->buf, BUFSIZE, (void *)buf))
93: (55) if r0 != 0x0 goto pc+356
R0_w=inv0 R6=invP0 R7=map_value(id=0,off=0,ks=4,vs=260,imm=0) R8=map_value(
id=0,off=4,ks=4,vs=260,imm=0) R9=inv256 R10=fp0 fp-8=mmmmmmmm
fp-16=mmmmmmmm fp-24=mmmmmmmm
94: (79) r2 = *(u64 *)(r10 -32)
invalid read from stack R10 off=-32 size=8
processed 593 insns (limit 1000000) max_states_per_insn 1 total_states 48
peak_states 48 mark_read 3
This issue can also be resolved by upgrading LLVM>=14 and recompile and install
bcc.
[0] iovisor/bcc#2623
Signed-off-by: Rong Tao <[email protected]>
The type of value in wakeuptime.c defined as int, it may be overflow and output as a negative numbers. Actually the type of value in map count is u64 defined at wakeuptime.bpf.c. So change the int to __u64 to fix it. Signed-off-by: Hailong Liu <[email protected]>
Add kprobe and use fentry_can_attach() to decide whether to use fentry or kprobe. Signed-off-by: Rong Tao <[email protected]>
The nfs4_file_open tracepoints are part of nfsv4.ko, which may not be loaded on systems using NFSv3 only. Do not attempt to attach probes in this case. For nfsdist.py, a similar issue has already been addressed likewise in commit a433ef9. Signed-off-by: Daniel Kobras <[email protected]>
…he_readahead
since commit 56a4d67c264e("mm/readahead: Switch to page_cache_ra_order") switch
do_page_cache_ra() to page_cache_ra_order() (v5.17), and commit bb3c579e25e5
("mm/filemap: Add filemap_alloc_folio") swap __page_cache_alloc() to
filemap_alloc_folio() (since v5.15)
Reprocude the error(fedora37, 6.1.7-200.fc37.aarch64):
$ sudo ./readahead.py
cannot attach kprobe, probe entry may not exist
Traceback (most recent call last):
File "/home/rongtao/Git/bcc/tools/./readahead.py", line 159, in <module>
b.attach_kprobe(event=ra_event, fn_name="entry__do_page_cache_readahead")
File "/usr/lib/python3.11/site-packages/bcc/__init__.py", line 840, in attach_kprobe
raise Exception("Failed to attach BPF program %s to kprobe %s" %
Exception: Failed to attach BPF program b'entry__do_page_cache_readahead' to kprobe b'do_page_cache_ra'
Signed-off-by: Rong Tao <[email protected]>
Add kprobe and use fentry_can_attach() to decide whether to use fentry or kprobe.
logs:
$ sudo ./numamove
libbpf: prog 'fentry_migrate_misplaced_page': failed to attach: ERROR: strerror_r(-524)=22
libbpf: prog 'fentry_migrate_misplaced_page': failed to auto-attach: -524
failed to attach BPF programs
Signed-off-by: Rong Tao <[email protected]>
…lab'
kernel commit 40f3bf0cb04c("mm: Convert struct page to struct slab in functions
used by other subsystems") introduce slab_address() function, commit 6e48a966dfd1
("mm/kasan: Convert to struct folio and struct slab") linux/kasan.h adds a
dependency on the slab struct, This leads to the following problems:
$ sudo ./slabratetop.py
In file included from /virtual/main.c:13:
include/linux/slub_def.h:162:26: warning: call to undeclared function 'slab_address';
ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
void *object = x - (x - slab_address(slab)) % cache->size;
^
include/linux/slub_def.h:162:46: error: invalid operands to binary expression ('void *' and 'unsigned int')
void *object = x - (x - slab_address(slab)) % cache->size;
~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~
include/linux/slub_def.h:164:8: error: incomplete definition of type 'struct slab'
(slab->objects - 1) * cache->size;
~~~~^
include/linux/kasan.h:13:8: note: forward declaration of 'struct slab'
struct slab;
^
...
At first, I wanted to fix this with a kernel patch [1], however, bcc as a
downstream project of the kernel, this issue should be solved inside the bcc
project. This is agreed by kernel maintainer and bcc maintainer @yonghong-song.
This solution is provided by @yonghong-song [0].
[0] iovisor/bcc#4438
[1] https://lore.kernel.org/all/[email protected]/
Signed-off-by: Rong Tao <[email protected]>
Signed-off-by: Yonghong Song <[email protected]>
Installed static libraries, namely libbcc.a, libbcc_bpf.a and libbcc-loader-static.a, do not contain all the symbols that are needed for linking BCC statically into one's project. The reason for this issue is that in CMake, when linking static library against another library (by using the target_link_libraries statement), the effect is not a combined static library. CMake will only record a dependency from the target library to the source library. This is why when bcc-static is linked against api-static, the bcc-static library (libbcc.a) does not contain symbols from api-static, making it necessary to install api-static to <prefix>/lib along other library files. Ditto for clang_frontend. This is not the case when building shared libraries. This changeset fixes the issue by using a CMake feature named "object libraries" (https://cmake.org/cmake/help/v3.25/command/add_library.html#object-libraries). Intermediate targets, such as api-static and clang_frontend, are built as object libraries. A feature that is only available since CMake 3.12 is linking such libraries using the target_link_library statement. Until the project transitions to requiring CMake version 3.12 or higher, we have to "link" object libraries using the target_sources statement instead.
clang15 prevents implicit type conversion of void * to u64 fix #4467 Signed-off-by: Y7n05h <[email protected]>
kernel commit abd4349ff9b8("mm: compaction: cleanup the compaction trace
events") change the arguments of 'mm_compaction_begin' from (start_pfn,
migrate_pfn, free_pfn, end_pfn, sync) to (cc, start_pfn, end_pfn, sync),
and change the arguments of 'mm_compaction_end' from (start_pfn,
migrate_pfn, free_pfn, end_pfn, sync, ret) to (cc, start_pfn, end_pfn,
sync, ret).
Replacing RAW_TRACEPOINT_PROBE with TRACEPOINT_PROBE solves this problem
and guarantees compatibility.
$ sudo ./compactsnoop.py
bpf_attach_raw_tracepoint (mm_compaction_begin): Invalid argument
Traceback (most recent call last):
File "/home/sdb/Git/bcc/tools/./compactsnoop.py", line 292, in <module>
b = BPF(text=bpf_text)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/bcc/__init__.py", line 483, in __init__
self._trace_autoload()
File "/usr/lib/python3.11/site-packages/bcc/__init__.py", line 1462, in _trace_autoload
self.attach_raw_tracepoint(tp=tp, fn_name=fn.name)
File "/usr/lib/python3.11/site-packages/bcc/__init__.py", line 1055, in attach_raw_tracepoint
raise Exception("Failed to attach BPF to raw tracepoint")
Exception: Failed to attach BPF to raw tracepoint
Signed-off-by: Rong Tao <[email protected]>
The verifier is unhappy, if data struct _pad_ is not initialized, see [0][1].
$ sudo ./nfsslower.py
...
; bpf_perf_event_output(ctx, (void *)bpf_pseudo_fd(1, -2), CUR_CPU_IDENTIFIER, &data, sizeof(data));
83: (79) r1 = *(u64 *)(r10 -144) ; R1_w=ctx(off=0,imm=0) R10=fp0
84: (18) r3 = 0xffffffff ; R3_w=4294967295
86: (b7) r5 = 96 ; R5_w=96
87: (85) call bpf_perf_event_output#25
invalid indirect read from stack R4 off -136+92 size 96
processed 84 insns (limit 1000000) max_states_per_insn 0 total_states 4 peak_states 4 mark_read 4
...
raise Exception("Failed to load BPF program %s: %s" %
Exception: Failed to load BPF program b'raw_tracepoint__nfs_commit_done': Permission denied
[0] iovisor/bcc#2623
[1] iovisor/bcc#4453
Signed-off-by: Rong Tao <[email protected]>
This commit adds support for reading contents of zip files. Only basic functionality is provided (enough to support Android usecase). In particular no zip64 or encryption is supported.
This commit enables code operating on elf files in bcc_elf.c to open elf files stored without compression in a zip archive. Elf files stored in zip archives are refered to by a special path consisting of the archive path followed by "!/" and name of the entry within the archive. This is the convention used by Android.
This commit recognizes *.zip and *.apk files mmapped with executable
flag set and changes name of associated module to the corresponding
zipped entry path following the "{zip path}!/{zip entry name}" format.
File offset of the module is updated as well to reflect offset within
the zipped file data instead of the offset within the archive.
With this commit, when bpf_attach_uprobe receives a non-existing binary
path it will try to treat it as a path of a zip entry in the
"{zip archive}!/{zip entry name}" format. If such archive and entry
exist then the binary path is replaced with archive path and the offset
within the archive entry is translated to offset within the zip archive.
Implement symbol lookup in zipped libraries.
tcp_mon_block uses netlink TC, kernel tracepoints and kprobes to monitor outgoing connections from given PIDs (usually HTTP web servers) and block connections to all addresses initiated from them (acting like an in-process firewall), unless they are listed in allow_list. This way the process can mitigate connections to unexpected malicious addresses, like to a C2 server, happening after a successful exploitation of the process Signed-off-by: Dor A <[email protected] >
security_inode_create presents only if CONFIG_SECURITY is on. Do not attach to it unconditionally. Signed-off-by: Hengqi Chen <[email protected]>
default c++ standard was bumped to c++17 when using clang16+ compiler via 907b89c, however, tests which use check tool is not yet ported to work with c++17 standard, therefore use c++14 on tests Signed-off-by: Khem Raj <[email protected]> Cc: Hengqi Chen <[email protected]>
prakashsurya
force-pushed
the
dlpx/pr/prakashsurya/f0f0cfcc-e3ce-41e7-b034-87aa8b12cf74
branch
from
ae73e3c to
09d5785
Compare
delphix-devops-bot
force-pushed
the
develop
branch
from
7aa2829 to
ef9a7c7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.