syscall_intercept

A user space library for intercepting syscalls on the RISC-V architecture.

Dependencies

Build dependencies:

• RISC-V toolchain -- tested with recent versions of GCC and Clang
• CMake ≥ v3.13 -- build system
• Perl -- required for code style checks
• Pandoc -- required to generate the manual page

Runtime dependencies:

• Capstone ≥ v5 (v6 recommended) -- the disassembly engine used under the hood

How to Build

The RISC-V toolchain can be built from the RISC-V GNU Toolchain.
Capstone can be installed from Capstone Engine.

Building libsyscall_intercept requires CMake:

cmake path_to_syscall_intercept -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release
make

Alternatively, use the CMake CUI (GUI):

ccmake path_to_syscall_intercept
make

There is an install target. For now, all it does is cp:

make install

Coming soon:

make test

Synopsis

API

#include <libsyscall_intercept_hook_point.h>

int (*intercept_hook_point)(long syscall_number,
                            long arg0, long arg1,
                            long arg2, long arg3,
                            long arg4, long arg5,
                            long *result);
void (*intercept_hook_point_clone_child)(void);
void (*intercept_hook_point_clone_parent)(long pid);
struct wrapper_ret syscall_no_intercept(long syscall_number, ...);
int syscall_error_code(long result);
int syscall_hook_in_process_allowed(void);

Compile

$ cc -lsyscall_intercept -fpic -shared source.c -o preloadlib.so

Run with LD_PRELOAD

$ LD_PRELOAD=preloadlib.so ./application

Description

The syscall-intercepting library provides a low-level interface for hooking Linux system calls in user space. This is achieved by hotpatching the machine code of the standard C library (glibc) in the memory of a process. Using this library, almost any syscall can be intercepted in user space via the API specified in libsyscall_intercept_hook_point.h.

Using intercept_hook_point()

int (*intercept_hook_point)(long syscall_number,
                            long arg0, long arg1,
                            long arg2, long arg3,
                            long arg4, long arg5,
                            long *result);

• To enable syscall interception, assign a callback function to intercept_hook_point.
• A non-zero return value from the callback function indicates that the syscall should proceed as normal and will be passed to the kernel.
• A zero return value indicates that the user takes over the syscall with the result value stored via the *result parameter.

Clone hooks

In addition to hooking syscalls, the user can be notified of thread creations with post-clone hooks. These hooks are executed immediately after a thread is created with a clone syscall:

void (*intercept_hook_point_clone_child)(void);
void (*intercept_hook_point_clone_parent)(long pid);

Bypassing interception

The library provides this function to execute syscalls that bypass the interception mechanism:

struct wrapper_ret syscall_no_intercept(long syscall_number, ...);

Detecting error codes

With the syscall_error_code() function it's easy to detect syscall return values indicating errors:

int syscall_error_code(long result);

Checking interception status

This function is used to query if interception is disabled with the INTERCEPT_HOOK_CMDLINE_FILTER environment variable.

int syscall_hook_in_process_allowed(void);

Environment Variables

INTERCEPT_LOG -- When set, the library logs each intercepted syscall to a file. If the variable ends with "-", the filename is suffixed with the process ID. E.g., for a process with PID 123 and INTERCEPT_LOG set to "intercept.log-", the resulting log file would be "intercept.log-123".

INTERCEPT_LOG_TRUNC -- When set to 0, the log file specified by INTERCEPT_LOG is not truncated.

INTERCEPT_HOOK_CMDLINE_FILTER -- When set, the library checks the command line used to start the program. Hotpatching and syscall interception occur only if the last component of the command matches the string provided in this variable. The library also provides a function for querying this state.

INTERCEPT_ALL_OBJS -- When set, all libraries are patched, not just glibc and pthread. Note: The syscall_intercept library and Capstone are never patched.

INTERCEPT_NO_TRAMPOLINE -- When set, the trampoline is not used for jumping from the patched library to the syscall_intercept library. In the RISC-V version of this library, the trampoline size is less than 30 bytes, requiring only one page of memory when allocated with mmap(). Consequently, setting this variable does not significantly reduce memory usage.

INTERCEPT_DEBUG_DUMP -- Enables verbose output.

Example

#include <libsyscall_intercept_hook_point.h>
#include <syscall.h>
#include <errno.h>

static int
hook(long syscall_number,
			long arg0, long arg1,
			long arg2, long arg3,
			long arg4, long arg5,
			long *result)
{
	if (syscall_number == SYS_getdents) {
		/*
		 * Prevent the application from
		 * using the getdents syscall. From
		 * the point of view of the calling
		 * process, it is as if the kernel
		 * would return the ENOTSUP error
		 * code from the syscall.
		 */
		*result = -ENOTSUP;
		return 0;
	} else {
		/*
		 * Ignore any other syscalls
		 * i.e.: pass them on to the kernel
		 * as would normally happen.
		 */
		return 1;
	}
}

static __attribute__((constructor)) void
init(void)
{
	// Set up the callback function
	intercept_hook_point = hook;
}

Compile and run:

$ cc example.c -lsyscall_intercept -fpic -shared -o example.so
$ LD_LIBRARY_PATH=. LD_PRELOAD=example.so ls
ls: reading directory '.': Operation not supported

Functional Changes

The key differences in functionality between the RISC-V and x86_64 versions of this library relevant to the user:

1. syscall_no_intercept() returns a struct containing both A0 and A1, the two return values of a syscall, instead of returning only the primary value (RAX) on x86_64.
2. All threads (clones) are intercepted, and results are logged. On x86_64, the result of thread creation is not logged for threads with separate stack spaces.
3. Post-clone hooks, intercept_hook_point_clone_child() and intercept_hook_point_clone_parent(), are triggered by every thread creation, not just by the threads with separate stack spaces.

Under the Hood

Assumptions:

To handle syscalls in user space, the library relies on the following assumptions:

• All syscalls are issued via glibc.
• No other facility hotpatches glibc.
• Glibc is loaded before library initialization.
• The glibc machine code is suitable for patching methods used.
• For some more basic assumptions, see limitations.

Disassembly:

The library disassembles the text segment of glibc loaded into the memory space of the process in which it is initialized. It locates all syscall instructions and replaces each of them with a jump. This is in common with both x86_64 and RISC-V, but RISC-V differs in how patching is implemented.

Patching RISC-V

Reasons to change implementation logic:

1. The relative jump instruction (jal) has a ±1 MB reach which is not enough to jump out of glibc like x86's jmp with ±2 GB reach.
2. Instructions are naturally better aligned so nops are rarely used in glibc removing the possibility of having the nop trampolines.

Jumping out of glibc is only doable with an indirect jump, auipc + jalr = ±2 GB reach. That requires much more patching space than x86_64 jmp instruction with its 5 bytes for a 2 GB jump or 2 bytes for a 127 B jump (used in combination with the nop trampoline). RISC-V needs 16 B (when RVC is supported) to perform a 2 GB jump. Not all syscalls are surrounded by that many relocatable instructions. To be able to patch all the syscalls, three different patch types are created based on the space available for patching:

1. Gateway -- the syscall is surrounded by many relocatable instructions which makes it suitable to create a ±2 GB jump. Gateways are the cornerstone of the library because smaller patch types jump here to get "forwarded" to the syscall_intercept library.
2. Middle -- the syscall is surrounded by enough relocatable instructions to replace them with the store instruction that stores ra on the stack. Jumps using jal ra, GW_addr to the gateway, from where it jumps to the syscall_intercept library.
3. Small -- not enough space to store the register used by jal on the stack. The small type relies on static analysis during the disassembly phase to store the syscall number (A7 value) in the patch's struct. Like the middle type, the small patch jumps to the gateway using jal (jal a7, GW_addr).

The final destination for all patches is the same assembly routine (asm_entry_point) inside the syscall_intercept library where C functions get called like in the x86_64 counterpart library.

In action:

Hotpatching the gateway type:

Before:                           After:

b2d28 <__open>:                   b2d28 <__open>:
...                               ...
b2dac: ld      a1,8(sp)           b2dac: ld      a1,8(sp)
b2dae: ld      a3,0(sp)           b2dae: addi    sp, sp, -48    # GW start
b2db0: mv      a2,s0              b2db0: sd      ra, 0(sp)
b2db2: li      a7,56              b2db2: auipc   ra, offset
b2db6: li      a0,-100            b2db6: jalr    ra, offset(ra)
b2dba: ecall                      b2dba: ld      ra, 0(sp)
b2dbe: lui     a4,0xfffff         b2dbc: addi    sp, sp, 48     # GW end
...                               b2dbe: lui     a4,0xfffff
...                               ...

The destination of the gateway's jalr is either directly the syscall_intercept library (if INTERCEPT_NO_TRAMPOLINE=1) or a trampoline where an absolute jump is performed to the syscall_intercept library.
The gateway and middle types are patched similarly. The difference is that the middle type uses jal instead of auipc/jalr and spares 4 bytes of patching space. Type small is a bit less straightforward, check the documentation.

Limitations

• Only supports GNU/Linux.
• Tested only with glibc, compatibility with other libc implementations is unverified.
• Syscall rt_sigreturn is not intercepted as it's used by the kernel for signal handling.

Debugging

Prevent interception of syscalls within the debugger by setting the INTERCEPT_HOOK_CMDLINE_FILTER variable described above:

INTERCEPT_HOOK_CMDLINE_FILTER=ls LD_PRELOAD=libsyscall_intercept.so gdb ls

Alternatively, set LD_PRELOAD within the gdb session or configure it in a .gdbinit file:

set environment LD_PRELOAD libsyscall_intercept.so