04: Chapter 2 | Windows OS System Calls

What is a System call?

Before we discuss what a direct or indirect system call is and how it is used by attackers (red team), it is important to clarify what a system call or syscall is in general. Technically, the syscall instruction itself is part of the syscall stub within a native API or native function. The syscall stub of a native API is a set of assembly instructions and allows the temporary transition (CPU switch) from user mode to kernel mode after execution. If we look at the syscall stubs of different native APIs, we would see that only the syscall number or System Service Number (SSN), which is moved into the eax register, differs between them. The syscall is thus the interface between a user-mode process and the task to be executed in the Windows kernel. We could also say that system calls provide the bridge from user mode to kernel mode.

What are some key facts about the syscall stub from native functions?

• Each native function contains a specific syscall ID or System Service Number (SSN).
• Only the SSN differs from native function to native function, the rest of the syscall stub structure is always the same.
• Syscall IDs can change from Windows to Windows and from version to version.
• Important, the syscall instruction is a separate instruction and not the syscall ID itself.
• The syscall ID or more precisely the opcode mov in the codeline mov eax SSN can be hooked by an EDR, but the syscall instruction syscall itself can't be hooked by an EDR (important later for indirect syscalls).

In the following screenshot we can see that the syscall ID 18 is related to the native API (NTAPI) NtAllocateVirtualMemory, but as already mentioned, the System Service Number (SSN) can change. Additional information, these alterations occur not solely because Microsoft intermittently adds or removes system services, but also due to the frequent randomization and reshuffling of the table, a strategy aimed at thwarting attacks that rely on hardcoded system call numbers (Windows Internals 7th, Part 2, S. 96).

Why do we need system calls at all?

Because a modern operating system like Windows 10 is divided into user mode and kernel mode, syscalls are necessary or responsible for initialising the transition from user mode to kernel mode. For example, system calls in Windows are needed to:

• Access hardware such as scanners and printers
• Network connections to send and receive packets
• Reading and writing files

As a practical example, in the context of writing a file to disk, if a usermode process such as notepad.exe wants to save content to disk in the form of a file, the process needs temporary "access" to kernel mode. Why is this necessary? Because the components that need to be accessed or that need to perform the task in kernel mode, such as the file system and the appropriate device drivers, are located in the Windows kernel. The following figure shows the principle and interaction between notepad.exe -> kernel32.dll -> kernelbase.dll -> ntdll.dll -> syscalls to write a file to disk.

In order for the save operation to be performed in the context of the user mode process notepad.exe, in the first step it accesses kernel32.dll and calls the Windows or Win32 API WriteFile. In the second step, kernel32.dll accesses kernelbase.dll in the context of the same Windows API. In the third step, the Windows API or Win32 API WriteFile calls the corresponding Native API NtCreateFile via ntdll.dll. The Native API contains all the necessary technical assembly instructions in form of the syscall stub (SSN, syscall, etc.) and enables the temporary transition (CPU switch) from user mode (ring 3) to kernel mode (ring 0) after execution. We also could say, that the Windows or Win32 APIs are used as wrapper functions to access the Native APIs in ntdll.dll.

Next the system service dispatcher aka KiSystemCall/KiSystemCall64 in the Windows kernel is called, which is responsible for querying the System Service Descriptor Table (SSDT) for the appropriate function code based on the executed system call ID (SSN or current index number in the eax register). Once the system service dispatcher and the SSDT have worked together to identify the function code for the system call in question, the task is executed in the Windows kernel.

It is important to note that ntdll.dll is not the only module in Windows user mode that is used to call native functions, furthermore to execute the appropriate system call. To interact with the Windows GUI Windows APIs (wrapper functions) in e.g. user32.dll or gdi32.dll are used to access or execute the correlated native functions in win32u.dll. Why is this important, because later we will see that EDRs set their hooks not only in ntdll.dll, but depending on the EDR they set (different) hooks in different modules (DLLs) in user mode.

If we use x64dbg to look at the native functions imported from ntdll.dll or win32u.dll, we will see that Nt* and Zw* functions are found in ntdll.dll and NtUser* and NtGdi* functions are found in win32u.dll. Just to check, by comparing the syscall stub of the native functions from ntdll.dll and win32u.dll, we can see that, as expected, the syscall stub is exactly the same, only the SSN differs from function to function.

In Summary and simple terms, system calls are needed in Windows to perform the (temporary) transition (CPU switch) from user mode to kernel mode, or to execute tasks initiated in user mode that require temporary access to kernel mode - such as saving files - as a task in kernel mode. So far, this knowledge should provide a basic understanding of sysall on Windows to better understand the concept of direct and indirect syscalls later on. If you want to dig deeper into the details of how syscalls work on Windows, I highly recommend reading Windows Internals 7th Edition Part 2, the "System Service Handling" section in Chapter 8 "System Mechanisms". I also highly recommend taking a look at the great blog post "A Syscall Journey in the Windows Kernel" by Alice Climent-Pommeret.

Summary: Windows OS System Calls

• System calls are executed by accessing native functions in ntdll.dll or win32u.dll.
• System calls are the bridge between user mode and kernel mode
• System calls allow a temporary transition to kernel mode.
• System calls are part of the syscall stub, which is part of the native function.
• The (x64) syscall stub structure always looks the same, only the system service number (SSN), which is placed in eax to prepare the SSN for execution, is unique.
• SSNs change from Windows to Windows or from version to version.
• The system service dispatcher and the system service dispatch table work together in kernel mode to identify which native function Nt* or Zw* should be executed based on the SSN.