TUM Logo

Specializing Operating System Kernels at Runtime

Specializing Operating System Kernels at Runtime

Supervisor(s): Marius Momeu
Status: finished
Topic: Others
Author: Johannes Wiesböck
Submission: 2021-12-15
Type of Thesis: Masterthesis

Description

Operating system kernels suffer from increasing feature bloat. This leads to steadily
growing code bases, which are a potential threat to security. Especially code belonging
to legacy features that are not frequently maintained can include bugs causing vulner-
abilities. It has been shown that vulnerable code from rarely used features can open
vulnerabilities enabling real-world attacks. Moreover, unused code provides a viable
source of code gadgets for CRAs. Contradictory to the feature bloat offered by many
operating systems, most applications only utilize a small subset of the provided features.
Still, all system calls and their implementations remain accessible, regardless of the
application’s actual requirements. Removing unused code from the kernel reduces the
amount of exposed vulnerabilities and gadgets, thus strengthening system security.
Various methods to reduce bloat in operating systems exist. Configuration-driven
debloating techniques tailor kernel build configurations to a particular target workflow.
This allows removing many features but renders them incompatible with multi purpose
systems, running many different programs. Dynamic debloating approaches keep
compatibility with all applications but suffer from high performance overhead and large
memory footprints. Furthermore, many solutions rely on a hypervisor component,
which narrows their applicability down to virtualized environments.
In this thesis, we present a novel mechanism creating specialized code segments for
the operating system kernel at runtime. With help of FG-KASLR, we assemble debloated
code segments, reduced to the features required by a particular program. The set of
required functions is obtained through static analysis of the kernel’s CFG and the set of
system calls used by the application. Only functions necessary for the used system calls
are included in the specialized code segment. Leveraging FG-KASLR’s ability to reorder
kernel functions, we randomize the function order in each specialized code segment.
Code segments are directly mapped into the virtual address space of their application,
enabling efficient kernel switches between processes without virtualization support.
The evaluation of our prototype on the Linux kernel shows that we can reduce the
number of gadgets valuable to code-reuse attacks by over 87 %. While performance
measurements indicate a particularly high influence on context switch latency, marco-
benchmarks show that the average performance overhead for a typical web server
workload is below 30 % which is on par with existing solutions.