StateForge - State-Aware Constraint Solving for Kernel Fuzzing via Hybrid Analysis

StateForge - State-Aware Constraint Solving for Kernel Fuzzing via Hybrid Analysis

Supervisor(s): Marius Momeu Manuel Andreas
Status: finished
Topic: Others
Author: Manuel Probst
Submission: 2026-03-23
Type of Thesis: Masterthesis

Description

Kernels are the central and most security-critical component of an operating system,
yet they expose a large attack surface through the system call interface. Vulnerabilities
in this interface can allow attackers to escalate privileges and compromise the entire
system. Fuzzing is a commonly used technique to find and mitigate such vulnerabili-
ties. However, fuzzing kernels comes with the challenge that their behavior not only
deterministically relies on direct inputs but also on their internal state. Oftentimes,
current approaches are unable to explore certain code parts that are hidden behind
state-dependent constraints that involve the interactions between multiple system calls.
We address these limitations by proposing StateForge, a state-aware constraint solver
for kernel fuzzing using hybrid analysis. StateForge is able to solve constraints in the
kernel that normally are considered fuzzing roadblocks due to their state-dependence
by analyzing fuzzing test cases across system call boundaries and with consideration of
the kernel state. For this, it uses a combination of symbolic execution and static pointer
analysis to identify the direct and indirect dependencies of constraints and solves them
by adjusting the system calls’ parameters and steering the execution towards write
statements, setting the kernel state accordingly.
We implemented a prototype of our design targeting the Linux kernel and show
during our evaluation that StateForge is capable of improving the number of detected
constraint dependencies compared to previous approaches by a median of 95.89%, as
well as solving multiple state-dependent constraints that usually would be hard to
solve with common means.