TUM Logo

Lightweight Process Isolation for the x86 Architecture via Custom Hardware Extensions in the gem5 Simulator

Lightweight Process Isolation for the x86 Architecture via Custom Hardware Extensions in the gem5 Simulator

Supervisor(s): Marius Momeu
Status: finished
Topic: Others
Author: Minu Föger
Submission: 2024-03-15
Type of Thesis: Masterthesis

Description

Context switching, or the act of switching between processes, tends to be a heavyweight operation
in modern operating systems. A major reason for this is that it involves flushing the TLB, the
cache for virtual-to-physical address mappings in modern microarchitectures. After such a TLB
flush, the newly scheduled process starts with a "cold" TLB, which increases the number of TLB
misses, resulting in expensive page table walks during address translation. Many TLB flushes
performed during context switches are unnecessary and exist only to avoid TLB conflicts when
switching to a different virtual address space. This is a consequence of traditional process isolation
techniques that assign each process in the system a different virtual address space.
We explore a new approach to process isolation that utilizes custom hardware extensions for
the x86 microarchitecture. We aim to provide a more lightweight mechanism for process isolation
that reduces the amount of TLB flushes during context switching while maintaining the same
qualitative security guarantees of traditional process isolation. Our design is based on the SPT
Linux thesis, which follows a similar approach but uses existing x86 techniques such as Intel
MPK. The main idea is to place all processes into a single shared virtual address space so that
TLB entries of different processes do not conflict and unnecessary TLB flushes during context
switches can be avoided. To ensure process isolation within the shared address space, we propose
the new microarchitectural PISeg mechanism, which is inspired by x86 memory segmentation
and prevents memory accesses from exceeding the processes’ PISeg slot.
We implement the design prototype using the gem5 microarchitectural simulator and a modified
Linux kernel. In benchmarks we observe mixed results, indicating smaller increases and decreases
in performance depending on use case. We conclude that sharing a virtual address space alone
may not be sufficient to improve the performance of context switching, but we believe that further
research may lead to novel ways of optimizing process isolation performance.