Decompilation of Virtual-Machine-Obfuscated Binaries with Binary Ninja
Decompilation of Virtual-Machine-Obfuscated Binaries with Binary Ninja
Supervisor(s): | Fabian Kilger |
Status: | finished |
Topic: | Others |
Author: | Markus Colombo |
Submission: | 2025-09-01 |
Type of Thesis: | Bachelorthesis |
DescriptionVirtualization-based obfuscation is common in malware, as it has proven to be one of the most effective techniques in hindering analysis. Here, malware is not compiled to machine code, but instead translated to custom instruction sets interpreted by virtual machines embedded in the executable. Prior work has been developed to reverse this obfuscation by extend- ing the Ghidra decompiler. While this proved effective, Ghidra does not allow for deep integration into its disassembly and decompilation phases, which led to missed opportunities and some unwanted arti- facts in the decompiled code. Binary Ninja forms a promising alternative, as it aims to be a fully extensible binary analysis platform. We adapt the aforementioned prior work and making use of Binary Ninja’s unique capabilities in the process, like the ability to vary instruction semantics based on the surrounding context. We evaluate our prototype implementation on 288 samples generated by obfuscating 48 programs with six configurations of Tigress each. We find that, our proof of concept is effective in accurately decompil- ing hash functions, but only fully preserves the semantics of 36% of basic algorithms tested. While the prototype is not yet viable for real world applications, our results demonstrate potential and enables future research in the area. |