TUM Logo

Black-Box Reverse Engineering and Feature Comparison of AV Engines

Black-Box Reverse Engineering and Feature Comparison of AV Engines

Supervisor(s): Fabian Franzen
Status: finished
Topic: Others
Author: Arxhend Zariqi
Submission: 2021-08-16
Type of Thesis: Bachelorthesis

Description

Current anti-virus (AV) engines targeting the consumer segment employ a variety of
techniques in order to reliably detect not only known, but also previously unknown
malicious software. The underlying details and inner-workings of AV engines are
typically not made public. This makes it difficult to determine which specific
techniques, functional features, and capabilities a given AV engine has to offer
to identify malware, especially if it is packed, encrypted, or otherwise obfuscated.
In this thesis, we investigate the extent to which code emulation, static unpacking,
and signature-based analysis and detection techniques are employed by current AV engines.
Furthermore, we research the feature of code emulation more closely by examining the
engines' emulation environments. We achieved the mentioned objectives by applying black-box
reverse engineering methods to eleven AV programs. Akin to existing work, we conducted
our research by providing crafted droppers and malware samples to the individual AV engines.
Observing the engines' behaviour, such as the detection results, allowed us to reason
about their supported features and capabilities. Furthermore, we present a new side-channel
to leak emulator fingerprints. Our results indicate that neither code emulation nor static
unpacking have been universally adopted, despite their effectiveness. While implementation
specifics thereof greatly vary from engine to engine, we also have strong evidence that
engines and signature sets are shared between vendors. Furthermore, we conclude that the
current implementation of used emulators can be improved in order to reduce the attack
surface for split-personality malware.