• Skip navigation
  • Skip to navigation
  • Skip to the bottom
Simulate organization breadcrumb open Simulate organization breadcrumb close
IT Security Infrastructures Lab
  • FAUTo the central FAU website
  1. Friedrich-Alexander-Universität
  2. Faculty of Engineering
  3. Department Computer Science
  • Campo
  • UnivIS
  • Jobs
  • Map
  • Help
  1. Friedrich-Alexander-Universität
  2. Faculty of Engineering
  3. Department Computer Science

IT Security Infrastructures Lab

Navigation Navigation close
  • Research
    • Forensic Computing Group
    • Human Factors in Security and Privacy Group
    • Multimedia Security
    • Security Education Development Group
    • System Security Group
    • Archive
    • Funded Projects
    • Publications
    Portal Research
  • Lab
    • Staff & Research Groups
    • Alumni
    • Partners
    Portal Lab
  • Teaching
    • Courses
    • Hinweise zu den Lehrveranstaltungen
    • Notes on Examinations
    • eTeaching
    • Theses
    • Writing a Thesis at Informatik 1
    Portal Teaching
  • How to reach us
  1. Home
  2. Research
  3. System Security Group
  4. VMAttack: Deobfuscating Virtualization-Based Packed Binaries

VMAttack: Deobfuscating Virtualization-Based Packed Binaries

In page navigation: Research
  • Forensic Computing Group
    • DiOS: Dynamic Privacy Analysis of iOS Applications
    • Fingerprinting Mobile Devices Using Personalized Configurations
    • Selective Deletion
  • Human Factors in Security and Privacy Group
    • Antivirus Usability
    • Browser Fingerprinting
    • IoT Security Update Labels
    • Phishing Susceptibility
    • Security Experts
    • ZigBee Security Research
  • Multimedia Security
    • Image & Video Forensics
    • Image Analysis & Enhancement
    • X-ray Phase Contrast
    • Blog
    • Code and Data
      • Copy-Move Forgery Detectors and Ground Truth Generator
      • Image Manipulation Dataset
    • Colloquium
  • Security Education Development Group
    • Open C3S Overview
    • Open-C3S-Projektergebnisse
    • Ulix – a Literate OS
  • System Security Group
    • AppAuth: On App-based Matrix Code Authentication in Online Banking
    • AppTAN (In)Security: (In)Security of App-based TAN Methods in Online Banking
    • AVX Crypto: AVX Instructions to Accelerate Crypto Primitives
    • Bispe: A Bytecode Interpreter for Secure Program Execution in Untrusted Main Memory
    • Centroid
    • CPU-bound Encryption (TRESOR, TreVisor, ARMORED)
    • FROST: Forensic Recovery Of Scrambled Telephones
    • How Android’s UI Security is Undermined by Accessibility
    • HyperCrypt: Hypervisor-based Encryption of Kernel and User Space
    • N26
    • Nomorp
    • One Key to Rule Them All: Recovering the Master Key from RAM to break Android’s File-Based Encryption
    • RamCrypt: Kernel-based Address Space Encryption for User-mode Processes
    • ReFuzz — Structure Aware Fuzzing of the Resilient File System (ReFS)
    • RISCoT – Security Analysis of Trusted Execution Environments on RISC-V
    • SED (In)Security: Hardware-based Full Disk Encryption (In)Security
    • SGX-Kernel: Isolating Operating System Components with Intel SGX
    • SGX-Timing: Cache Attacks on Intel SGX
    • SoK: The Evolution of Trusted UI on Mobile
    • Soteria: Offline Software Protection within Low-cost Embedded Devices
    • STARK / MARK: Tamperproof/Mutual Authentication to Resist Keylogging
    • TEEshift: Protecting Code Confidentiality by Selectively Shifting Functions into TEEs
    • VMAttack: Deobfuscating Virtualization-Based Packed Binaries
  • Archive
    • ContrOWL: A new security app based on crowed intelligence
    • Ext4 File Recovery
    • Forensic Email Visualization
    • Forensic RAID Recovery
    • Forensig²: File System Images for Training Courses in Forensic Computing
    • Mobile Hotspots
    • Mobile-Sandbox & ADEL: Automated Malware Analyses / Mobile Phone Forensics
    • Privacy Aspects of Forensic Computing
    • PyBox – A Python Sandbox
    • TrustedPals: Framework to Help Establish Security in a Mutually Untrusted Distributed System
    • VirMA: Windows NT pagefile.sys Virtual Memory Analysis
    • Win Vista/7/8/10 Thumbnails Analyzer
  • Funded Projects
  • Publications
    • Technische Berichte in Digitaler Forensik

VMAttack: Deobfuscating Virtualization-Based Packed Binaries

VMAttack: Deobfuscating Virtualization-Based Packed Binaries

Abstract

We present VMAttack, a deobfuscation tool for virtualization-packed binaries based on automated static and dynamic analysis, which offers a simplified view of the disassembly. VMAttack is implemented as a plug-in for IDA Pro and as such, integrates seamlessly with manual reverse engineering. The complexity of the disassembly view is notably reduced by analyzing the inner working principles of the VM layer of protected binaries. Using static analysis, complex bytecode sequences of the VM are mapped to easy-to-read pseudo code instructions, based on an intermediate representation specifically designed for stack-based virtual machines. Using dynamic analysis, we identify structural components like the interpreter loop and compress instruction sequences by filtering out semantically redundant instructions of the execution trace. The integrated result, which rates both static and dynamic analysis’s results, provides the reverse engineer with a deobfuscated disassembly that tolerates weaknesses of a single analysis technique. VMAttack is currently limited to stack-based virtual machines like VMProtect. We evaluated VMAttack using binaries obfuscated with VMProtect and achieved an average execution trace reduction of 89.86% for the dynamic and 96.67% for the combined static and dynamic analysis.

Paper: VMAttack: Deobfuscating Virtualization-Based Packed Binaries (by Anatoli Kalysch, Johannes Götzfried, and Tilo Müller)
Slides: tba

 

Get the Code

Installation instructions and implementation:

https://github.com/anatolikalysch/VMAttack

Lehrstuhl für Informatik 1
Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

Martensstrasse 3
91058 Erlangen
  • Impressum
  • Datenschutz
Up