1 of 19

REx86: A Local Large Language Model for Assisting in x86 Assembly Reverse Engineering

ACSAC 2025

2 of 19

Introduction

  • Evaluate local LLMs for assisting with x86 Reverse Engineering (RE)
  • Propose fine-tuned x86-focused LLM (REx86)
  • Goal: improve human understanding of x86 assembly using LLMs
  • Focus:
    • Which local LLMs perform best for x86 comment generation?
    • How much do fine-tuned, local LLMs improve RE understanding?

3 of 19

Motivation

  • RE is complex and time-consuming:
    • Compilation removes variable names, types, and comments
    • Compiler optimizations reduce readability
  • RE tools fail to recover semantics
  • LLMs widely adopted across cyber security tasks
  • API-based LLMs raise security and data leakage concerns
    • Need for effective locally deployable LLMs for RE

4 of 19

Background

  • Low-Rank Adaptation (LoRA)
    • Efficient method for fine-tuning large models
    • Freeze pretrained weights W (no updates during training)
    • Add small low-rank adapters A and B
    • Output: h = Wx + BAx
  • Quantization
    • Reduce model size by lowering weight precision
    • Small accuracy drop in exchange for large memory savings
  • Unsloth
    • Fine-tuning framework optimized for speed and low VRAM usage

5 of 19

Methodology

  • Dataset Creation
    • Five tasks:
      • Code Intent, Complete the Code, Inline Comments, Header Comment, Q&A
    • Data sources:
      • Asm shellcode: asm code snippets and the corresponding intent
      • Rosetta Code
      • Shellstorm
      • xorpd solutions
      • x86 manuals → Q&A pairs generated using GPT-4o

x86 samples, including both �header and inline comments

6 of 19

Methodology

  • Dataset Creation

7 of 19

Methodology

  • Model selection:
    • Open-weight models runnable locally (NVIDIA RTX 5090)

8 of 19

Methodology

  • Fine-tuning using Unsloth
    • Unsloth used for all fine-tunings
    • Key techniques:
      • LoRA
      • Quantization
  • Process

9 of 19

Evaluation

  • RQ1: Which local LLM architectures and parameter sizes are most effective at generating accurate comments for disassembly output after fine-tuning on x86-specific datasets?
  • RQ2: To what extent can fine-tuned, local, open-weight LLMs improve the understanding of x86 assembly code for reverse engineering tasks?

10 of 19

Evaluation

  • Quantitative Evaluation
    • Goal: evaluate comment generation (RQ1)
    • Metrics: Cross Entropy (CE), semantic embedding Cosine Similarity (CosSim)

11 of 19

Evaluation

  • Qwen2.5-Coder-7B achieves highest performance
    • Was named REx86

12 of 19

Evaluation

  • User Case Study
    • Compare performance of REx86 against base model
    • Participants
      • 43 undergraduate students
      • Groups: REx86, Base, Control (no model-generated comments or chat)
    • Study Design
      • Given crafted Windows/x86 malware with known ground truth
      • IDA-based analysis to determine malware intent

13 of 19

Evaluation

  • User Case Study
    • Questionnaire:
      • Improvement in understanding of overall malware functionality
      • Improvement in understanding of individual assembly lines
      • Helpfulness of chat responses during reverse engineering

14 of 19

Evaluation

  • User Case Study
    • Scoring: 5-point Likert scale, mean-centered (−2 to +2)

15 of 19

Evaluation

  • Qualitative Evaluation
    • Compare REx86 vs base model (RQ2)
    • Evaluation on two x86 snippets:
      • Bitwise operation example
      • Obfuscated-style example

16 of 19

Evaluation

  • Bitwise operation

17 of 19

Evaluation

  • Obfuscated Code Sample

Fail

18 of 19

Limitations and Future Work

  • Limited fine-tuning dataset size
  • Focused only on x86 architecture
    • Future work: extend to ARM, MIPS
  • User study: 43 participants, single sample
    • Cannot be fully generalized

19 of 19

Strengths and Weaknesses

  • Strengths
    • Systematic evaluation across multiple model families and sizes
    • Focus on local, deployable LLMs (realistic RE setting)
    • Releases fine-tuned models and dataset
  • Weaknesses
    • Writing quality:
      • Typos, inconsistencies → confusing in places
      • Some claims feel overstated compared to evidence
    • Methodology:
      • Q&A dataset may contain hallucinations
      • Very few epochs of training
      • No runtime cost for fine-tuning
    • Evaluation:
      • Scope feels limited despite available tasks (only 2 RQs)
      • Participants are just students