DocPrune: Document Token Pruning for Efficient Document Question Answering
2026.03.11
Index
1. Motivation
Task A. Long Document Question Answering
문서 : 3366개
페이지 : 41114개
Question
For which film did Ben Piazza play the role of Mr. Simms?
페이지 : 4개
페이지 : 6개
.....
1. Motivation
Task A. Long Document Question Answering
문서 : 3366개
페이지 : 41114개
Question
For which film did Ben Piazza play the role of Mr. Simms?
페이지 : 4개
페이지 : 6개
.....
Stage 1. Evidence Page Retreival
Stage 2. Question Answering
1. Motivation
Task A. Long Document Question Answering
Stage 1. Evidence Page Retrieval
Stage 2. Question Answering
Top-K Retrieved Pages
The title is W. C. Handy.
Head
Vision Encoder
LLM
...
...
...
Document token
Prompt token
Question token
Top-K Retrieved Pages
3rd
2nd
1st
Answer
Describe
the
image
What
is
the
title
.
Prompt
Question
Document Encoder
...
...
Document
Question �Encoder
What
is
the
title
?
Question
?
N pages
...
Question embedding
Retrieval
...
...
...
Document embeddings
N pages
Select
Output�token
RAG-QA
1. Motivation
Task B. Token Pruning
Output : 10 Token
38% ↓
Input : 16 Token
1. Motivation
Task B. Token Pruning
Output : 10 Token
38% ↓
Input : 16 Token
Point 1. How to define unnecessary tokens?
Point 2. Which layer to drop these tokens?
1. Motivation
Task B. Token Pruning
Stage 1. Evidence Page Retrieval
Stage 2. Question Answering
Top-K Retrieved Pages
The title is W. C. Handy.
Vision Encoder
LLM
...
...
...
Document token
Prompt token
Question token
Answer
Describe
the
image
What
is
the
title
.
Prompt
Question
Document Encoder
...
...
Document
Question �Encoder
What
is
the
title
?
Question
?
N pages
...
Question embedding
Retrieval
...
...
...
Document embeddings
N pages
Select
Output�token
QA
Head
3rd
2nd
1st
K Pages
1. Motivation
Task B. Token Pruning
Stage 1. Evidence Page Retrieval
Stage 2. Question Answering
Top-K Retrieved Pages
The title is W. C. Handy.
Vision Encoder
LLM
...
...
...
Document token
Prompt token
Question token
3rd
2nd
1st
Answer
Describe
the
image
What
is
the
title
.
Prompt
Question
Document Encoder
...
...
Document
Question �Encoder
What
is
the
title
?
Question
?
N pages
...
Question embedding
Retrieval
...
...
...
Document embeddings
N pages
Select
Output�token
QA
Head
Output : 10 Token
Top-K Retrieved Pages
✓
Point 1. How to define the objective of pruning?
→ Question-Dependent Objective
1. Motivation
Task B. Token Pruning
Stage 1. Evidence Page Retrieval
Stage 2. Question Answering
Top-K Retrieved Pages
The title is W. C. Handy.
Vision Encoder
LLM
...
...
...
Document token
Prompt token
Question token
3rd
2nd
1st
Answer
Describe
the
image
What
is
the
title
.
Prompt
Question
Document Encoder
...
...
Document
Question �Encoder
What
is
the
title
?
Question
?
N pages
...
Question embedding
Retrieval
...
...
...
Document embeddings
N pages
Select
Output�token
QA
Head
Output : 10 Token
38% ↓
Top-K Retrieved Pages
Layer 2, 3, 5...
Point 1. How to define the objective of pruning?
Point 2. Which layer to drop these tokens?
24ECCV : FastV
→ Question-Dependent Objective
→ Heuristic selection ( Layer 2, 3, ... )
1. Motivation
Task B. Token Pruning
Stage 1. Evidence Page Retrieval
Stage 2. Question Answering
Top-K Retrieved Pages
The title is W. C. Handy.
Vision Encoder
LLM
...
...
...
Document token
Prompt token
Question token
3rd
2nd
1st
Answer
Describe
the
image
What
is
the
title
.
Prompt
Question
Document Encoder
...
...
Document
Question �Encoder
What
is
the
title
?
Question
?
N pages
...
Question embedding
Retrieval
...
...
...
Document embeddings
N pages
Select
Output�token
QA
Head
Output : 10 Token
38% ↓
Top-K Retrieved Pages
Layer 2, 3, 5...
Point 1. How to define the objective of pruning?
Point 2. Which layer to drop these tokens?
Point 3. Image-targeted pruning design
→ Question-Dependent Objective
→ Heuristic selection ( Layer 2, 3, ... )
2. Method
Observation
Method
2. Method
O1. Dominance of background regions
2. Method
Q1. In which film did Ben Piazza play Mr. Simms?
Q2. What animals race in the Kentucky Derby?
A. Horses
A. Mask
O1. Dominance of background regions
1. Foreground
2. Background
→ background : 36%
2. Method
M1. Background Token Pruning
2. Method
O2. Sparse localization of supporting evidence
Q. In which film did Ben Piazza play Mr. Simms?
Q. What animals race in the Kentucky Derby?
A. Horses
A. Mask
2. Method
O2. Sparse localization of supporting evidence
Q. In which film did Ben Piazza play Mr. Simms?
Q. What animals race in the Kentucky Derby?
A. Horses
A. Mask
2. Method
O2. Sparse localization of supporting evidence
Q. In which film did Ben Piazza play Mr. Simms?
Q. What animals race in the Kentucky Derby?
A. Horses
A. Mask
-1.1
-0.8
2. Method
M2. Question-Aware Token Pruning
1. compute the cosine similarities and aggregated by text tokens
2. smooth the similarity map using Gaussian filtering
3. remain only relevance tokens exceed the threshold
2. Method
O3. Layer-dependent pruning effect
L14
L0 – L14 : random > attn based
L15 – L27 : random < attn based
2. Method
O3. Layer-dependent pruning effect
L0 – L14 : random > attn based
L15 – L27 : random < attn based
L14
1. Weak identification of salient visual tokens before layer 14
2. Underdeveloped inter-token interactions before layer 14
2. Method
O3. Layer-dependent pruning effect
L0 – L14 : random > attn based
L15 – L27 : random < attn based
L14
1. Weak identification of salient visual tokens before layer 14
2. Underdeveloped inter-token interactions before layer 14
How can we know layer 14?
2. Method
L2 Norm ↑ → Comprehension ↑
O3. Layer-dependent pruning effect
→ Model Comprehension
2. Method
O3. Layer-dependent pruning effect
→ Model Comprehension
L2 Norm ↑ → Comprehension ↑
2. Method
O3. Layer-dependent pruning effect
→ Model Comprehension
L2 Norm ↑ → Comprehension ↑
F1 ↑
F1 ↑
2. Method
O3. Layer-dependent pruning effect
→ Model Comprehension
L2 Norm ↑ → Comprehension ↑
2. Method
M3. Comprehension-Aware Token Pruning
1. determine the optimal layer with comprehension threshold
2. tokens with attention values below are dropped
2. Method
1
2
3
Dominance of background regions
Sparse localization of supporting evidence
Layer-dependent pruning effect
1
2
3
Background Token Pruning
Question-Aware Token Pruning
Comprehension-Aware Token Pruning
Observation
Method
Summary. Observation vs Method
3. Pipeline
Overview of Pipeline
3. Pipeline
Overview of Pipeline
1. Maximize Information Preservation
1
1
3. Pipeline
Overview of Pipeline
1. Maximize Information Preservation
2. Leverage Retrieval-Guided Feature
2
3. Pipeline
Overview of Pipeline
1. Maximize Information Preservation
2. Leverage Retrieval-Guided Feature
3. Query-Centric Pruning
3
3. Pipeline
Query-Dependency Only
Heuristic Selection
Image-Centric Design
Previous
OURS
Pruning Objective
Layer Selection
Target Domain
Previous vs OURS
3. Pipeline
Previous vs OURS
Query-Dependency Only
Heuristic Selection
Image-Centric Design
Pipeline-aware Adaptive Pruning
Previous
OURS
Pruning Objective
Layer Selection
Target Domain
3. Pipeline
Previous vs OURS
Query-Dependency Only
Heuristic Selection
Image-Centric Design
Pipeline-aware Adaptive Pruning
Comprehension-Based Optimal Selection
Previous
OURS
Pruning Objective
Layer Selection
Target Domain
3. Pipeline
Previous vs OURS
Query-Dependency Only
Heuristic Selection
Image-Centric Design
Pipeline-aware Adaptive Pruning
Comprehension-Based Optimal Selection
Document-Centric Design
Previous
OURS
Pruning Objective
Layer Selection
Target Domain
4. Experiment
E1. Qualitative Analysis-A
4. Experiment
E1. Qualitative Analysis-B
4. Experiment
E2. Ablation Study
+1.2
x2.2
+0.6
x2.3
+1.0
x2.2
+1.2
x2.2
+1.0
x3.3
+1.5
x3.2
4. Experiment
E3. Main Table
Performance on M3DocRAG
4. Experiment
E3. Main Table
Performance on MMLongBench
Performance on VDocRAG
5. Conclusion