SEGFAULT 2026
SYED AMAANUDDIN · SAJID ZUBAIR
COMPILER OPTIMIZATION PROFITABILITY
Compiler Cost Model
LLVM program analysis + hand-designed heuristics + machine learning
to predict whether a loop optimization is worth applying.
Problem
System
Analysis
Models
Results
Outlook
THE PROBLEM
Optimizations are not universally beneficial
The same transformation that speeds up one loop can slow down another. Whether it pays off depends on the loop, not on the optimization.
Loop size
Instruction count and body weight
Memory behaviour
Contiguous vs strided access
Control flow
Branch density inside the body
Cache effects
Working set vs cache capacity
Trip count
Known, small, or fully unknown
Target costs
Per-instruction cost on this ISA
Production compilers answer this with static analysis, target cost models, and hand-tuned heuristics. This project asks whether the decision can be learned instead.
Problem
System
Analysis
Models
Results
Outlook
THE GOAL
Predicting profitability per candidate
“Given a candidate optimization and its compiler-level characteristics, predict whether applying that optimization is likely to be profitable.”
What the model answers
Is this particular optimization configuration profitable for this particular loop?
unroll factor = 4, on loop for.cond44
What it does not answer
Is this loop generally worth optimizing in some unspecified way?
too coarse to drive a compiler decision
Problem
System
Analysis
Models
Results
Outlook
ARCHITECTURE
From source code to a profitability decision
Source Code
.c benchmark
Clang
front end
LLVM IR
analysable form
Annotated Candidates
loop + optimization
Feature Extraction Pass
instructions, memory ops, branches, depth, trip count
LLVM Heuristic
TargetTransformInfo instruction costs + rules
Optimization-specific Random Forest
Heuristic Score vs Threshold
Profitability Decision · Comparison
Problem
System
Analysis
Models
Results
Outlook
MODEL DESIGN
One model per optimization
Different transformations depend on different loop characteristics, so a single shared classifier would have to learn four unrelated decision surfaces at once.
1
Loop Unrolling
Random Forest
trip count · loop size · instruction overhead · unroll factor
2
Loop Vectorization
Random Forest
vectorization factor · interleave factor · contiguous and strided accesses
3
Loop Tiling
Random Forest
tile size · memory locality · access patterns · loop nest structure
4
Loop Fusion
Random Forest
loop structure · instruction counts · dependence characteristics
Problem
System
Analysis
Models
Results
Outlook
CANDIDATE SELECTION
Optimizations are annotated, not guessed
__costmodel_vectorize(4, 2); /* vf, interleave */
for (i = 0; i < _PB_N; i++)
for (j = 0; j < _PB_N; j++)
w[i] = w[i] + alpha * A[i][j] * x[j];
optimization = vectorize
function = kernel
loop = for.cond
parameters = { vf: 4, if: 2 }
Why this matters
The configuration is part of the input. vf=4 and vf=16 on the same loop are two different questions, and the model is allowed to answer them differently.
Problem
System
Analysis
Models
Results
Outlook
LLVM ANALYSIS
What the passes extract from IR
Structure
Instructions
Memory
Configuration
The exact feature set is serialised alongside each trained model, so inference uses the same features, in the same order, that training used.
Problem
System
Analysis
Models
Results
Outlook
LEARNED MODEL
Random Forest classifiers
Annotated candidate
Feature normalization
Random Forest
Label + probability
Binary profitability label
0 Not profitable
1 Profitable
VECTORIZE
Probability : 0.5165
Prediction : PROFITABLE
Read the probability as the model's confidence, not as a promised speedup.
Problem
System
Analysis
Models
Results
Outlook
BASELINE
The LLVM heuristic cost model
A deliberately simple, fully transparent scorer built on LLVM's TargetTransformInfo. It exists to give the learned model something to be measured against.
Loop size
more work to transform
Memory ratio
favours vectorize, tile
Branch ratio
heavy control flow penalised
TTI cost
target instruction throughput
Weighted heuristic score
score ≥ threshold → PROFITABLE
instructions : 39 memory ratio : 0.5897
memory ops : 23 branch ratio : 0.1795
branches : 7 score : 0.9000
LLVM cost : 34 profitable : YES
Problem
System
Analysis
Models
Results
Outlook
COMPARISON
Learned model vs heuristic baseline
OPTIMIZATION
FUNCTION
LOOP
ML
HEURISTIC
TILING
kernel
for.cond
PROFITABLE
PROFITABLE
FUSION
kernel
for.cond21
PROFITABLE
PROFITABLE
UNROLL
kernel
for.cond44
NOT PROFITABLE
PROFITABLE
VECTORIZE
kernel
for.cond56
PROFITABLE
PROFITABLE
3/4
75.0%
agreement between the two models on the same candidates
This is agreement, not accuracy.
Both sides are predictors. No independently measured runtime ground truth enters this number — it answers how often the learned model agrees with the compiler-inspired baseline, and nothing more.
Problem
System
Analysis
Models
Results
Outlook
HONEST ASSESSMENT
Limitations of the prototype
Limited training data
Quality depends on benchmark count, loop diversity, configuration coverage and hardware targets. Right now all four are small.
Binary prediction only
The models say profitable or not. They cannot say how profitable, so configurations cannot yet be ranked against each other.
Weak hardware awareness
TTI cost reaches the heuristic, but the ML models carry no explicit hardware features and are effectively tied to one target.
Annotation-driven candidates
Candidates come from source markers rather than automatic discovery, so the system does not explore the full transformation space.
None of these are hidden in the write-up — a prototype that states its own boundaries is easier to defend than one that overclaims.
Problem
System
Analysis
Models
Results
Outlook
NEXT
Where this goes from here
Speedup prediction
Replace the binary label with a regression target so the model reports an expected speedup instead of a verdict.
expected speedup = 1.23x
Configuration ranking
With a continuous score, competing configurations for one loop can be ordered and the best one chosen.
VF=4 → 1.12x
VF=8 → 1.19x ← chosen
VF=16 → 0.97x
More transformations
interchange · distribution · software pipelining · inlining
Hardware-aware models
cache sizes · SIMD width · cores · memory bandwidth · ISA
Measured labels
real before/after runtimes from PolyBench kernels
Problem
System
Analysis
Models
Results
Outlook
SUMMARY
Three layers, one decision
01
Compiler analysis
LLVM supplies accurate low-level program and target information.
02
Machine learning
Random Forests learn the mapping from those features to profitability.
03
Heuristics
A transparent rule-based score keeps the system explainable end to end.
The long-term goal is to predict gain/loss of performance through speedup prediction and verdict to a ranked set of optimization configurations, so the compiler can choose the transformation with the highest expected benefit.
SEGFAULT 2026 · Syed Amaanuddin · Sajid Zubair