From source to binary
How Compiler Works
ソースからバイナリへ
從原始碼到二進制
Von Quelle Zu Binären
De source au binaire
Desde fuente a binario
Binarium ut a fonte
From source to binary
Luse Cheng
Sep 13, 2012 / 新竹碼農
Jim Huang (黃敬群)
jserv.tw@gmail.com
春秋: 微言大義
微言大義: �現代Linux環境, 程式啟動流程
Shell
eglibc
kernel
execve
./hello
SYS_execve
驗證執行檔
啟動loader
ld-linux.so
libc_start_main
hello
main
exit
SYS_exit
棄置Process
動態連結器
配置Process
動態連結
靜態連結
從原始碼到二進制: �常見的誤解
Source Code
Compiler (gcc, llvm)
HDL Source
Design Compiler (DC)
RTL Compiler (RC)
Assembly
Netlist
Selecting�Instruction
Technology�Mapping
軟體
硬體
A compiler is a computer program (or set of programs) that transforms source code written in a programming language (the source language) into another computer language (the target language, often having a binary form known as object code)
source : http://en.wikipedia.org/wiki/Compiler
編譯 Hello world 程式的流程
Executable: hello 執行檔
Linker (LD) 連結器
Object File: hello.o 目的檔
Assembler (AS) 組譯器
Assembly File: hello.s 組合語言
Compiler (CC1) 編譯器
C program: hello.c 原始碼
編譯
組譯
連結
三大步驟
圖解簡易完整編譯流程
int main() {
printf (“Hello World”);
}
int printf (const char* fmt, …);
main:
LDR R1, [R2 + 12]
BAL printf
#include <stdio.h>
int main() {
printf (“Hello World”);
}
Preprocess
Compile
Assemble
Main:
EC 00 00 12
F0 ?? ?? ??
Linking
0x2000 <Main>:
EC 00 00 12
F0 00 10 00
0x1000 <printf>:
EC 00 00 12
…
stdio.h
libc.a
常見的誤解: GCC 是個 C 編譯器�答案: GCC 是一個 Compiler Driver
gcc
cc1
as
cpp
collect2
Preprocessed source (.i or .ii)
Asm (.s)
Reloadable (.o)
Binary
(Executable)
library
crt*
link script
Source (.c and .h)
gcc
binutils
glibc
ld
GCC 這個執行檔, 可以當Compiler, 也能當 Preprocessor, 也能當 Assembler, 更能當 Linker
如果有人想當你的程式碼�那你怎樣才能做她的編譯器 (Compiler) ?
Target Language
High Level Language
Optimized Low Level Language
Compiler
編譯器小歷史:
Image source:�http://www-history.mcs.st-and.ac.uk/PictDisplay/Hopper.html http://upload.wikimedia.org/wikipedia/commons/thumb/5/55/Grace_Hopper.jpg/300px-Grace_Hopper.jpg
Grace Hopper, inventor of
A-0, COBOL, and the
term “compiler.”
先有雞還是先有蛋
編譯器技術=屠龍之技
GCC 的架構
int main()
{
return 0;
IamDeadCode();
}
資料流程分析 (Data-flow analysis)
Code Motion & Pointer Aliasing
while (y < z) {
int x = a + b;
y += x;
}
void foo(int *a, int *b, int *y, int z) {
while (*y < z) {
int x = *a + *b;
*y += x;
}
}
int t1 = a + b;
while (y < z) {
y += t1;
}
Q: How about this?
Strict Aliasing Rule & Restrict Pointer Aliasing
Static single assignment, SSA Form
Diego Novillo, GCC Internals - IRs
使用 SSA 可以加強/加快以下的最佳化
SSA @ Constant propagation (常數傳遞)
main:
mov r0, #0
bx lr
int main()
{
int a = 11,b = 13, c = 5566;
int i, result;
for (i = 0 ; i < 10000 ; i++)
result = (a*b + i) / (a*c);
return result;
}
main:
…
mov r4, #0
.L5:
mov r1, #61184
add r0, r4, #143
add r1, r1, #42
add r4, r4, #1
bl __divsi3
cmp r4, r5
ble .L5
…
GCC-4.x (SSA)
GCC-3.x (No-SSA)
GCC中端: Gimple & Tree SSA Optimizer
GCC後端: Register Transfer Language (RTL)
(set (reg:SI 60 [b])
(plus:SI (reg:SI 61 [a])
(const_int -56 [0xffffffc8])))
b = a - 56
GCC後端: RTL Patten Match Engine
(define_insn "*addsi3"
[(set (match_operand:GPR 0 "register_operand" "=d,d")
(plus:GPR (match_operand:GPR 1 "register_operand" "d,d")
(match_operand:GPR 2 "arith_operand" "d,Q")))]
"!TARGET_MIPS16"
"@
addu\t%0,%1,%2
addiu\t%0,%1,%2"
[(set_attr "type" "arith")
(set_attr "mode" "si")])
MIPS.md
d代表是Register
Q代表是整數
指令的屬性(用於管線排程)
指令限制(非MIPS16才可使用)
addiu $2, $3, -56
b = a - 56
(set (reg:SI 60 [b])
(plus:SI (reg:SI 61 [a])
(const_int -56 [0xffffffc8])))
GCC後端 : 暫存器分配
GCC後端 :管線排程
IF
ID
EX
ME
WB
lw $8, a
lw $9, b
mul $10,$8,$8
add $11,$9,$10
Clock 0
Clock 1
Clock 2
Clock 3
Clock 4
Clock 5
Clock 6
Clock 7
Clock 8
lw $12,c
add $13,$12,$0
lw $8, a
lw $9, b
lw $12,c
mul $10,$8,$8
add $13,$12,$0
IF
ID
EX
ME
WB
GCC後端: Peephole optimization
;; subs rd, rn, #1
;; bcs dest ((unsigned)rn >= 1)
;; This is a common looping idiom (while (n--))
;; sub rd, rn, #1
;; cmn rd, #1 (equivalent to cmp rd, #-1)
;; bne dest
分很多個.c file or not 分很多個.c file
深入淺出 Compilation Unit
Compilation Unit
Internal Data
Visible Data
Internal Function
Visible Function
Internal Function
External Data Reference
External Func Reference
Compilation Unit�盡可能使用區域變數
main:
mov r3, #0
stmfd sp!, {r4, lr}
movw r4, #:lower16:.LANCHOR0
movt r4, #:upper16:.LANCHOR0
mov r1, r3
str r3, [r4, #0]
.L2:
ldr r0, .L4
bl printf
ldr r1, [r4, #0]
add r1, r1, #1
str r1, [r4, #0]
cmp r1, #9
ble .L2
ldmfd sp!, {r4, pc}
.L4:
.word .LC0
static int i;
int main()
{
for (i = 0 ; i < 10; i++)
printf ("Hello %d\n", i);
}
int main()
{
int i;
for (i = 0 ; i < 10; i++)
printf ("Hello %d\n", i);
}
main:
stmfd sp!, {r4, lr}
mov r4, #0
.L2:
mov r1, r4
ldr r0, .L4
add r4, r4, #1
bl printf
cmp r4, #10
bne .L2
ldmfd sp!, {r4, pc}
.L4:
.word .LC0
11
GCC 內建函式 (Built-in Function)
IPO (Inter-Procedural Optimization)
IPO (Inter-Procedural Optimization)
Source 1
Source 2
Source 3
Compiler
Compiler
Compiler
IR 1
IR 2
IR 3
Linker
Full IR
Compiler
Optimized Program
21世紀的另外一種選擇: LLVM
LLVM: 全時最佳化編譯器平台
空閒階段
執行階段
載入/安裝階段
連結階段
編譯階段
正所謂是
橫看成嶺側成峰
遠近高低各不同
GNU Toolchain 的守備範圍
LLVM核心觀念: �LLVM is a Compiler IR
�LLVM 的架構 : 從原始碼到二進制
C Front-End
C++ Front-End
LLVM bitcode
… Front-End
Stream-out
Optimized
LLVM bitcode
LLVM x86
Back-End
LLVM ARM
Back-End
X86 ASM
X86 Object
ARM ASM
ARM Object
Execution (JIT)
Execution (JIT)
In LLVM
Not in LLVM
LLVM Optimizer
BitCode �Passes
Selection DAG
Passes
Machine
Passes
LLVM C/C++
Back-End
C/C++ Source
Use LLVM Target
LLVM Toolchain (llvm-gcc)
C Front-End
C++ Front-End
Generic
C Source
C++ Source
Gimple
Gimplify
LLVM Bitcode
NOTE: LLVM早期的也沒有自己的 Assembler (後來有 MC) 和 Linker 來建立完整編譯流程
LLVM Toolchain (Clang)
範例: Clang Expressive Diagnostics
xx.cc: In function 'void test(foo*)':
xx.cc:9:24: error: no match for 'operator+' in '(((a*)P) + ((sizetype)(*(long int*)(P->foo::<anonymous>.a::_vptr.a + -32u))))->a::bar() + * P'
xx.cc:9:24: error: return-statement with a value, in function returning 'void' [-fpermissive]
struct a {
virtual int bar();
};
struct foo : public virtual a {
};
void test(foo *P) {
return P->bar() + *P;
}
xx.cc:9:21: error: invalid operands to binary expression ('int' and 'foo')
return P->bar() + *P;
~~~~~~~~ ^ ~~
g++-4.7
clang-3.1
Clang/LLVM Toolchain
CLANG
Language
Optimizer
C Source
C++ Source
LLVM Bitcode
AST Stream-out
Binding Script .. etc
來源: http://www.aosabook.org/en/llvm.html
LLVM 千秋萬世, 一統江湖
滅諸侯,成帝業,為天下一統
LLVM TableGen (Target Description)
def ADDPS : PI<0x58,
(outs VR256:$dst),
(ins VR256:$src1, VR256:$src2),
"addps $src2, $src1, $dst",
[(set VR256:$dst, (fadd VR256:$src1, VR256:$src2))]>;
Instruction Encoding (For MC Assembler)
Input/Output
Assembly (For ASMPrinter)
Instruction Selection Pattern
(For Selection DAG)
完美Compiler進化論: LLVM的多變化
Source 1
Source 2
Source 3
Frontend
Frontend
Frontend
LLVM IR 1
LLVM IR 2
LLVM IR 3
LLVM Link
LLVM Optimizer
LLVM ARM Backend
IPOed executable/
Share object
Compile Time
Link Time
Source 1
Source 2
Source 3
Frontend
Frontend
Frontend
LLVM IR 1
LLVM IR 2
LLVM IR 3
LLVM Link
LLVM Optimizer
LLVM ARM Backend
Processor Specific
executable/
Share object
Compile Time
Link Time
Load /Installation Time
LLVM IR
Idle Time
LLVM Optimizer
Profile Data
LLVM ARM Backend
PGOed/ Runtime Opted
executable/ Share object
Load Time
LLVM 的發展
相信你的編譯器
相信你的編譯器: 解答
References
Any Question ?