1 of 16

UCLA Samueli Computer Science

Yang-gon Kim

CSI-256 : GPU Microarchitecture

SnakeByte: A TLB Design with Adaptive and Recursive Page Merging in GPUs

Jiwon Lee, Won Woo Ro et al. Yonsei University. HPCA 2023

2 of 16

Contents

  1. Introduction
  2. Contribution
  3. Results
  4. Questions

2

3 of 16

Introduction : TLB

  • To translate the virtual address to physical address, we need TLB and Page Table.
  • One page table entry usually point to 4KB page.
  • The 512 entries means that TLB has a 2MB coverage (512x4KB).
  • They are parts of Memory Management Unit(MMU).

[image] : https://www.csa.iisc.ac.in/~arkapravab/papers/asiaccs-gpu-covert-channel-slides.pdf

3

4 of 16

Introduction : GMMU

SM

…

GMMU

GPU

  • GPU has its own MMU to Vir2Phy address translation and page table walker inside of the GPU.
  • GPU has a L1 TLB per one SM / L2 shared TLB for many SMs.

4

SIMD

SIMD

SIMD

L1$

page walk cache

page table walker

TLB

L1/L2

L2$

ctl

5 of 16

Contribution : Summary

NW : Heterogeneous computing benchmark

BH : Irregular programs on GPU

XSB : Monte Carlo reactor Analysis

  • Problem
    • GPU needs much wider TLB coverage
    • GPU applications show various page contiguity.

​

  • Solution
    • Recursive Page Merging(promotion) at runtime
    • Hardware-based Solution for page merging

5

6 of 16

Contribution : observations

NW : Heterogeneous computing benchmark

BH : Irregular programs on GPU

XSB : Monte Carlo reactor Analysis

  • GPU applications use an increasingly large amount of data.
  • [Fig. 2] By just increasing the numbers of TLB entries, performance increase by avoiding PTW penalties. (TLB pressure)
  • [Fig. 4] Many program have their own various contiguities. So, TLB should be adaptive to different workloads.

6

TLB coverage

page size

7 of 16

Contribution : Promotion

  • The baseline TLB entry cover 4KB and has 512 entries.
  • If one entry covers 4KB, then TLB can totally cover 2MB address space.
  • But, if one entry covers 2GB, then TLB can cover 1TB address space.
  • So, let’s promote one entry’s coverage from 4KB to max 2GB at runtime.

7

4KB

256KB

32KB

x8

x8

8 of 16

8

9 of 16

Contribution : Promotion

  • SnakeByte recursively consolidates contiguous pages to compose large page groups.
  • [Fig. 5] The size/valid bits come from unused bits in the baseline TLB.
  • [Fig. 7] If we have 8 contiguous Page Table entries, SnakeByte promote the Base PTE by updating the size bits and valid bits.

9

10 of 16

Contribution : Recursive Promotion

  • [Fig. 8] After one promotion(4KB to 32KB), the promotion continues.
  • [Fig. 9] After the promotion, one base-entry can point to much larger page space.
  • The non-base entries will be evicted soon.

10

11 of 16

Result

  • On average, SnakeByte with L1 TLB has 1.71x speedup .
  • This gain comes from reducing the overhead of accessing Page Table.
  • SnakeByte with L1+L2 TLB has 19.5% improvement over L1 only.
  • They don’t include the page fault penalty.
  • Memory footprint of these workloads range from 4MB to 1.54GB.

11

12 of 16

12

Thank You

Thank You

Thank You

13 of 16

Appendix

Q0. How does SnakeByte help page fault(not-present page)?

A0. SnakeByte does not support for page fault. It only supports TLB misses and PTW only. It doesn’t interfere with OS/driver.

​

Q1. If GMMU heavily relies on hardware only, how are programs controlled? (OS usually stalled the program and resume after page fault)

A1. when a warp is stalled by long memory access, then the warp scheduler activates the other warps to hide the latency.

13

14 of 16

Appendix

  • Recursive coalescing is the difference between the previous ones.
  • Previous ones have fixed size of page.
  • SnakeByte has much wider coverage with much various page group sizes.

14

15 of 16

Appendix

  • They added some micro-architecture in MMU.
  • When the new page is allocated, SnakeByte updates the valid bits.
  • And check the contiguity using valid bits and address via simple shift and subtraction.
  • If contiguity guarantees, then the base PTE is promoted
  • This process is off the critical path of accessing the PTE and PTW.

​

15

16 of 16

Appendix

  • SnakeByte is evaluated using GPGPU-sim v4.0.
  • They extend it to cover multi-level TLB and PTW.
  • The page demand overhead is not considered as SnakeByte focuses on reducing the address translation overhead. (hmmm…)
  • 22 cycles for L1 TLB access / 88 cycles for L2 TLB access

​

16