1 of 8

Half-precision SIMD(FP16)

Ilya Rezvov @ Google

2 of 8

Motivation

  • Demand from ML applications for interaction with GPU or fallback on CPU
  • Hardware is here, lets let to use it
    • X64: F16C extension and AVX512-FP16
      • Part of AVX10 extension set
    • ARMv8.2 includes Android phones starting with Pixel 3, Galaxy S9 (Snapdragon SoC), Galaxy S10 (Exynos SoC), iOS devices with A11 or newer SoCs, all Apple Silicon Macs, and Windows ARM64 laptops based with Snapdragon 850 SoC or newer.

3 of 8

Game plan

  • Introduce F32.load_f16 and F32.store_f16 to support scalar scenarios via F32
  • SIMD F16x8 shape and vector operations
    • Replicate F32x4 instruction set with roundToEven semantic
    • No additional memory instruction needed because of i16x8 covers it
    • F16x8.demote_f64x2_zero because it is not composable
  • Move the proposal to SIMD opcode space(0xFD)
    • Except f32.[load|store]_f16. It could live in Numeric(0xFC) namespace

4 of 8

Hardware support and emulation

  • Arm64
    • Arm64 support via FEAT_FP16 is no trouble and straightforward
  • X64
    • Full-scale F16 instruction set only for AVX512
      • Deployed only on Sapphire Rapids Xeon chips and no signs of broader adoption
      • Looking forward on AVX10(AVX512 with fixed extension set and vector 256-bit registers)
    • Emulation via F16C and F32 AVX/AVX2 instructions
      • Multiple tiers of emulation
        • F16x8 -> F32x8 -> op -> F16x8 with AVX2 and F16C
          • Needs an optimization pass to be on parity with explicit F32x4 performance
        • F16x8 -> 2 X F32x4 -> op -> F16x8 with SSE/AVX and F16C
      • F16C supports only F32 <-> F16 conversions so F64 -> F16 should be emulated

5 of 8

Nondeterminism and NaNs

  • FMA
    • Could be kept deterministic up to NaNs because emulated only with higher precision
  • Saturated and relaxed int <-> float conversions
    • Same as with “big” floats
  • NaNs
    • Expected source of non-determinism
      • Could be fixed with deterministic profile to use qNaNs as result
    • Shouldn’t force payload preservation because of F32 emulation path conversions
      • But it falls under NaNs non-determinism anyway

​

6 of 8

Instructions

7 of 8

Implementation status

  • Prototyped in V8
    • Full software support
    • Full Arm64 support
    • Liftoff only X64 support
  • Basic support landed in the main branch off LLVM behind a flag
    • Via SIMD intrinsics

​

8 of 8

Poll for Phase 2

  • The proposal repo is set up
  • Reasonably high level of consensus has been reached