1 of 1

ICGMM: CXL-enabled Memory Expansion with

Intelligent Caching Using Gaussian Mixture Model

Hanqiu Chen*, Yitu Wang*, Luis Vitorio Cargnini, Mohammadreza Soltaniyeh, Dongyang Li, Gongjin Sun, Pradeep Subedi, Andrew Chang, Yiran Chen and Cong Hao (*Equal Contribution)

Hanqiu Chen: hanqiu.chen@gatech.edu

Sharc Lab | Georgia Tech | https://sharclab.ece.gatech.edu/

Introduction

DRAM Cache for SSD: Challenges

2D Gaussian Mixture Model (GMM): Motivations

  • Spatial: Distribution can be fitted with a combination of different gaussian functions
  • Temporal: Uneven access frequency within a specific range of addresses

2D GMM for Intelligent Caching and Eviction

ICGMM Hardware Architecture Design

Experiment Results

  • Cache policy engine is a free-running kernel equipped with an optimized GMM kernel for predicting the likelihood of future page access frequency
  • Cache control engine is responsible for cache management, hit/miss determination and cache replacement, which also includes a SSD access latency emulator
  • Signal controller is used for interfacing with cache control and policy engines and managing data flow between HBM, on-board buffers, and different modules and blocks.

2D GMM for caching

2D GMM for eviction

Summary

  • Compared to Least Recently Used (LRU) cache policy, GMM can achieve 0.32% ~ 6.14% reduction in cache miss rate, 16.23% ~ 39.14% reduction in SSD access latency

ICGMM is a hardware-managed DRAM caching system for CXL-enabled memory expansion prototyped on FPGA using Gaussian Mixture Model as cache policy engine

  • Compared to LSTM cache controller, GMM can achieve more than 10,000x latency reduction with only 2% BRAM utilization, and can completely hide GMM overhead with dataflow streaming
  • The lightweight LSTM has low cache hit rate (<50%)

A dataflow architecture using FIFOs for module interconnect

ICGMM is prototyped on FPGA, using FPGA DRAM as cache

  • System – Hardware-managed DRAM cache system design
  • Algorithm – GMM-based cache policy for intelligent caching and eviction
  • Hardware – Optimized GMM hardware design with dataflow architecture
  • Performance – Improved DRAM cache hit rate and reduced average SSD access latency