1 of 10

Corundum status updates

Alex Forencich

3/27/2023

2 of 10

Agenda

  • Status updates

3 of 10

Status update summary

  • Bugs
    • FIFO memory inference issue (in progress)
  • Priority flow control (todo)
  • AXI Virtual FIFO (mostly done)
  • Simulation model updates for DRAM/HBM (in progress)
  • Switch version of Corundum

4 of 10

Bugs: FIFO memory inference issue

  • Seeing TX packets with incorrect IP layer checksums, only on Intel devices
  • MLE traced the issue to FIFO between TX engine and TX checksum compute block incorrectly setting “enable” bit
  • Appears to be a Quartus tool bug related to merging pipeline registers into MLABs
    • Connecting RAM output register to logic analyzer or adding “preserve” attribute results in the bug disappearing
  • Status: reported to Intel

5 of 10

Priority Flow Control

  • Starting to look at supporting PFC in Corundum
  • HW
    • PFC frame TX/RX, pause quanta counters
    • Per-TC queues
    • Connection to TX/RX queues and PFC frame logic
    • Internal flow control
    • TC-aware/per-TC scheduling
  • SW
    • Driver support
  • Outstanding questions
    • How to map RX traffic to priority levels (and is this necessary)?
    • How to efficiently handle multiple traffic classes in HW?
    • What needs to be done in the driver?

6 of 10

Per-TC, per-port queues

  • Need to keep TCs separate so high-priority traffic can be preferentially dequeued, but static partitions are not great
  • IBM Prizma switch arch. paper has an interesting idea: use FIFOs to manage memory addresses, then store the addresses in much narrower FIFOs
    • TCs can use varying amounts of buffer space as necessary
    • Unused TCs and ports do not consume any memory
  • Working this for the main TX and RX sync. FIFOs in Corundum
    • FIFOs vs. linked lists?

7 of 10

AXI virtual FIFO

  • Large packet buffer capability in DRAM
  • Store both packet data as well as sideband data
  • Intent is to support operation at 100G with all packet sizes
    • 2x DDR4-2400 channels or 2-4 HBM ports
    • Main bottleneck is memory BW, so need efficient encoding scheme for framing and sideband data
  • Status
    • Working in HW at 100G line rate, reset and control mostly done
    • Updating Corundum designs to support RAM simulation
    • Finalizing DMA benchmark design updates for DRAM/HBM

8 of 10

Simulation model updates for DRAM/HBM

  • Added SparseMemory to cocotbext-axi
    • Supports arbitrarily large address spaces with a sparse cache
      • Write data stored in fixed-size blocks in a python dict
  • Adding DRAM/HBM interface to corundum simulation models
    • Creates DRAM/HBM channels based on configured parameters
    • Supports HBM internal switch (all ports can access full address space)
      • Not cycle-accurate - no contention, etc.

9 of 10

Switch version of Corundum

  • Capabilities
    • 10G-100G operation with reasonable port count – 16x 100G?
    • Multiple traffic classes/virtual channels
    • Switching capabilities – Ethernet switching, IP routing, match/action…
    • Reasonable resource utilization
  • Status
    • Functional design exploration (simulation-only RTL)
    • Set up new ethernet-switch repo for prototyping/development
  • Join #switch channel on Zulip for more information
    • https://corundum.zulipchat.com

​

10 of 10

Switch design meeting

  • Same time and zoom room as current dev meeting, but alternate weeks
  • Anyone is welcome to join

​

  • Join #switch channel on Zulip for more information
    • https://corundum.zulipchat.com

​