1 of 8

Corundum status updates

Alex Forencich

5/22/2023

2 of 8

Agenda

  • Announcements
  • Status updates

3 of 8

Announcements

  • Next Corundum developer meeting: June 5 at 9:00 AM PDT
  • Next switch development meeting: May 29 at 9:00 AM PDT

4 of 8

Status update summary

  • Bugs
    • DMA write done mux bug (done)
    • Possible Xilinx PCIe HIP bug (done?)
  • New queue management progress

​

5 of 8

Bugs: DMA write done mux bug

  • Interleaved operations on different ports on the same segment coupled with a difference in delay can cause a hang
  • Replicated in sim by adding delay to wr_done
    • Problem with readout of internal port mapping FIFO
  • Status: Fixed in development repo
    • Initial version of fixed mux added 1 cycle of latency
    • Updated version has the same latency

6 of 8

Bugs: Possible Xilinx PCIe HIP bug

  • Added delay in wr_done signal caused hang in HW
    • Delay in wr_done was causing DMA IF status FIFO to fill up, resulting in backpressure on completion TLP path to PCIe HIP
    • Observed corrupt completions on RC streaming interface, but handshaking looked OK
  • Possibly the same completion buffer management issue that was observed earlier
  • Added logic to PCIe DMA IF to track outstanding read data, block issuing new read requests at specified byte limit
    • Hangs with 64 KB limit, works (so far) with 32 KB limit)
  • Status: fixed in development repo

7 of 8

New queue management progress

  • Implemented new layouts for EQ/CQ/TXQ/RXQ
  • Merged TX CQ/RX CQ into single CQ
  • Split completion and event writes into separate instances
    • Currently, the same logic
  • Resurrected batched completion write logic
    • Updated layout, replaced pointer writeback with phase tag bit
    • Performance: slightly better for medium # of queues, slightly worse for lots of queues
  • Next: new descriptor fetch logic, merge TXQ/RXQ into QP SQ/RQ

8 of 8