Corundum status updates

Alex Forencich

10/09/2023

Agenda

• Announcements
• Project ideas
• Status updates

Announcements

• Next Corundum developer meeting: October 23 at 9:00 AM PDT
• Next switch development meeting: October 16 at 9:00 AM PDT

Project ideas

• Some relatively self-contained project ideas:
• Oversampled 1000BASE-X on a 10GBASE-R transceiver
• Open-source 40G MAC+PCS
• MAC+PHY+GT wrappers for switchable 1/10/25/40/100 G
• RISC-V management core
• Open-source FT4232 JTAG adapter with Alveo connectors
• Improved transmit scheduler (WFQ, rate limiting, etc.)
• SR-IOV support
• Multi-head support (multiple PCIe and/or AXI host interfaces)
• Improved application section SoC support

Status update summary

• New PTP subsystem progress
• WR time sync ideas
• Architectural changes
• Single HW interface, split in SW
• Switch integration
• Additional app section passthrough

​

New PTP subsystem

• Distribute time from PHC via one wire serial + PTP ref clock
• Leaf clocks perform CDC into target clock domains
• Supports truncated timestamp format from which full 96 bit timestamps can be reconstructed
• Status:
• Protocol more or less defined
• PHC simulation model mostly done
• Working on HDL for leaf clock

PTP TD protocol

• 16 bit UART-like protocol
• 1 start bit, 16 data bits, 0 stop bits, baud rate = ref clock freq
• 17 clock cycles to cross parallel data to a different clock domain
• 256 cycle update period
• 256/17 = 15.05, can fit 14 16-bit words in each message and have 1 full word of idles for frame sync to work correctly
• Format similar to GPS almanac – data sent at different rates
• All information to set 64 bit clock sent in every message
• Other portion of message cycles through 96 bit information and timestamp reconstruction information
• Timestamp reconstruction
• “Current” and “alternate” offsets for ~500 ms reconstruction window

WR time sync ideas

• Original WR switch runs everything synchronous at 125 MHz
• Only need to measure phase offset of each RX clock with DDMTD
• Software “fixes up” RX timestamps with DDMTD measurements
• Predictable PHY latency: synchronous gearboxes
• Have some ideas for PHY/MAC modifications to support this
• 10G/25G/100G require different clocks
• Most likely 161.1328125, 322.265625 and 402.83203125
• Integer multiples of 80.56640625 (2, 4, 5) or 40.283203125 (4, 8, 10)
• Modification of DDMTD with clock enables to handle ratios
• What about 250 MHz PCIe clock + PTM?

Single HW interface

• Currently, Corundum supports multiple HW interfaces, with each interface potentially connecting to multiple ports
• Is this really necessary?
• New queue management logic trades area for performance
• Dynamic queue allocation in driver + RX indirection tables mean software can smoothly expose each port as on OS-level interface
• With embedded switch, it’s likely going to make even more sense to manage this sort of thing in software

Switch integration

• Sharing code between NIC and switch opens up some interesting possibilities
• Switchable/splittable 10G/25G/100G ports
• How to handle splitting ports with application section interfaces?
• E-switch for internal routing for SRIOV, etc.
• Additional internal switch ports for application section?
• Switch likely will use additional clocks
• Would the application section interface clocking change?
• Should the PCIe clock be decoupled from the core clock?

​

App section passthrough

• Some applications require more “stuff” to get passed through to application section
• Can this be done with macros?
• Probably need to `include two files
• Port definitions (included in module port list)
• Port connections (included in instance port list)
• What about parameters?

​