1 of 19

2 of 19

SERVER

OCP NIC Community Update�November 2024

�����Sub-group Project wiki: https://www.opencompute.org/wiki/Server/NIC�

Mailing List: https://ocp-all.groups.io/g/OCP-NIC

NIC3.0

3 of 19

Agenda

  • OCP Global Summit
  • SFF-TA-1002 Connectors
  • Thermal Workstream Updates

4 of 19

OCP Global Summit

All presentations (videos and slides):�https://www.opencompute.org/events/past-events/2024-ocp-global-summit

​

Introduction to the Dual Small Form-Factor (DSFF) �Jason Rock/Dell, Joseph Hanson Vazquez/Intel,Thomas Ng/Intel�https://www.youtube.com/watch?v=xwCQoDBXZH8 �

Propagating Timing Synchronization Across OCP NICs �Jason Rock/Dell�https://www.youtube.com/watch?v=GULS00Dgklc

​

OCP NIC 3.0: PCIE Gen 6 Support with Next Generation Si and Thermal Test Fixtures

Hemal Shah/Broadcom, Jason Rock/Dell, Jon Lewis/Dell

https://www.youtube.com/watch?v=dJ9GJjY0SEE

​

​

5 of 19

OCP Global Summit

6 of 19

OCP NIC 3.0 Spec Ver 1.5 contributed

https://www.opencompute.org/contributions

7 of 19

SFF-TA-1002 Connectors - Type 1/Type 2

Cautionary tales for Baseboard vendors

SFF-TA-1002 Rev 1.5 added “Type 1” and “Type 2” connectors

  • Type 1 connectors have discrete GND pins
  • Type 2 connectors bond several GND pins together

​

OCP NIC: Compatible with both. Not an issue.

DC-SCM: Compatible with Type 1 only; Type 2 does not work. DC-SCM repurposes several GND pins as signal pins.

​

DC-SCM Rev 2.1 Ver 1.1 Errata https://www.opencompute.org/w/index.php?title=Server/MHS/DC-SCM-Specs-and-Designs

SFF-TA-1002 Rev 1.5 (SNIA doc, must be a member) https://members.snia.org/document/dl/27231

Snippet from Rev 1.5.1 (draft)

8 of 19

SFF-TA-1002 Connectors - Power Excursion

When we started OCP NIC 3.0, the typical card power was ~35 W. The connector supports up to 80 W, so any inrush was likely absorbed under the 80 W limit.

Newer generations of OCP NIC 3.0 cards are higher performance/higher power implementations that have higher DC power dissipation.

How do we quantify support for short term excursions past 80 W such as:

  • In-rush
  • Short term workload spikes

​

  • Do we define power excursion limits similar to PCIe CEM 6?
  • If so, what power intervals do we support? Follow CEM 6?
  • Need to work with SNIA/SFF-TA-1002 committee.

​

References:

  • Table 4-5 of CEM 6.0 (must be a PCI-SIG member)
  • SFF-TA-1002 → 1.1 A/pin, 12V, 6 pins (79.2W ~> 80W) (must be a SNIA member)

​

​

9 of 19

SFF-TA-1002 - 1.1 A/pin

OCP NIC operates at 12 V nominal.

Rail tolerance is +8%, -12%.

​

Is the connector 1.1 A/pin max regardless of the voltage?

​

At -12% → 10.56 V:

  • 1.25 A/pin → 79.2 W
  • 1.1 A/pin → 69.7 W

​

At +8% → 12.96 V:

  • ~1.02 A/pin → 79.2 W

10 of 19

SFF-TA-1002 Connectors - Call to Action

Question to Connector vendors:

  • How do you distinguish between Type 1 and Type 2 connectors?
  • Is the part number with “T1” and “T2”? or similar?
  • Are there physical markings on the connector body?

​

  • Is there test data to support power excursion limits similar to PCI-SIG CEM 6 Table 4-5?
  • Can we work with SNIA/SFF-TA-1002 on this?

​

  • SNIA/SFF-TA-1002 TWG to confirm 1.1 A/pin limit. → confirmed.

​

  • OCP NIC spec will be updated to incorporate any changes.

11 of 19

Thermal Workstream Update

  • Thermal workstream will meet every other Thursday at 9am PST
    • Next meeting will be 11/14
    • Email to be added: joseph.s.hanson@intel.com

​

  • Met last week (10/31)
    • Had good NIC vendor representation
    • Need system vendor participation

12 of 19

Thermal Workstream Goals

Top Priorities:

  1. TSFF thermal modeling and spec update
    1. Use SFF modeling assumptions, improved thermal solution

​

  1. DSFF/TDSFF thermal modeling and spec update
    1. Need thermal test fixture geometry finalized
    2. Plan is to align model to SmartNIC use case
      1. Large package, high power, advanced thermal solution

​

  1. DSFF thermal test fixture
    1. Provide community input/review before Dell finalizes?

​

  1. Community discussion/alignment on future thermals
    1. 1U max height option
    2. Liquid cooling enablement

13 of 19

OCP NIC Thermal Call to Action

  • FNIC and SmartNIC powers are increasing with higher speeds and feeds

​

  • SFF/DSFF hitting limits of cooling capabilities well before being power limited
    • At worst case boundary conditions

​

  • Need community (NIC vendors and system vendors) agreement on future enablement

14 of 19

DSFF vs 1U DSFF topside volume (WIP)

15 of 19

Backup slides

16 of 19

OCP Global Summit

  • October 15-17, 2024 San Jose, CA
  • 3 presentation in Break-out session
  • 1 lightning talk at Expo hall
  • OCP Innovation Village OCP NIC demo planned

​

17 of 19

Revision 1.5.0 Published

  • Review cycle completed. Published Sept 20th, 2024.
  • Minor comments received (clarification items)

​

​

​

​

​

​

  • https://www.opencompute.org/w/index.php?title=Server/NIC#Released_Specification_Docs

18 of 19

DSFF SI Update

  • Dell contributing DSFF CBB SI fixture, Gen 6 by default
  • Incorporated feedback from Broadcom & Intel
  • Gerber sent for fabrication
    • Board files here: Link
    • Proto assembly est Nov 22nd
    • Ordered FR4 for OCP Innovation Village demo

​

​

  • For DSFF CLB use two of CLB SFFs

19 of 19

DSFF Thermal Test Fixture

  • New DSFF TTF will support SFF / TSFF / DSFF form factors at PCIe® speeds up to 64 GT/s
  • Metal housing with a PCB designed to extend PCIe® lanes from a DC-MHS compatible server to a mounted OCP 3.0 DUT card
  • Intended to attach to a wind tunnel or flow bench
  • Pre-defined sensor locations for test repeatability
  • Onboard MCU shall operate same as prior thermal test fixtures and codebase will be uploaded to the Github repository
  • Currently in schematic Dates pending
  • Ordered FR4 PCBs for OCP Innovation Village Demo

MCIO

160pin

+�PICPWR

MCIO

160pin

+�PICPWR

Primary

4C+

Secondary

4C+

MCU

DSFF TTF Block Diagram

x16

x16

DSFF TTF Rendering