1 of 25

2 of 25

SERVER

OCP NIC Community Update�October 2023

�����Sub-group Project wiki: https://www.opencompute.org/wiki/Server/NIC�

Mailing List: https://ocp-all.groups.io/g/OCP-NIC

NIC3.0

3 of 25

Agenda

  • OCP Global Summit – Break-out session
  • OCP Global Summit – Experience Lab
  • Minor spec edit heads-up
  • PCIe Gen6 update
  • Double-wide SFF

​

​

​

4 of 25

OCP Global Summit 2023

October 17 to 19: San Jose Convention Center

​

Break-out session/Presentation:

2 presentation tentatively accepted

  • OCP NIC in 2023: 800G, PCIe Gen6 & Test Integration
  • Ensuring Predictable Thermal Performance for Higher Speed OCP 3.0 NICs

​

Experience Center demo:

Request for space approved

Amphenol, Broadcom, Dell, Intel, Meta, Portwell, TE

5 of 25

OCP NIC 3.0 minor spec edits

Edits that still in draft related to MCTP over USB binding, reference to DSP0222 etc

Offline email discussion within CLA members and would be send out for our standard public review soon

6 of 25

SIWG PCIe Gen6 Channel Simulation

With Channel loss limited to the spec and lower xtalk, it can be recovered

Tanmoy

7 of 25

Problem Statement

SFF and LFF are not plug-compatible

Platforms and adapters are commonly SFF today

LFF face lack of system and lack of device card

​

Requires Different Connector

Uses same connectors

Motherboard with 1x LFF

Motherboard with 2x SFF

SFF

SFF

LFF

Arnon/Harvey

Presented on: 8/9/2023

4C

4C+

4C+

4C+

8 of 25

Double-wide SFF for Smart NICs

Spans over two 4C+ connectors

Regular and Tall options

160 Watt

x32 PCIe lanes

13% more board space than LFF

​

Opens:

    • Management scheme
    • Latching mechanism
    • More

Illustration only

Arnon/Harvey

Presented on: 8/9/2023

9 of 25

D-SFF option preference

Option 1: Follow M-FLW

Option 2: Increase gap support thumbscrews

Customer A

Required cost for surprise removal is high.

Platform incline to internal latch version

Width of faceplate of thumbscrew became a limitation trade-off for other system feature

Customer B

Embrace internal lock since the start of OCP

​

Customer C

Majority thumbscrews. Some internal latch is niche

​

​

Vendor B

Ejector for Customer D, Internal lock Customer A & B

Thumbscrews for channel ODM CSP partners

On website wide distributions is thumbscrews

​

​

Vendor A

Internal lock for OEM

Thumbscrews for the rest of market

Able to support all flavors of OCP NIC including thumbscrews. Give the flexibility to customer

​

​

Customer D Wearing OCP HAT

Which form factor can drive most adoptions?

​

​

FLW adjusted the gaps to squeeze in tubing manifold or boot E1.S

While unable to support dual-thumb screws, it support dual internal lock or dual ejector latch

Trending

10 of 25

Type-7 / Class D MHS feedback

  • Custom zone 1 on SDNO Class-D has been updated to cover OCP NIC on the right
  • We can submit a motion to update Custom Zone 2 reference to be D-SFF once we aligned and closed on connector pitch

11 of 25

D-SFF Face-to-Face during OCP summit

10/17 (Tuesday, during OCP summit) in San Jose Convention Center 4-5pm

OCP Foundation is glad to sponsor the room and I will share more when locked down the room info

Interested parties so far:

  • Broadcom, Dell, Intel, Nvidia, Meta

12 of 25

D-SFF future work list

Must Have:

  • Something to manage 1 DSFF vs 2 SFF
    • Primary, secondary
  • Manage diverse lane mappings (more than LFF definition)
    • Rootport, endpoint mixed usage
  • Insertion loss for Gen6 D-SFF
    • -6dB trend is only for SFF
  • Ejection mechanism
    • Latch mechanism on both side?
    • Ejector rail reference?
  • Card height
    • SFF, TSFF, 1RU-DSFF?

​

​

Preferred:

  • Pin-out stayed identical
  • Accurate Timing Sync feature
    • PPS signal

Must Have:

  • Mechanical tolerance
  • Pin-out discussion
    • Direction of PERST and CLK
    • Flex pins

​

​

13 of 25

14 of 25

Gather feedback throughout the year

Provide your feedback at:�https://tinyurl.com/feedback2ocpnic

15 of 25

SIWG for PAM4 Gen6 spec

Xtalk spec to meet BER requirement

    • SFF-TA-1002 proposal(rev 1.4.1) in progress, will leverage
    • SFF-TA-1009 proposal(rev 3.1.1) for device spec
    • WIP to check for channel simulations
    • Working with device vendor to meet Rx spec

Anthony/Tanmoy

Connector Spec

16 of 25

Channel Simulation

Based on Gen5 models as same Nyquist for Gen6

    • Root complex/Switch ~ 20dB + Package model
    • AIC/OCP ~7dB + package model
    • Tighter PSNEXT proposal (-60dB @16Ghz), ccICN comparison
    • Gen5 models are close to meet the spec

​

Anthony/Tanmoy

Straddle Mount

AIC/OCP

17 of 25

Channel Gen5 vs Gen6

~32dB

~28dB

CPU

NIC

CPU

NIC

7dB

6dB?

Gen 5

Gen 6

18 of 25

SI Test Fixtures

CLB 5.0 32GT/s status:

    • Dell to present at IC meeting to release design to Wiki

​

CBB 5.0 32GT/s status:

    • Multi-vendor testing results within predicted performance, suitable for CEM equivalent testing
    • Dell to present at IC meeting to release design to Wiki

OCP NIC 3.0 SFF CLB 5.0 prototype

Calibration

CLB board 1

Impedance Coupon

CLB board 2

Jason

OCP NIC 3.0 SFF CBB 5.0 block diagram

19 of 25

OCP NIC 3.0 ejection mechanism

Customer A

Customer B

Customer D

Customer C

Customer X,Y,Z

20 of 25

D-SFF Option 1: FLW rev1.0 SFF recommendation

thumbscrew possible

No thumbscrew

  • Dimensions follow M-FLW rev1.0
  • Middle NIC card unable to support Pull-tab/Thumbscrew version
  • Opens: Does this dimension support “ejector latch” faceplate?

​

21 of 25

D-SFF Option 2: Define separation to fit Pull-tab/Thumbscrew

Thumbscrew support

?

  • Dimensions follow M-HPM Type4 (M-DNO4) and M-HPM Type7 (M-DNO7, aka wide 21” FLW)

​

Thumbscrew support

22 of 25

D-SFF option preference

Option 1: Follow M-FLW

Option 2: Increase gap support thumbscrews

Customer A

Required cost for surprise removal is high.

Platform incline to internal latch version

Width of faceplate of thumbscrew became a limitation trade-off for other system feature

Customer B

Embrace internal lock since the start of OCP

​

Customer C

Majority thumbscrews. Some internal latch is niche

​

​

Vendor B

Ejector for Customer D, Internal lock Customer A & B

Thumbscrews for channel ODM CSP partners

On website wide distributions is thumbscrews

​

​

Vendor A

Internal lock for OEM

Thumbscrews for the rest of market

Able to support all flavors of OCP NIC including thumbscrews. Give the flexibility to customer

​

​

Customer D Wearing OCP HAT

Which form factor can drive most adoptions?

​

​

23 of 25

OCP NIC 3.0 spec 1.2.1 PCIe mapping

Paul

24 of 25

Mixed mode LFF use case examples

 

PCIe Interface A

PCIe Interface B

#

0,1

2,3

4,5

6,7

8,9

10,11

12,13

14,15

16,17

18,19

20,21

22,23

24,25

26,27

28,29

30,31

1

Endpoint

Root Port

Root Port

Root Port

Root Port

 

REFCLK0

REFCLK2

 

PERST_N0

EXT_PERST_0

EXT_PERST_1

EXT_PERST_2

EXT_PERST_3

​

​

​

​

​

​

​

​

​

2

Endpoint

Endpoint

Endpoint

Endpoint

Root Port

Root Port

Root Port

Root Port

 

REFCLK0

REFCLK1

REFCLK2

REFCLK3

REFCLK4

 

PERST_N0

PERST_N1

PERST_N2

PERST_N3

EXT_PERST_0

EXT_PERST_1

EXT_PERST_2

EXT_PERST_3

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

3

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

Root Port

 

REFCLK0

 

EXT_PERST_0

EXT_PERST_1

EXT_PERST_2

EXT_PERST_3

EXT_PERST_4

EXT_PERST_5

EXT_PERST_6

EXT_PERST_7

EXT_PERST_8

EXT_PERST_9

EXT_PERST_10

EXT_PERST_11

EXT_PERST_12

EXT_PERST_13

EXT_PERST_14

EXT_PERST_15

​

​

​

4

Endpoint

Root Port

 

REFCLK0

REFCLK2

 

PERST_N0

PERST_N1

Paul

25 of 25

SDNO Class A to D