OCP NIC Community Update�May 2024

�����Sub-group Project wiki: https://www.opencompute.org/wiki/Server/NIC�

Mailing List: https://ocp-all.groups.io/g/OCP-NIC

Agenda

• Specification updates & DSFF power discussion
• Signal integrity updates
• Mechanical updates & discussion

LFF Deprecated

• LFF moved to appendix (Chapter 9)
• Deprecated form-factor
• LFF content will no longer be maintained
• Content for reference only

​

​

Power Slot Envelope Definition

• Combined baseboard power slot for SFF and DSFF
• SFF Cards continue to use existing power slots
• DSFF cards may use SFF power envelopes + 160 W case

​

• Query card power requirements in ID Mode from 0xC0 OEM Record

​

• Need discussions around power balancing between Primary and Secondary Connector

​

​

Edits made to draft 1.4.5

Next Steps

• Declare draft 1.4.4 as the electrical Release Candidate
• Future updates for Mechanical/Thermal/SI (Gen 6)
• Allows for DC-MHS M-FLW/M-SDNO specs to move forward

​

• Notify e-mail distribution lists of intent:
• OCP NIC + entire workgroup
• OCP Server
• DC-MHS
• DC-SCM

​

​

Latest Draft Spec

See wiki page:

https://www.opencompute.org/w/index.php?title=Server/NIC#Working_Draft_Docs

​

Note: Refer to version 1.4.3 for the bulk of the spec changes; version 1.4.4 only contains deltas from 1.4.3.

SI Progress

• PCIe Gen 5 DSFF channel budget aligned to SFF, included in the latest draft spec
• Insertion loss limit: -7.0 dB @ 16 GHz for DSFF This limit is tight!
• Intel demonstrated 0.5dB margin in simulation if AC caps can invade into the mylar KOZ region
• Recommend ME workstream investigating this feasibility to gain this SI margin

​

• Workstream has shifted focus to PCIe Gen 6 (includes both SFF and DSFF)
• OCP NIC should set limit criteria in similar fashion on Insertion Loss, Return Loss, PSNEXT, PSFEXT
• Initial goal is not to create new conformal testing, informative limits beyond CEM
• Notables:
• SNIA has published TA-1002 Gen 6 limits on April 29th
• PCI SIG to ratify their limits very soon

​

• Dell has kicked off DSFF CLB/CBB SI test fixture designs, Gen 6 by default
• Will include dual SFFs, same MMCX connectors used with Gen 5 fixtures
• Schematics nearing completion and will be provided in the OCP NIC cloud account soon

Intel simulated DSFF board route

• Note this is an initial representative routing study with 32 lanes
• -7dB insertion loss target assumes ultra low-loss material and adherence to strict board routing to control loss
• Blue rectangular regions indicate the secondary side pkg keep out. PCIe AC caps placed north of this zone
• Route assuming PCIe region of ASIC is centered, this may not necessarily be the best position for thermals
• PCIe routes do not account for nearby heatsink mount holes

Mechanical Progress

• D-SFF interference in 1U rack with HH (LP) cards above
• Dell confirmed D-SFF is obstructed/captured in this scenario
• Need study results from other system vendors
• Appears there would be multiple KOZs required to alleviate this interference
• Negative impact to EMI?
• Negative impact to thermal solution?
• Negative impact to connector placement?

Mechanical Progress (Cont.)

• D-SFF and TD-SFF internal faceplate initial designs completed
• CAD files and drawings out for review

​

• D-SFF board assembly and KOZs drawing completed
• Drawing out for review

​

Mechanical Progress (Cont.)

• Bottom side insulator is under investigation to enable AC caps closer to gold finger edge
• Move to metal backing plate option?
• Can existing plastic insulator be adhered to caps directly without SI issue?
• Consider replacing mandatory plastic insulator with different options, only highlighting requirement that bottom side components are somehow protected.

​

Manageability

• Use USB 2.0 for DSFF manageability
• Routed on Primary Connector
• Declared in 0xC0 OEM Record
• USB CDC-EEM manageability type was introduced in version 1.3.0
• Less signal integrity challenges compared to RBT
• Not mandatory for DSFF

​

• SFF can continue to use RBT
• Not mandatory for DSFF
• Also add USB CDC-EEM manageability to SFF

Manageability, continued

• RBT changes
• DSFF RBT timing budget is 900 ps (align w/SFF)
• Suggest minimum RBT route lengths as well
• NIC (SFF): Min 350 ps (~2 in.) / max 900 ps
• DC-SCM: Min 350 ps (~2 in.?) / max 680 ps
• Add additional text to urge system vendors to minimize TSKEW; regain timing margin
• OCP NIC assumes 1500 ps (max) skew, worst case per DSP0222
• CLK/DATA paths on each individual card HPM/NIC/DC-SCM likely similar lengths
• Easier for HPMs to minimize TSKEW.

​

TDCSCI = 110 ps, max

TNIC_CONN = 110 ps, max

​

Model doesn’t include buffers, or isolation logic propagation delays

​

Spec edits in progress

• Edits still in progress
• Electrical section is in fairly good shape
• Mechanical needs update
• Signal integrity needs update

​

• Suggest releasing “1.4.x” Release Candidate spec with electrical updates
• Target end of April 2024
• Accept tracked changes (after review)
• Mechanical / SI portions to follow, clean slate for follow on changes

SI Workstream Goals

• Align DSFF to PCIe^® Gen 5, 6 limits or modify for OCP form factors
• OCP NIC 3.0 card channel budget defined from gold finger edge to end point silicon
• This workstream should factor the following use cases
• Consider DC-MHS HPM/motherboards as well as vendor specific (non-OCP) systems
• Slots shall be backward compatible to SFFs
• Various SFF & DSFF configurations (ex. 2 SFFs <->1 DSFF -vs- purpose build for DSFF only)
• 1^st DSFF NICs may use PCIe Gen 5 ASICs, with PCIe Gen 6 DSFFs approaching fast

Disclaimer: Current working material restricted by the OCP CLCA

SI Call to Action

• GOAL: Release PCIe Gen 5 limits into the next v1.4 draft spec by 4/9
• Review working material and the workstream’s converging proposal for the PCIe Gen 5 loss limit target: -7.0 dB @ 16 GHz for DSFF
• Once Gen 5 is drafted, the workstream will shift focus onto PCIe Gen 6 studies

Mechanical Update: Board Outline

• Board Outline Change
• Minor change to board outline dimensions
• Updated files posted to wiki 4/1

Mechanical Update: Help Needed

• Request all hands on deck for next mechanical sync
• 4/8, 9am PST
• Need to close on direction for when there is interference in 1U chassis with half height PCIe card above OCP NIC
• FLW and SDNO both have this problem

Insulator Keep out Zone

• Insulator keep out zone is clearly defined in the spec today – 14.06 mm from edge of PCB
• PCIe AC caps far away from board edge. Not good use of space for PCB.

​

Request

• Mechanical WG investigate reduction of secondary side insulator adhesive keep out zone.
• Minimize attachment points to “keyway” areas only? (in red)

​

Mechanical workgroup meeting 2/26/2024: 14.06 mm keep out defined as-is for potential for vertical card tilt on insertion. Prevents secondary side components from being sheared off.��Potential to define tighter rail guide tolerances or limit maximum component height in this region (e.g. 0402’s only). Needs follow up analysis.

Latest Draft Spec

See wiki page:

https://www.opencompute.org/w/index.php?title=Server/NIC#Working_Draft_Docs

Back up slides

DSFF Power Topologies

• Keep single +12V_EDGE power domain
• Management on Primary Connector
• Shared board side power supplies
• Reduced power supply area
• BOM cost reduction
• As-is in spec today

​

• Remove split +12V_EDGE from card side
• Duplicated NIC rail voltages
• Additional BOM cost
• More complex baseboard sde implementation

Examples only. One or more network controllers permissible for each power domain.

DSFF Card Edge Downshifting

• Use single DSFF card outline
• Card edge features must be present
• Gold pads may be reduced if not using all x32 lanes / cost savings.
• OK to detect DSFF with PRSNTB[0:3] when plugged into single connector HPM
• Use all PRSNTB[0:7] pins in dual conn HPM
• Re-evaluate impact on connector if pushing against PCB material vs gold pad.

​

• No card edge feature downshifting in DSFF
• SFF / LFF allowed this

LFF Content - Move to Appendix

• LFF has not been adopted by major baseboard and NIC vendors.
• Suggest moving LFF content to Appendix

​

LFF has been in the spec since the initial drafts.

​

Mechanical workgroup meeting 2/26/2024: OK to move to appendix.

Electrical workgroup meeting 2/26/2024: OK to move to appendix.

​

Move to appendix after March 6th public meeting

Mechanical tolerance analysis update

TE completed tolerance analysis similar to Amphenol's

• Analysis included true position tolerance as part of analysis, DPPM varied slightly due to this but still in agreement with initial analysis

​

• TE analysis agrees with Amphenol recommendation to increase AIC slot width from 1.85mm to 2.1mm

​

Mechanical workstream call to action

• Workstream participation has been varied
• Need commitment and help to complete key tasks
• Owners need for:
• Face plate creation
• Internal latch
• External latch
• Detailed drawings
• Thermal model
• Thermal test fixture
• Shock and vibe test fixture