1 of 11

AI Data Centres:�Networking Problems and Solutions

Shraddha Hegde

Connections 2026

2 of 11

Agenda

  • Problem space
  • Traffic management
    • SRv6 deterministic forwarding
    • BGP Routing planes
    • MP-TE
  • Congestion mitigation
    • DCQCN and PFC
    • C-SIG
    • FANN WG at IETF
  • MRC (Multipath Reliable Connection)
  • RNG (Random Network Graphs)

3 of 11

Problem Space

AI Training workloads powered by collective communications

GPU-GPU training traffic

Frontier Models

High bandwidth

Low entropy

High cost GPUs

Training step dependency

Tail latency governs completion

Size of the models

Scale out limits

Scale across DCs

Multi-site training

Traffic Characteristics

Failure Sensitivity

Scale out/Scale across

Synchronous bursts

4 of 11

Traffic management solutions

Deterministic forwarding: SRv6 micro-sid

Micro-Sid stack on the NIC

Alternate paths on failure/congestion

draft-filsfils-srv6ops-srv6-ai-backend

​

BGP based routing planes

Micro-SID based SR policies within the planes

Per routing plane locators

draft-hss-bgp-srv6-routing-planes

​

Multi-path DAG, unequal cost load-bala

Reduced signalling and fwd state

Strategically bandwidth managed

Multicast for CCL

draft-kompella-teas-mpte/draft-kompella-teas-mcte

​

​

5 of 11

Congestion Mitigation

  • DCQCN:
    • E2E congestion control mechanism
    • ECN based regulation
    • Application driven fast ECN notification

​

​

  • Priority Flow control (PFC)
    • Ethernet based mechanism to create lossless network
    • Downstream congestion for a traffic class triggers a

“pause” frame to be sent to upstream neighbors

    • Head-of-line blocking for all traffic of same priority class
    • Un-related traffic through congested link may be victimized

6 of 11

C-SIG

  • Layer 2 tagging, 4 byte or 8 byte tag
  • Different signal types
  • In-band signalling
  • High resolution observability in sub-ms granularity
  • Low overhead and cost
  • Efficient backward compatibility mechanisms
  • Being standardized in UEC

7 of 11

FANN WG

  • Origin
    • FANTEL (Fast Network Telemetry) BOF in IETF 123
  • Motivation
    • Fast detection, signalling and mitigation of congestion
  • Scope
    • Problem space, Requirements, gaps
    • Framework for network notification
    • link failures, signal degradation reported as link errors, and port output queue congestion
    • ensuring extensibility for additional conditions in the future
    • Primary focus on DC/DCI networks

​

8 of 11

Multi-path Reliable Connection

  • Topology
    • Two tier, multiplanar
    • Reduced switch hops
    • Reduced bw exposure to failure

​

  • Deterministic fwding
    • SRv6 uSID based pinned paths
    • No dynamic routing
    • Static source routing

​

  • MRC Protocol
    • NIC based, packet spraying
    • Reordering efficiently
    • Best effort ethernet , no PFC
    • Recovery based on packet trimming and selective retransmission
  • Outcome
    • Outperforms ROCEv2 at higher message size and random packet loss scenarios

9 of 11

Random Network Graph

  • Limitation of FAT Trees
    • Bottlenecks: Multiple Tiers causes expensive hops
    • Fragility: Top layer failures have no redundancy
    • Inefficiency: scaling requires entire tier to be added, expensive
  • Key Innovation
    • Shuffle box: Physical hardware to make connections, internally wired fiber optics
    • Spray point routing: spray to neighbors and then SPF to way points, much larger set of paths
  • Benefits
    • 69% fewer routers
    • 33% higher throughput
    • 40% reduction on power
    • Losing 1 percent routers reduces 1 percent capacity and is not catastrophic as in fat trees

​

10 of 11

References

​

  • MRC

https://arxiv.org/pdf/2605.04333v1

  • RNG
    • https://arxiv.org/pdf/2604.15261

11 of 11

THANKS

shraddha.hegde@hpe.com