1 of 34

Discussion 11

1

CS 168, Summer 2026 @ UC Berkeley

Slides credit: Sylvia Ratnasamy, Rob Shakir, Peyrin Kao, Iuniana Oprescu

Host Networking 🧑‍🍳

2 of 34

Problem

  • Datacenter requirements

3 of 34

Problem: Datacenter Requirements

  • Datacenter applications have extreme performance requirements
    • Ex. 100ms can cost lots of $$
  • Cores are valuable
    • In cloud context, any cycle you waste could have been rented out!
  • Kernel development is hard
    • If we are making changes to the networking stack, we might be stuck doing a lot of kernel development
    • Ideally, we could skip this entirely

4 of 34

Host Networking

  • Kernel Bypass
  • RDMA
  • Congestion Control
  • Load balancing
  • Traffic shaping
  • QoS

5 of 34

What is Host Networking?

  • Everything at the host that enables it to use the network
    • Linux networking stack, network driver, NIC, etc…

Host Networking Stack

Network Interface Card

Sender Application

6 of 34

Advanced Host Networking

  • Datacenter operators have the incentive to put a lot of effort into optimization
    • And have the means: total administrative control

​

  • Some opportunities for optimization
    • Kernel bypass (avoid kernel development)
    • NIC offloads (save CPU cycles)
    • RDMA (save CPU cycles, and better performance)
    • Other features (better performance)
      • Congestion control
      • Load balancing
      • Traffic shaping
      • QoS

​

7 of 34

Kernel Bypass

  • Run networking stack in user space rather than kernel space

​

​

App

OS

NIC

App

OS

NIC

User-Space Process

Shared Memory

vs

Data

Data

Data

Data

Data

Data

Data

8 of 34

NIC Offloads

  • What is offloading?
    • Implementing some tasks in hardware to free up CPU cycles
    • In our case, putting some networking tasks in the NIC instead of the kernel
  • Why offload?
    • Save CPU cycles
    • Performance gains

9 of 34

NIC Offloads

  • Range from simple (checksum computation) to advanced (protocol offload)

​

​

← Simple Complex →

Checksum

Segmentation

Match-Action

Protocol Offload

10 of 34

RDMA

  • Remote Direct Memory Access
  • Removes CPU from transfers (almost entirely)

​

​

Sender

CPU

NIC

Memory

Recipient

CPU

NIC

Memory

Without RDMA: CPU involved in data transfer.

11 of 34

RDMA

  • Remote Direct Memory Access
  • Removes CPU from transfers (almost entirely)

​

​

Sender

Recipient

CPU

NIC

Memory

CPU

NIC

Memory

RDMA: CPU is minimally involved in transfer!

12 of 34

RDMA - How?

  • Queue pairs
    • Send and receive queues with Work Queue Elements (WQEs) in them
  • WQEs
    • Pointer to memory to transfer / receive data in
  • Completion Queue and Completion Queue Elements (CQEs)
    • Signals about whether transfers have completed or not

​

RDMA NIC

Software

CPU

Memory

Queue Pair

Send

Receive

"Send this data."

"Put the data here."

13 of 34

RDMA Example

There are many different RDMA operations:

  • RDMA Write.
  • RDMA Read.
  • RDMA Atomic. Perform an operation in remote host's memory.
  • RDMA Write with Immediate. Send additional value along with writing data.

​

Let's look at an RDMA Send/Receive.

14 of 34

Steps of RDMA (1/6)

1. Each server designates some memory to be accessible by NIC for RDMA transfers.

Software

CPU

Memory

Software

CPU

Memory

RDMA NIC

Sender

RDMA NIC

Recipient

Readable by NIC.

Writable by NIC.

15 of 34

Steps of RDMA (2/6)

2. Each server sets up queues.

This can be done out-of-band, e.g. use TCP to coordinate between servers.

RDMA NIC

Sender

RDMA NIC

Recipient

Queue Pair

Send

Receive

Completion

Queue Pair

Send

Receive

Completion

Software

CPU

Memory

Software

CPU

Memory

16 of 34

Steps of RDMA (3/6)

3. Servers set up Work Queue Entries (WQEs) in the queues.

WQE contains pointer to buffer in memory.

RDMA NIC

Software

CPU

Memory

Queue Pair

Send

Receive

Sender

RDMA NIC

Software

CPU

Memory

Queue Pair

Send

Receive

Recipient

Completion

Completion

CPU writes WQE to queue.

CPU writes WQE to queue.

17 of 34

Steps of RDMA (3/6)

3. Servers set up Work Queue Entries (WQEs) in the queues.

WQE contains pointer to buffer in memory.

Software

CPU

Memory

Software

CPU

Memory

RDMA NIC

Queue Pair

Send

Receive

Sender

RDMA NIC

Queue Pair

Send

Receive

Recipient

Completion

Completion

"Send this data."

"Put the data here."

18 of 34

Steps of RDMA (4/6)

4. NIC transfers data between memory, without CPU involvement!

Software

CPU

Memory

Software

CPU

Memory

RDMA NIC

Queue Pair

Send

Receive

Sender

RDMA NIC

Queue Pair

Send

Receive

Recipient

Completion

Completion

19 of 34

Steps of RDMA (5/6)

5. NICs generate Completion Queue Entries (CQEs), and delete WQEs.

Software

CPU

Memory

Software

CPU

Memory

RDMA NIC

Queue Pair

Send

Receive

Sender

RDMA NIC

Queue Pair

Send

Receive

Recipient

Completion

Completion

Data successfully sent!

Data successfully received!

20 of 34

Steps of RDMA (6/6)

6. Applications read CQE to understand what happened to the transfer.

Software

CPU

Memory

Software

CPU

Memory

RDMA NIC

Queue Pair

Send

Receive

Sender

RDMA NIC

Queue Pair

Send

Receive

Recipient

Completion

Completion

21 of 34

Advanced Features - Congestion Control

  • Problem:
    • TCP is buffer-filling
    • Loss is a coarse signal
  • Solution: use delay as a signal
    • Challenging to get a correct measurement
    • Why is this approach better in a DC?
  • Note: this is not the only way to do advanced CC
    • Very active area of research!

​

if RTT < Target

increase cwnd

(Additively)

else

decrease cwnd

(Multiplicatively)

22 of 34

Advanced Features - Load Balancing

  • Problem: ECMP can have collisions that cause hotspots
  • Solution: repath based on congestion signals
    • Note: ECMP could place it badly again – repeat as necessary

​

​

23 of 34

Advanced Features - Traffic Shaping

  • Problem: Need to share bandwidth according to policy
    • Note: congestion control alone cannot do this
  • Solution:
    • Classify traffic and rate limit it
    • Use a time wheel because managing multiple queues is difficult

To NIC

Shaper

Socket Buffers

Timestamper

24 of 34

Advanced Features - QoS

  • Problem: Need a way to express how much of a resource a flow should get
  • Solution: priority classes and enforcement in the queue

​

​

bandwidth

Receiver

Senders

mid

lo

Switch

Packets in time slots in Egress Link

hi

1: buffer allocation

2: bandwidth allocation

Egress Link

weight

4

2

1

QoS

25 of 34

Worksheet

  • Multiple Choice
  • Software-Defined Networking
  • Network Virtualization Offload
  • RDMA

26 of 34

Question 1: Multiple Choice

27 of 34

Worksheet

  • Multiple Choice
  • Software-Defined Networking
  • Network Virtualization Offload
  • RDMA

28 of 34

Question 2: Software-Defined Networking

29 of 34

Worksheet

  • Multiple Choice
  • Software-Defined Networking
  • Network Virtualization Offload
  • RDMA

30 of 34

Question 3: Network Virtualization Offload

31 of 34

Question 3: Network Virtualization Offload

32 of 34

Worksheet

  • Multiple Choice
  • Software-Defined Networking
  • Network Virtualization Offload
  • RDMA

33 of 34

Question 4: RDMA

34 of 34

Questions?

Feedback Form: https://tinyurl.com/cs168-su26-disc-feedback