School of Computing Science

Simon Fraser University

​

​

CMPT 471: Networking II

​

SDN Case Studies

​

Instructor: Mohamed Hefeeda

1

SDN Deployment

• Cloud providers, e.g., Google, Microsoft, and Facebook, have been using SDN for:
• Managing datacenter networks: SD-Fabric, and
• Optimizing inter-datacenter links: TE for WAN
• Examples: Google B4, Microsoft SWAN 
• Major ISPs, e.g., AT&T, Comcast, and Bell, are gradually deploying SDN for:
• Implementing traffic engineering (replacing MPLS), and
• Managing network functions (e.g., FW, LB, IDS)
• Enterprises, e.g., companies and universities, are gradually deploying SDN for:
• Creating and managing virtual networks, and
• Managing WAN traffic and services (SD-WAN)

​

2

SDN Case Studies

• We will present key case studies for SDN:

​

• Inter-Datacenter Traffic Engineering (or TE in WANs)
• Microsoft SWAN
• Google B4

​

• Network Virtualization
• VLANs
• VPNs
• SD-WANs

​

​

​

​

​

3

SDN: TE for WANs

• Major IT companies, e.g., Microsoft, Google, Facebook, have many datacenters around the world
• They are interconnected with expensive WAN links
• Carry huge amount of traffic, e.g., to sync web index, user data, …

4

Google Datacenters as of Jan 2018: 33

Expensive links

Source: Hong et al., B4 and After: Managing Hierarchy, Partitioning, and Asymmetry for Availability and Scale in Google’s Software-Defined WAN, SIGCOMM’18.

SDN: TE for WANs

• The utilization of expensive WAN links used to be kept low
• ~40—60%. Why?
• To ensure QoS in presence of high traffic variations

​

• Using SDN, Microsoft and Google were able to optimize their WAN links and reported utilization ~100%

​

• Let us see an example from Microsoft to understand better

5

SDN: TE for WANs

• Example from Microsoft: reason behind poor utilization

​

​

​

​

​

​

6

Source: Hong et al., Achieving High Utilization with Software-Driven WAN, SIGCOMM’13. (Microsoft SWAN)

Provision for peak 🡺 < 50% utilization

Smaller peak 🡺 better utilization

Adapt using SDN

Aggregate traffic

SDN: TE for WANs

• Why SDN was able to achieve much better utilization?
• Because it performs “global” traffic optimization instead of local optimization as before
• Illustrative example:
• Every link has 1 unit capacity in each direction
• We have 3 flows A, B, C going to node 5 from nodes 1, 2, 3

7

Distributed: each source chooses its local optimal

A

B

C

Central optimization using SDN 🡺 global optimal

A

B

C

SDN: TE for WANs

• Architecture of Microsoft SWAN (SDN WAN)

​

​

​

​

​

8

Source: Hong et al., Achieving High Utilization with Software-Driven WAN, SIGCOMM’13. (Microsoft SWAN)

SDN: TE for WANs

• Required Reading
• Chapter 2 of Software-Defined Networks: A Systems Approach
• For more information (optional reading):
• Paper from Microsoft on SWAN:
• Hong et al., Achieving High Utilization with Software-Driven WAN, SIGCOMM’13.
• Introduces the SDN WAN ideas and early implementation at Microsoft
• Paper from Google on B4 network
• Hong et al., B4 and After: Managing Hierarchy, Partitioning, and Asymmetry for Availability and Scale in Google’s Software-Defined WAN, SIGCOMM’18.
• Describes multiple generations of B4 and provides details on various protocols

​

9

Network Virtualization

• Network virtualization is an old concept
• But has substantially evolved with SDN and datacenter networks
• Network virtualization is a highly overloaded term
• Means different things to different people
• Here are some common terms/concepts …
• Virtual Circuits (VCs)
• Virtual LANs (VLANs)
• Virtual Networks in Datacenters (L2/L3 virtualization)
• Virtual Private Networks (VPNs)
• Virtual Switches
• …
• Let us start the virtualization story from the beginning …

10

Virtualization in Computing

• Virtualization is a fundamental, old, concept in computing
• Virtualization ~= abstraction

​

• Virtualization has many benefits, e.g.,
• High resource utilization (via sharing and multiplexing)
• Better security (via isolation)
• Easier programming models (via common abstractions)
• Efficient resource deployment (via software instead of hardware)

​

11

Two Common Virtualization in Computing

• Virtual Memory
• Abstraction of physical memory
• Physical address is different from virtual address (used by process)
• mapping between virtual and physical space is needed
• Allows memory sharing (better utilization) and offers simple address space (easier programming)

​

​

12

Two Common Virtualization in Computing

• Virtual Machine (VM)
• Complete virtualization of all components: processor, memory, storage, I/O, …

​

• VM Offers
• Better resource utilization
• Faster deployment of VMs compared to physical servers
• Better security (VM isolation)

​

​

​

13

e.g., VirtualBox

Two Common Virtualization in Computing

• Container ...
• Is a package of software with all necessary elements to run in any environment
• Shares OS resources
• 🡺 much lighter than VMs and runs faster
• Common in datacenters

​

​

​

14

e.g., Docker

Virtual LANs: Layer 2 (Ethernet) VLANs

• Example: create two separate virtual LANs
• H1, H3, H5: VLAN 100
• H2, H4: VLAN 200

L3 Router

L2 Switch

L2 Switch

To other IP subnets or Internet

H1

H2

H3

H4

H5

H6

VLAN 100

VLAN 100

VLAN 100

VLAN 200

VLAN 200

• Extension of the Ethernet standard to support VLANs

802.1 Ethernet frame

16 bits: Type

4 bits: Control

12 bits: Tag (VLAN ID)

Recomputed

CRC

802.1Q VLAN Frame Format

• This is old, not using SDN, but still deployed

L2 (Ethernet) VLANs

• Switches are configured to associate port numbers to VLANs
• By a controller
• Switches forward traffic of a VLAN to its hosts only
• Based on VLAN ID 🡺 Logical isolation

​

• L2 VLANs somewhat resemble Virtual Memory
• They offer basic sharing of physical links
• Good enough for traditional enterprise/campus networks
• Not sufficient for datacenters that host many tenants, with different workloads, services, and requirements

​

17

Network Virtualization: Modern Definition

• A virtual network is a lot more than just links
• It can contain switches, firewalls, NATs, load balancing, etc.

​

• We want virtual networks that resemble virtual machines

​

• A Virtual Machine offers complete server
• CPU, memory, storage, I/O, …

​

• A Virtual Network should offer complete network
• Switches, links, firewalls, NATs, topologies, addressing schemes, …

​

​

​

​

18

Network Virtualization: Modern Definition

• Virtual Network should offer/enable:
• Control of address space
• Allow using various address schemes (e.g., MAC, IP v4, IPv6 ) over the same physical network
• Allow migration of VMs from one subnet/rack/server to another
• Control of network topology
• Allow creating various topologies over the same physical network
• Control of network functions
• Allow creating/managing various network functions, e.g., FW, LB

​

• 🡺 All of the above are critical for multi-tenant datacenters
• Let us see what datacenters looks like to understand the demands

​

​

19

Datacenters: Common Design

Server racks

• 20- 40 servers
• Each runs multiple VMs

Top of Rack (TOR) switch

• one per rack
• 40-100Gbps

Tier-2 switches

• connecting to ~16 TORs below

Datacenters: Multi-Tenants

Tenant 1:

Star topology, IPv4

100 servers

10.1.1.100

10.1.1.1

10.1.1.2

Virtual links (tunnels) between VMs

Server failed 🡺 VM migrated but kept same (virtual) IP address

Datacenters: Multi-Tenants

Tenant 2:

Mesh topology, 3 switches, IPv6, FW, LB, 10 servers

Tenant 2 network will be overlayed over the same physical network

Tenant 1 network

Network Virtualization: Basic Architecture

• Network Virtualization App (on top of SDN controller)
• Has global view of the whole network
• Gets requirements of the virtual network (through GUI or APIs)
• Creates flow rules to realize the requested network
• Sends the flow rules to devices
• Tracks network changes and updates rules to reflect them

23

Virtual Networks: Building Blocks

• Tunnelling
• Decouples virtual addresses from physical ones
• Done through Encapsulation: putting a packet inside another

24

Seen and processed by physical network

Added and removed at start/end of tunnel

Seen and processed by vSwitch and VMs

Virtual Networks: Building Blocks

• Virtual Switch (vSwitch)
• Implemented in software
• Offers similar functionality/interface as physical switch
• Essential to optimize for high-speed packet processing, otherwise becomes a bottleneck
• Common example: Open vSwitch (OVS)
• OVS has many implementation optimizations

​

• See the following paper for more details on OVS:
• Pfaff, et al., The Design and Implementation of Open vSwitch, NSDI 2015

​

25

Virtual Networks: Building Blocks

• Example optimization in vSwitch:
• Physical switch has TCAM, which performs prefix matching and supports wild cards at high speed
• But this is slow to implement in software
• vSwitch Solution (guess?):
• hint: a connection typically has �many packets, all use same patterns
• 🡺 do expensive matching once and �re-use/cache for following packets
• First packet:
• perform prefix matching in user space
• Compute all possible matches and install them in cache in kernel space
• Subsequent packets
• Use cache in kernel space

​

​

26

Virtual Networks: Distributed Functions

• Consider a firewall (FW) as an example network function
• In traditional approach (before SDN), FW is at one location (ingress point) and traffic should go through it
• Suppose VM A is sending to VM C: needs to go through FW
• Extra delay, FW overloaded, may become a bottleneck
• Extreme: VM A is sending to VM B (aka hairpinning)

​

27

Virtual Networks: Distributed Functions

• SDN enables “distributed” network functions
• How would you implement a distributed FW using SDN?
• Controller computes all needed rules for each vSwitch
• 🡺 more efficient, less delay, less overhead, more scalable
• 🡺 good for datacenters as they need to support numerous virtual networks for their customers

28

Network Virtualization

• Required Reading
• Chapter 8 of Software-Defined Networks: A Systems Approach
• For more information (optional reading):
• Koponen, et al.,  Network Virtualization in Multi-tenan Datacenters, NSDI 2014
• Pfaff, et al., The Design and Implementation of Open vSwitch, NSDI 2015

​

​

​

29

Software Defined Wide Area Network: SD-WAN

• Consider a large organization with multiple sites/branches
• e.g., Royal Bank of Canada (RBC)
• Traditionally (and still widely used)
• RBC would lease virtual circuits (VCs) from ISP(s), e.g., Telus
• ISP uses MPLS (Multi Protocol Label Switching) to create VCs
• ISP provisions VCs to guarantee delay and bandwidth
• Disadvantages for RBC: slow to establish VCs, costly, ISP lock-in

​

30

Branch office

Main office

Corporate datacenter

ISP(s)

Software Defined Wide Area Network: SD-WAN

• SD-WAN: Recent (gradually being deployed)

​

31

Branch office

Main office

Corporate datacenter

ISP(s)

SD-WAN Controller

VPN Tunnel

SD-WAN: Advantages

• Fast deployment (Zero touch)
• Switches are shipped to sites with address of the controller
• Controller distributes all configurations including security certificates, all from one location 🡺 much simpler
• Cost effective, no leased VCs, no vendor lock-in
• Bandwidth on the internet is becoming abundant
• VPNs offer more security (through encryption)
• Will discuss more on VPNs later (security part)

32

SD-WAN: Advantages

• SDN switches can implement distributed network functions, e.g., Firewalls 🡺 more efficient
• Instead of always going to corporate datacenter for enforcing security policy (hair pinning)

​

• Easily integrate with and use cloud services, e.g., Office 365, Box, Salesforce, Azure/Amazon/Google cloud services, …
• Becoming popular and powerful

33

SD-WAN: Advantages

• SDN switches can implement distributed network functions, e.g., Firewalls 🡺 more efficient
• Instead of always going to corporate datacenter for enforcing security policy (hair pinning)

​

• Easily integrate with and use cloud services, e.g., Office 365, Box, Salesforce, Azure/Amazon/Google cloud services, …
• Becoming popular and powerful

34

• SDN for TE: Inter-datacenter traffic
• Global view of WAN 🡺 better traffic management 🡺 higher utilization
• Examples: Google B4 and Microsoft SWAN
• SDN and Network Virtualization
• L2 (Ethernet) VLANs
• Simple, link sharing, traffic isolation
• Complete Network Virtualization
• All elements, including topology, address space, network functions
• Important for datacenters
• SD-WAN
• Managing multiple sites over the Internet
• Efficient, cost effective, gradually deployed

​

​

35

Summary