SDN is the easy part: Production OpenFlow at SC18

Nick Buraglio

ESnet Planning Team

Lawrence Berkeley National Laboratory

Brad Cowie

Waikato University

TNC19

06/21/2019

Timelines and high hopes

A Definition of “SDN”

• What is SDN?
• How has it been realized?
• What filled in the gaps?

​

Reality sets in

• 99% of networks are not

a hyper-scaler

• Developers who know

networking well are rare

• Network Engineers who write

production code are also rare

• Many products were/are not

production ready

• Many products are not “to market”

The technology is largely irrelevant

• “SDN” is technology progression like any other
• Difficulty lies in a mental and culture adjustment
• Change comes with social baggage
• Technological adaptation is precise
• Socialization and culture change is complicated and messy
• Change takes time

​

Brownfield integration is even more intricate

• Greenfield deployments are easier*
• Brownfield deployments:

* May have poor or no integration

with existing process, procedures

* Often operate as an island

* May cause support

discrepancies

None of these are truly technical problems

​

​

Technical debt is a hidden limiting factor

• Requirements for integration with existing systems
• Lack of a source of truth for network information
• Unclear business logic surrounding technical processes
• Technical sprawl and lack of consistent workflow can cloud needs and requirements processes
• Technical bias can cause dramatically skewed needs and requirements

Case study: SC18 edge as an SDN network

​

Needs and requirements:

• Works with existing database*
• Low touch
• Feature complete
• Low overhead
• Redundancy options
• Supportable

​

* Middleware required

​

​

​

SC18 Faucet

Network

noc-rtr-2

​

​

​

Cisco

NCS5500

noc-faucet

​

NoviFlow 32x100G Switch

with Barefoot Tofino ASIC

noc-rtr-1

​

​

​

Juniper

MX10008

faucet-controller

Control Plane Network

2x100G

LACP

10G

NFV

100G

dci

Cisco

C9500-32C

dci

Allied Telesis

x950-28XSQ

dci

Cisco

C9500-48Y4C

dci

Allied Telesis

SBx908Gen2

dci

NoviFlow

2122

Faucet provides:

• VLANs
• IPv6�Router Advertisements
• Inter-VLAN Routing
• Network Security Policy

NFV Services:

• DHCP
• BGP

SC18 edge as an SDN network

What problem are we trying to solve?

​

• Minimize human complexity from the configuration process
• Implement a centrally controlled, multi-vendor network
• Prove faucet as a large, fast, production system
• Seamless integration with traditional network

​

Stretch goals (all met):

• NFV all routing protocols
• Operate independently
• 0% failure rate due to configuration of the system as a whole

​

​

​

​

FAUCET Configuration

YAML based configurations

dps:

'dnoc1034-faucet':

dp_id: 0xe01aea43e46f

hardware: 'Allied-Telesis'

drop_spoofed_faucet_mac: false

arp_neighbor_timeout: 300

ignore_learn_ins: 0

lacp_timeout: 60

metrics_rate_limit_sec: 5

nd_neighbor_timeout: 300

timeout: 900

interfaces:

1:

name: '1'

description: 'Xilinx (Booth 927)'

native_vlan: '927-A-l2'

max_hosts: 100

acl_in: protect_from_access

​

​

​

​

< - Define datapaths

< - Define interfaces

< - Define ACLs

FAUCET advantages

• Vendor options are clear and well defined
• Existing integration with SCinet database
• Previous successful deployment
• Low overhead of provisioning
• “Single file” configuration
• Integrated telemetry
• Pre-deployment and staging of entire access layer is minimally difficult
• Similar to zero touch provisioning.
• Native configuration linter

​

​

FAUCET Telemetry

ESnet production office graphs

​

FAUCET considerations

• All work with the existing database needs to include FAUCET hooks
• Assumes Enterprise style networking - i.e. VLAN tagging (no MPLS, etc.)
• Requires stable control plane network before services are provisioned
• Vendor options are more limited (but do we really need that many edge port vendors?)
• Allowing all access layer devices to be FAUCET controlled minimizes deployment time *significantly*

We made this work - you can too!

• Understand the needs and requirements
• Understand the landscape
• Detail work up front makes less work later
• Don’t obsess about the technology - the first three points should dictate it

​

Conclusions

• Successful SDN deployments are actually very straightforward
• Integration with existing workflows and systems are the largest part of the process
• Cultural shifts take time
• Sifting through vendor marketing is important
• The technology is typically straightforward

​

Useful Links

https://github.com/wandsdn/sc18-faucet-configs

SC18 Faucet Podcast on IPSpace.net

ESnet-CU office graphs

​

​

Nick Buraglio - buraglio@es.net

​

​