ODL Summit

Seattle - Sep 2016

NetVirt - Advanced Tutorials

Alon Kochba (alonko@hpe.com)

Josh Hershberg (jhershbe@redhat.com)

Sridhar Gaddam (sgaddam@redhat.com)

Aswin Suryanarayanan (asuryana@redhat.com)

Victor Pickard (vpickard@redhat.com)

Introduction

• Goal: High level understanding of the packet-forwarding pipeline, with a deep dive into some of the services.

​

We’ll get back to this later..

• Basic OpenStack scenarios, and their implementation when using OpenDaylight NetVirt.

​

• Deep dive into OpenStack network types, L3 routing, ACL and IPv6 services implementation, with a focus on OpenvSwitch flows forwarding logic.

​

• Overview of L2 Gateway integration.

​

​

OpenStack Networks

(and Provider Network Types)

VxLAN Networks

Provider Network Type: VxLAN

​

• This means that any traffic between compute/control nodes OpenvSwitch will be encapsulated in VxLAN, and sent to the correct destination.

• This packet is sent through a predefined tunnel port (eth0), configured on the machine, and passed to ODL via OVSDB other_config:local_ip.

​

​

• ​

When a user creates a new network, the default network type used is VxLAN.

ITM (Internal Tunnel Manager)

Configuration Modes

​

• Automatic Mode - tunnels on demand (default):
• Creation of VxLAN tunnel endpoints in OVSDB are configured on demand by NetVirt, by defining transport zones for every network.
• This allows a minimal mesh of connectivity between compute nodes, and high scalability of the number of compute nodes.

​

• Manual Mode - pre configured tunnels via REST API:
• Automatic tunnels creation can be disabled via a configuration file, before starting OpenDaylight.
• When configuring this it is important to preconfigure the transport zones via REST API, otherwise there may be missing rules and connectivity might break.
• Config File: <odl_dir>/etc/opendaylight/datastore/initial/config/ netvirt-neutronvpn-config.xml (set to false)

ITM - Automatic Tunnel Creation

​

• When a network is created with DHCP enabled, it triggers the creation of a neutron port on the first node - in this case tap5fd..
• A new transport zone is automatically created - only the first compute (DPN IP 10.0.42.161) is inserted into the transport zone, since the qdhcp neutron port is created on it.
• No VxLAN tunnel is created.. yet..

Boot a VM, automatic tunnel creation

• Now if we boot a VM on the second compute (10.0.42.162), using the private-net, it is automatically inserted into the transport-zone.
• This triggers creation of VxLAN tunnel ports in OVSDB, with OpenFlow rules directing to this port when required.

OpenFlow rules

• Let’s take a look at some of the rules created, referencing this VMs’ private-net port MAC address (fa:16:3e:fe:2a:38)

• The first devstack is configured with a rule to forward traffic destined to the VM port MAC address through the tunnel port (this is part of the ELAN a.k.a L2 service).
• We can see that this tunnel port received the OpenFlow port 3, and the tunnel ID is set with a unique identifier allowing quick processing on the receiving side.
• The second devstack, which is running the compute node with the VM on it, has two sets of rules:
• The first directs traffic destined to the VM MAC, to the VM port (through an indirection table 220)
• The second is an optimization rule that matches on traffic arriving from the VxLAN tunnel with ID 0x4 - sending it directly to the VM port.

Flat and VLAN

Provider Networks

Flat/VLAN Provider Network Overview

• A pre-existing network, not managed by OpenStack
• First tell OpenStack/NetVirt how to connect to the pre-existing network
• ...using OpenVSwitch schema’s Open_vSwitch table - in the other_config column, “provider_mappings=<phys_net>:<interface>”
• Devstack will configure OVS for you per local.conf: “ODL_PROVIDER_MAPPINGS=<phys_net>:<interface>”
• Provider networks are often used for:
• External network access
• East/West traffic (between OpenStack nodes) over an existing network infrastructure

Flat/VLAN Provider Networks

Provider Network Type: Flat/VLAN

​

OVS encapsulates any traffic between compute/control nodes in a VLAN (for VLAN network type), or sends without tagging (for Flat network type).

The packets exit via the “Physical Network”.

​

Let’s create a VLAN network.

​

Note two new mandatory parameters:

• Physical Network - this is a logical name, mapped to the interface on the compute node via other_config.
• Segmentation ID - this is the VLAN ID configured on the network.

When creating a new network from the Admin tab, the provider network type can be chosen.

Flat and VLAN provider networks

• The provider physical_network name we provided when creating the network is mapped to an actual interface on the compute node via OVSDB other_config:provider_mappings.
• This port may have a different name across compute nodes, but it is assumed all compute nodes are connected to this physical network. Thus, packets from this physical network are never sent back to the network unchanged (split horizon).

• Packets received on the provider network with the correct VLAN ID will be stripped of VLAN tagging, and inserted into the pipeline.

... table=0,in_port=1,dl_vlan=1010 actions=pop_vlan, write_metadata:0x20000000001/0xffffff0000000001, goto_table:17

• Packets outputted to this provider network will be tagged with the VLAN ID.

... table=220,reg6=0x200 actions=push_vlan:0x8100, set_field:5106->vlan_vid, output:1

Provider Mappings - Advanced Use Cases

• In the previous example, we configured:

other_config:provider_mappings=physnet1:eth1

​

This causes eth1 port to be added to the br-int bridge.

(screenshot 1)

​

• However, for some more advanced topologies, sometimes a bridge is already preconfigured by the user.

NetVirt supports providing a bridge name instead of eth1, by connecting the bridge via a patch port to br-int.

​

other_config:provider_mappings="physnet1:br-dpdk0"

​

• An example where this is required, is using OVS-DPDK, and having both VxLAN and Flat/VLAN network traffic arriving from the same port. In this case we need to preconfigure a bridge containing the dpdk0 port with NORMAL output (L2 switching), to handle VxLAN, and adding a patch port to that bridge solves both scenarios (screenshot 2).

Routing

Routing

• Routers can be configured to provide connectivity between OpenStack networks.
• NetVirt always implements DVR (Distributed Virtual Router), and all the routing logic is done by OpenFlow rules.
• This approach has many benefits compared to the native OpenStack usage of linux namespaces (qrouter).

Two networks are defined.

Two VMs are booted, each with an interface on a separate network.

A router is defined, with an interface in each of these networks.

Routing between networks

• Let’s observe the flow of a packet from 10.0.123.4 (private-net1) to 10.0.124.5 (private-net2):
• VM1 will send an ARP request to 10.0.123.1 (router interface), which is forwarded to ODL via OpenFlow packet-in.
• ODL responds with an ARP response via OpenFlow packet-out.
• VM1 sends a ping with the destination MAC of the router interface and the destination IP of VM2.
• This packet is detected as destined to the router (table=19), passed to the routing table (table=21) and outputted to the correct VM port after replacing MAC addresses.
• Note everything done via OpenFlow instead of forwarding through linux qrouter - this provides many performance benefits, as the packet never leaves the OVS pipeline during the packet processing, until its final output.

External Networks

External Networks

External networks typically provide internet access to instances, via an OpenStack router.

By default, this network only allows internet access from instances using SNAT (N:1 NAT).

​

Internet access to individual instances can be enabled using floating IPs (1:1 NAT).

​

NetVirt supports multiple routers sharing the same external network.

• In this example, we configure a flat external network, connected to physical network “physnet1”.
• Then a subnet is configured for the network, with a gateway IP address and a floating IP allocation pool.

External Networks - Routers

​

• To connect the external network to instances, a router is used.
• This router has an external gateway set as the external network, and an internal interface in the private network(s).
• This configuration results in a topology similar to this, with two VMs running using private-net interfaces:

Floating IPs (1:1 NAT)

Connectivity originating from and to instances.

• Fully OpenFlow based forwarding allowing high packet forwarding performance.
• Floating IP rules are configured only on the compute running the VM.
• The VM only sees its private IP 10.0.123.4 - and the external network only sees the floating IP 10.64.99.4

10.64.99.4 <-> 10.0.123.4

SNAT (N:1 NAT, NAPT)

Connectivity originating only from instances.

• Centralized NAPT supported in Boron
• All packets are forwarded through the designated NAPT switch.
• ODL based NAT connection initiation
• The first packet of each connection initially passes through the ODL.
• Only supports TCP and UDP (no ICMP)
• Looking to support decentralized, fully OpenFlow based NAPT in Carbon.
• OpenFlow based NAPT requires nat conntrack features in OVS, not yet available in OVS 2.5.

Support is expected in OVS 2.6 - not certain about OVS-DPDK.

• Decentralized NAPT requires each compute having its own floating IP

​

Outbound Packet:

table=21, priority=10,ip,metadata=0x222e6/0xfffffffe actions=goto_table:26

table=26, priority=5,ip,metadata=0x222e6/0xfffffffe actions=goto_table:46

table=46, idle_timeout=300, send_flow_rem priority=10,tcp,metadata=0x222e6/0xfffffffe,nw_src=10.0.123.4,tp_src=36567 actions=set_field:10.64.99.2->ip_src,set_field:49152->tcp_src,write_metadata:0x222e0/0xfffffffe,goto_table:47

table=46, priority=5,ip,metadata=0x222e6/0xfffffffe actions=CONTROLLER:65535,write_metadata:0x222e6/0xfffffffe

​

Inbound Packet:

table=21, priority=42,ip,metadata=0x222e0/0xfffffffe,nw_dst=10.64.99.2 actions=goto_table:44

table=44, send_flow_rem priority=10,tcp,nw_dst=10.64.99.2,tp_dst=49152 actions=set_field:10.0.123.4->ip_dst,set_field:36567->tcp_dst,write_metadata:0x222e6/0xfffffffe,goto_table:47

table=47, priority=6,ip,metadata=0x222e0/0xfffffffe,nw_src=10.64.99.2 actions=group:200004,write_metadata:0x222e0/0xfffffffe

External Networks:

Under the Hood

The external network interface is represented as additional interface on the ELAN attached to the physical network

ELAN Instance

eth1

br-int

ELAN Interface

IETF Interface

ELAN (L2) model for networks - Flat

The external interface can be attached to provider bridge

In this case the attached ELAN interface will be the integration bridge patch port

ELAN Instance

dpdk0

br-dpdk

ELAN Interface

IETF Interface

br-int

br-dpdk0-patch

ELAN (L2) model for networks - VLAN

• Each ELAN network can be attached to one external interface per OVS node
• The external network interface ietf-interfaces model is marked with odl-interface:external augmentation
• This special demarcation is used to apply special behavior for provider network ports:
• Setting the Split-Horizon bit for ingress traffic from provider port to avoid network loops
• Adding provider ports to ELAN BC group
• Setting up DMAC table rules only on the local OVS node

External interface implementation details

VPN instance is created for each provider network. The external interface is represented as an additional VPN interface for this instance.

VPN Instance

eth1

VPN Interface

L3VPN model for external provider networks

​

Security Groups

Security Groups

​

• Model is based on ietf-access-control-list
• Port Security Extension Driver is supported
• Supports allowed_address_pair
• Stateful mode uses netfilter framework for connection tracking
• Pseudo stateful mode mode uses nicira learn action
• Anti-Spoofing rules are added by default
• Supports neutron CRUD operations
• Port range is supported via Nicira TCP/UDP port mask matches

Stateful Security Groups

Stateful - using OVS 2.5+ using conntrack features.

• Ideal solution for long term.
• Better performance, expected increase with future DPDK support.
• Packets are submitted to netfilter to track a connection.
• Connection is uniquely identified by layer-3 protocol, source address, destination address, layer-4 protocol and layer-4 key
• Uses ct_zone for overlapping ip addresses.

​

Potential issues to expect when using Stateful SG

• No conntrack support prior to OVS 2.5
• No OVS-DPDK support as of OVS 2.5
• Requires conntrack kernel modules when using vanilla OVS - sometimes problematic especially when using containers.
• Expecting full support in future (2.6?)

Stateful Security Groups

Implementation using OVS conntrack

• Below is an example of the stateful security group implementation for a VM having IP 10.100.5.3 with all ingress IPv4 traffic allowed. Egress IPv4 traffic is allowed from 10.100.5.4
• Ingress:
• Table=40 - any packet arriving from the VM port from the allowed address pairs is passed to the conntrack module for tracking and resubmitted to table 41
• Table=41 - new ipv4 connections are committed to the conntrack module, and is re-submitted to dispatcher table.(first packet in a new connection)
• Table=41 - any established/related connections are re-submitted to dispatcher table.(subsequent packets) Others are dropped
• Egress:
• Table=251 - any untracked packet(like traffic from a vm not connected to this ovs) arriving to the VM port from the allowed address pairs is passed to the conntrack module for tracking and resubmitted to table 41. Tracked packets are directly passed to 252
• Table 252 - new ipv4 connections from 10.100.5.4 are committed to the conntrack module, and is re-submitted to dispatcher table.(first packet in a new connection)
• Table=252 - any established/related connections are allowed, and other traffic is dropped.

table=40, priority=61010,ct_state=-trk,ip,dl_src=fa:16:3e:41:94:59,nw_src=10.100.5.3 actions=ct(table=41,zone=5000)

table=40, priority=0 actions=drop

table=41, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,17)

table=41, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,17)

table=41, priority=61010,ct_state=+new+trk,ip,metadata=0x20000000000/0x1fffff0000000000 actions=ct(commit,zone=5000),resubmit(,17)

table=41, priority=36007,ct_state=+new+trk actions=drop

table=41, priority=36007,ct_state=+inv+trk actions=drop

table=251, priority=61010,ct_state=-trk,ip,dl_dst=fa:16:3e:41:94:59,nw_dst=10.100.5.3 actions=ct(table=252,zone=5000)

table=251, priority=62020,ct_state=+trk,ip,dl_dst=fa:16:3e:41:94:59,nw_dst=10.100.5.3 actions=goto_table:252

table=251, priority=0 actions=drop

table=252, priority=62020,ct_state=-new+est-rel-inv+trk actions=resubmit(,220)

table=252, priority=62020,ct_state=-new-est+rel-inv+trk actions=resubmit(,220)

table=252, priority=61010,ct_state=+new+trk,ip,metadata=0x20000000000/0x1fffff0000000000,nw_src=10.100.5.4 actions=ct(commit,zone=5000),resubmit(,220)

table=252, priority=36007,ct_state=+new+trk actions=drop

table=252, priority=36007,ct_state=+inv+trk actions=drop

Learn Security Groups

Learn based - for OVS without conntrack support.

​

Uses an OVS action called “learn”, which is a Nicira extension to the OpenFlow spec.

This action allows configuring rules in the reverse flow automatically.

​

For example a packet pinging the IP destination 8.8.8.8, can automatically trigger a rule to allow traffic originating from the source IP 8.8.8.8

Learn Security Groups

Implementation using OVS learn action

• Below is an example of the stateful security group implementation with only OpenStack egress TCP port 80 traffic allowed (traffic from VM), after a VM sends an HTTP request to http://8.8.8.8
• Ingress (from VM port):
• Table=40 - any packet is re-submitted to both tables 41 and 42 (consecutively).
• Table=41 - no actions in this example.
• Table=42 - packets from the VM are allowed, and a reverse rule is configured in table=252 for future egress traffic to the VM, using the “learn” action.
• Egress (to VM port):
• Table=251 - any packet is re-submitted to both tables 252 and 253 (consecutively).
• Table=252 - traffic arriving to the VM is matched and allowed (by marking reg6=0x1), if it matches the original connection details (TCP/UDP matches on connections by port).
• Table=253 - packets marked with “allow” (reg6=0x1) are re-submitted to dispatcher - others are dropped.

table=40, priority=61010 actions=resubmit(,41),resubmit(,42)

table=42, priority=61010,reg6=0x1 actions=resubmit(,17)

table=42, priority=61010,tcp,metadata=0x30000000000/0x1fffff0000000000,tp_dst=80 actions=learn(table=252,idle_timeout=18000,fin_idle_timeout=60,fin_hard_timeout=60,priority=61010,cookie=0x6900000,eth_type=0x800,nw_proto=6,NXM_OF_IP_SRC[]=NXM_OF_IP_DST[],NXM_OF_IP_DST[]=NXM_OF_IP_SRC[],NXM_OF_TCP_SRC[]=NXM_OF_TCP_DST[],NXM_OF_TCP_DST[]=NXM_OF_TCP_SRC[],load:0x1->NXM_NX_REG6[0..7]),resubmit(,17)

table=42, priority=0 actions=drop

table=251, priority=61010 actions=resubmit(,252),resubmit(,253)

table=252, idle_timeout=3600, hard_timeout=3600, priority=61010,tcp,nw_src=8.8.8.8,nw_dst=10.0.123.3,tp_src=80,tp_dst=44291 actions=fin_timeout(idle_timeout=60),load:0x1->NXM_NX_REG6[0..7]

table=253, priority=61010,reg6=0x1 actions=resubmit(,220)

table=253, priority=0 actions=drop

​

IPv6

IPv6 Tenant Networks

ipv6-ra-mode: This attribute is used to specify if networking service should transmit the Router Advertisements.

ipv6-address-mode: This attribute is used to control how addressing is handled.

​

• Openstack Neutron provides APIs to create IPv6 subnets and associate it with neutron routers which enables IPv6 functionality in the VMs attached to those subnets.

​

• Neutron Subnet API allows us to create an IPv6 subnet with various addressing modes like SLAAC, DHCPv6-Stateless and DHCPv6-Stateful subnets using the attributes ipv6-ra-mode and ipv6-address-mode.

​

• Sample IPv6 tenant network:
• Create a Neutron tenant network.
• Create an IPv6 Subnet.
• Create a Neutron router
• Associate the router to the IPv6 Subnet.
• Spawn VMs in the network.

​

​

​

IPv6 Service

• IPv6 Control Path:
• Support Neighbor discovery for Router interfaces.
• Support periodic unsolicited Router Advertisements.
• Support Multiple IPv6 subnets per network.
• Implement Service Binding with Genius for IPv6 ports and program the necessary punt flows.

​

• IPv6 Forwarding: Any changes required to support IPv6 East-West and North-South communication would be done in ELAN and L3 service of NetVirt.

​

• IPv6 Security Groups: ACLService would be updated to support IPV6 Security Groups.

​

​

​

​

​

IPv6 Current Status

​

• IPv6 IPAM (SLAAC, DHCPv6 Stateful and DHCPv6 Stateless modes) are supported when used with Neutron dhcp-agent.
• Supports Neighbor Discovery for Router Ports.
• Supports Multiple IPv6 Subnets per network.
• IPv6 VM to VM communication on the same network is supported.
• IPv6 Stateful Security Groups along with allowed_address_pairs is supported.

​

Ping6: VM1 to VM2

​

Ping6 from VM1 to VM2 GUA

Ping6 from VM2 to VM1 LLA

IPv6 Roadmap

​

• Support IPv6 East-West and North-South Forwarding.
• Support IPv6 Provider Network use-case.
• IPv6 Underlay Support (i.e., infra on IPv6 network)
• IPv6 Prefix Delegation Support.
• Other advanced use-cases.

​

IPv6 High level architecture, gap analysis, current status and roadmap activities are captured at the following document.

https://docs.google.com/document/d/1siQUwfMi-Q9r4XcTLDkBIP01iL3PwQvFJ9UW-cC_jLg/edit

​

​

L2 Gateway

L2 Gw Basics

​

• VTEP - VXLAN Tunnel End Point
• Entity which originates and/or terminates VXLAN tunnels

​

• Hardware_vtep database schema defined in vtep(5)
• http://openvswitch.org/docs/vtep.5.pdf

​

• ODL uses OVSDB protocol used to configure hardware_vtep database on device
• Manager, Physical_Switch, Physical_Port, Ucast_Macs_Remote, Ucast_Macs_Local, Mcast_Macs_Remote, etc

​

Architecture

DC Infrastructure

OF + OVSDB

OVSDB

Neutron

Virtual Network* (VxLAN)

Virtual Network* (VxLAN)

Compute Blade

Compute Blade

VMs on compute connected to remote appliances and remote VMs part of same Virtual Network (VxLAN)

VN* (VxLAN)

HWVTEP

Servers/Appliances

VxLAN tunnel provisioning

MAC@ table population

Slide title

44 pt

​

​

​

​

​

​

Text and bullet level 1

minimum 24 pt

​

Bullets level 2-5

minimum 20 pt

​

​

​

​

Characters for Embedded font:�!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™ĀĀĂĂĄĄĆĆĊĊČČĎĎĐĐĒĒĖĖĘĘĚĚĞĞĠĠĢĢĪĪĮĮİĶĶĹĹĻĻĽĽŃŃŅŅŇŇŌŌŐŐŔŔŖŖŘŘŚŚŞŞŢŢŤŤŪŪŮŮŰŰŲŲŴŴŶŶŹŹŻŻȘș−≤≥fifl

​

ΆΈΉΊΌΎΏΐΑΒΓΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΪΫΆΈΉΊΰαβγδεζηθικλνξορςΣΤΥΦΧΨΩΪΫΌΎΏ

ЁЂЃЄЅІЇЈЉЊЋЌЎЏАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯЁЂЃЄЅІЇЈЉЊЋЌЎЏѢѢѲѲѴѴҐҐәǽẀẁẂẃẄẅỲỳ№

​

​

​

​

​

​

​

​

Do not add objects or text in the footer area

L2 Gateway

--tenant-id = tenant id

​

--device name = this is the name of the Physical Switch on the device

​

--device interface_names = this is the name of the physical port on the device where Bare Metal servers are connected.

​

Neutron L2GW API allows bridging two or more networks together to make them look at a single broadcast domain. A typical use case is bridging the virtual with the physical networks

​

​

Example:

neutron l2-gateway-create gw1 --tenant_id 21cce60dfe7d4e04a8f9b0f32d90959c --device name=hwvtep,interface_names=hwvtep-eth1,interface_names=hwvtep-eth2�

​

​

​

​

neutron l2-gateway-create [-h]

[-f {html,json,json,shell,table,value,yaml,yaml}]

[-c COLUMN] [--max-width <integer>]

[--noindent] [--prefix PREFIX]

[--request-format {json}]

[--tenant-id TENANT_ID]

[--device name=name,interface_names=INTERFACE-DETAILS]

<GATEWAY-NAME>

​

L2 GW Connection

--default-segmentation-id = default segmentation-id that will be applied to interfaces for which seg id was not specified in l2-gateway-create command

​

--GATEWAY-NAME = this is the name of the logical gateway

​

--NETWORK-NAME = network name or uuid

​

Neutron L2GW Connection API associates the L2 GW with a network.

​

​

​

Example:

neutron l2-gateway-connection-create gw1 vx-net --default-segmentation-id 0�

​

​

​

​

​

​

​

​

neutron l2-gateway-connection-create [-h]

[-f {html,json,json,shell,table,value,yaml,yaml}]

[-c COLUMN]

[--max-width <integer>]

[--noindent] [--prefix PREFIX]

[--request-format {json}]

[--default-segmentation-id SEG_ID]

<GATEWAY-NAME/UUID>

<NETWORK-NAME/UUID>

​

​

Want to learn more about L2 GW?

L2 GW Wiki

• https://wiki.opendaylight.org/view/NetVirt:_L2Gateway_HowTo
• Instructions for installing and configuring ODL, OpenStack, L2Gw

​

​

• https://wiki.openstack.org/wiki/Neutron/L2-GW#L2_gateway_setup_using_devstack
• Openstack L2GW overview, use cases

​

• Visit demo booth

​

Topology: After Adding VMs and L2GW wiki setup

odl32-compute

odl31-control

br-int

vmvx1

10.100.5.3

192.168.56.10

dhcp

10.100.5.2

vxlan-192.168.254.32

tap883f9022-bd

tapd0d15959-1f

eth1

br-int

eth2

vmvx2

10.100.5.4

192.168.56.11

vxlan-192.168.254.31

tap5d62515a-be

eth1

OpenDaylight

router-node

eth1

eth0

192.168.56.1

External: VB Internal: 192.168.56.0/24

eth3

Data: VB Internal: 192.168.254.0/24

eth3

Management: VB Host-only: 192.168.50.0/24

eth2

192.168.254.31

192.168.254.32

192.168.50.31

192.168.50.32

HW VTEP

BM1

10.100.5.20

BM2

10.100.5.21

192.168.254.20

HWVTEP Emulator - Wiki on next slide

Debugging

Debugging - OVS

• ovs-appctl ofproto/trace
• e.g. for DHCP:

sudo ovs-appctl ofproto/trace br-int in_port=13,ip,dl_dst=ff:ff:ff:ff:ff:ff,nw_src=0.0.0.0 | grep “Rule\|actions”

• ovs-ofctl dump-flows, dump-groups, using watch
• watch -d -n 1 'ovs-ofctl -OOpenFlow13 dump-flows br-int | grep -v packets=0'
• watch -d -n 1 'ovs-ofctl -OOpenFlow13 dump-flows br-int table=17'
• ovs-ofctl -OOpenFlow13 show br-int
• ovs-ofctl -OOpenFlow13 dump-ports br-int
• ovs-ofctl -OOpenFlow13 dump-ports-desc br-int
• ovs-appctl dpctl/dump-flows

​

• OVS-DPDK only
• ovs-appctl dpif-netdev/pmd-stats-show (first run pmd-stats-clear while traffic is running)
• ovs-appctl dpif-netdev/pmd-rxq-show

​

• Others: ovs-appctl list-commands

Debugging - ofproto/trace

Debugging - ODL

• Restconf viewing of models
• URL: http://<ODL_IP>:8181/apidoc/explorer/index.html
• Requires knowledge of internal models, but allows debugging of what exists and where - most ODL logic is determined by data store content.
• Counters in logs (new in Boron, using infrautils project)
• 2016-09-21 16:26:56,289 | INFO | CountersThread | CountersDumperThread | 295 - org.opendaylight.infrautils.counters-impl - 1.0.0.SNAPSHOT | nsc.subnetmap_update: +17, nsc.install_default_nat_flow: +2, nsc.subnetmap_add: +1 , nsc.remove_default_nat_flow: +1, nsc.subnetmap_remove: +1
• Enabled by configuring writelog=true in <odl_dir>/etc/org.opendaylight.counters.cfg
• Configurable interval of log prints - each log prints the counters that have advanced since the last print.
• Each module has its own prefix - e.g. nsc. are NatServiceCounters

​

• Future:
• Tap features to allow viewing various neutron port traffic via Wireshark.
• More meaningful debug counters to ease debugging and allow easier tracking

OpenDaylight - NetVirt Pipeline

Group

Local VM

Group

Remote VM

(220)

Egress Dispatcher

(36) Internal Tunnel

(19) DMAC Service Filter

Other DMAC

L3 DMAC

(18*) BGPoMPLS DHCP

(38*) BGPoMPLS (Ext Tunnel)

Output to Controller

Resubmit to Dispatcher (17)

DHCP

Non-DHCP

Output to OpenFlow port

Group

Provider Network

IPv6 Service (2)

OpenDaylight - NetVirt Pipeline

(20) LFIB

Questions?

View this presentation: https://goo.gl/sWkjKQ

​

NetVirt Project Wiki: https://wiki.opendaylight.org/view/NetVirt

​

Mailing List: https://lists.opendaylight.org/mailman/listinfo/netvirt-dev

​