Sonata

Divya Joshi�CS395T

Problem

• Telemetry system - used for network monitoring / analysis
• Telemetry systems used either programmable switches (data plane) or stream processors (application layer)
• Tradeoff between scalability and expressiveness
• Switches could process queries at line rate (scalable) but had limited capability
• Stream processors could process much wider range of queries, but not scalable

​

• Idea of Sonata: combine these two approaches to achieve both expressiveness and scalability

Unified Query Interface

Dataflow queries on tuples:

• Extensible tuple abstraction
• Parsing of tuples, but allows for customizable packet parsing as well (using P4 or similar)
• Expressive dataflow operators
• Supports sequential computation of operators
• Stateful operators over a time window
• Limitations
• No more granular than packet
• Single switch
• Examples: SYN attack detection, Slowloris attack detection

Abstract Packet Processing Model

• Parser makes packet header vector (PHV), which is sent through match-action tables
• Tuples are sent through dataflow operators
• Both can be represented as DAG
• Match tables as vertices, packets as streams
• Dataflow operators as vertices, tuples as streams
• Convert dataflow operators to match tables

Conversions

• Filter / Map - single match table
• Reduce / Disjoint - stateful; require two match tables
• Join - divide and conquer-like approach, partially done in the stream processor

​

Conditions: preserve packet forwarding decisions, report results to stream processor, collision resistance

​

Query Partitioning

• Constraints
• Number of fields that can be parsed
• Number of actions per stage
• Limited memory in registers
• Number of stages
• Given these constraints, how much traffic should be sent to stream processor?
• Use training data to estimate amount of state / number of fields, etc. required to compute the query
• Repeat for several windows of given size, use medians
• Feed this to ILP to determine how much traffic to send to each component

Dynamic Query Refinement

• Simply partitioning may still lead to too much load on stream processor
• “Narrow Down” approach
• Identify refinement keys (must have hierarchical structure)
• Enumerate refinement levels
• Augment input entries
• Finer levels should only consider traffic that has satisfied the query at coarser levels
• Account for smaller amount of traffic
• Extend ILP to account for this

Implementation

• Core
• Query planner: Solves ILP, generates refined / partitioned queries
• Runtime: determines which dataflow operators go to switch, configures emitter, sends updates to drivers
• Drivers
• One for switch, one for stream processor
• Compile queries from runtime
• Emitter
• Sends results to stream processor

Evaluation

• Less lines of code
• Compatibility with different hardwares / protocols
• Reduced load on stream processor by up to 7 orders of magnitude for single query computation, up to 3 for multiple query
• Overhead for dynamic refinement was 5% of window time
• Detection of SYN attack worked within 1 second

What You Said - Design

“Does some other backend have to configured/implemented for each switch? Or will Sonata run out of the box on every switch which has a P4 compiler? It does mention Sonata is platform-agnostic, but don't the drivers have to be setup for the switch and stream processor?” - Asmita

​

“But I have concerns about the query planning ILP solver’s run time.” - Xiao

​

What You Said - Extensions

“I'm curious about the costs for mapping this system onto multiple switches, as currently it compiles queries to a single programmable switch” - David

​

“If we wanted to extend this for multiple switches, perhaps a natural extension would be to implement a portion of the query (maybe partitioned at an operator level) on each switch along the packet path, to further increase the total hardware available.” - Jiahan

​

“Did they every follow up on any extensions to Sonata, and is Sonata (or other similar telemetry systems that use programmable switches and query partitioning/refining) currently being used by network operators?” - Vinay

​