1 of 70

This slide deck contains extensive docs about our�OFC 2024 demo

2 of 70

It’s intended�for offline reading.�Feel free to reach us at:

jan.kundrat@cesnet.cz

3 of 70

Demo

  • The demo showcases:
    • DevOps approach towards L0 SDN
    • Integration of existing tools
    • Power of open source SW and open HW
    • Combination of telco traffic & sensing

​

4 of 70

Demo topology

5 of 70

Topology: a triangle in the lab

6 of 70

7 of 70

Lab setup

8 of 70

With a production fiber�carrying live traffic

9 of 70

10 of 70

…with a sensing setup

11 of 70

12 of 70

built on open hardware

13 of 70

Czech Light ™ SDN ROADM

Scalable,

Modular,

Open

​

Flexgrid, Colorless, Directionless, Contentionless

​

SDN Spectrum Control and Monitoring

14 of 70

Information & Documentation

  • Product web
  • Journal Publications
    • YANG/NETCONF ROADM: Evolving Open DWDM Toward SDN Applications (JLT 2018).
    • Opening up ROADMs: Let Us Build a Disaggregated Open Optical Line System (JLT 2019).
    • Opening up ROADMs: a filterless add/drop module for coherent-detection signals (JOCN 2020).
    • Opening up ROADMs: streaming telemetry [Invited] (JOCN 2021).
  • Conference Demos
    • Chatty ROADMs: Streaming Telemetry with Open Source SW and Open Hardware (ECOC 2021)
    • GNPy & YANG: Open APIs for End-to-End Service Provisioning in Optical Networks (OFC 2021)
    • Physical-layer awareness: GNPy and ONOS for end-to-end circuits in disagg. networks (OFC 2020)
    • Opening up ROADMs: Let's Build a Disaggregated Open Optical Line System (ONF Connect 2019)
    • …

15 of 70

more HW & SW details

at the end

of the deck

16 of 70

optics tends to be complex

17 of 70

Is Optics Complex?

18 of 70

It does not have to be.

19 of 70

standard IETF YANG models�for standard functionality

20 of 70

21 of 70

YANG Models

  • Standard models (incomplete list)
    • ietf-access-control-list
    • ietf-hardware
    • ietf-interfaces
    • ietf-ip
    • ietf-routing
    • ietf-netconf-acm
    • ietf-system
    • …
  • Extensions
    • czechlight-lldp
    • czechlight-network
    • czechlight-roadm-device
    • czechlight-system

​

https://github.com/CESNET/CzechLight-yang

22 of 70

YANG model for ROADMs

23 of 70

YANG: API Which Fits on One Slide

module: czechlight-roadm-device

+--rw channel-plan

| +--rw channel* [name]

| +--rw name string

| +--rw lower-frequency opendevice-t:dwdm-frequency-mhz

| +--rw upper-frequency opendevice-t:dwdm-frequency-mhz

+--rw media-channels* [channel]

| +--rw channel -> /channel-plan/channel/name

| +--rw description? string

| +--rw add!

| | +--rw port device-dependent-port-type

| | +--rw (mode)

| | +--:(attenuation)

| | | +--rw attenuation attenuation-type

| | +--:(power)

| | +--rw power opendevice-t:optical-power-dBm

| +--rw drop!

| | ...

| +--ro power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

| +--ro leaf-in opendevice-t:optical-power-dBm

| +--ro leaf-out opendevice-t:optical-power-dBm

​

+--ro aggregate-power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

+--rw leaf-ports* [port]

| +--rw port device-dependent-port-type

| +--rw description? string

+--rw line {hw-line-9}?

| +--rw output-voa? attenuation-type <0.0>

| +--ro osc

| +--ro tx-power? opendevice-t:optical-power-dBm

| +--ro rx-power? opendevice-t:optical-power-dBm

+--rw spectrum-scan!

+--rw period union

+--rw pin-source? union {pre-wss-ocm}?

+--ro common-in

| +--ro lowest-frequency frequency-ghz

| +--ro step frequency-ghz

| +--ro p? anydata

+--ro common-out

+--ro lowest-frequency frequency-ghz

+--ro step frequency-ghz

+--ro p? anydata

24 of 70

#1: define the channel plan�(because this is flexgrid)

25 of 70

#1: Define Spectrum

module: czechlight-roadm-device

+--rw channel-plan

| +--rw channel* [name]

| +--rw name string

| +--rw lower-frequency opendevice-t:dwdm-frequency-mhz

| +--rw upper-frequency opendevice-t:dwdm-frequency-mhz

+--rw media-channels* [channel]

| +--rw channel -> /channel-plan/channel/name

| +--rw description? string

| +--rw add!

| | +--rw port device-dependent-port-type

| | +--rw (mode)

| | +--:(attenuation)

| | | +--rw attenuation attenuation-type

| | +--:(power)

| | +--rw power opendevice-t:optical-power-dBm

| +--rw drop!

| | ...

| +--ro power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

| +--ro leaf-in opendevice-t:optical-power-dBm

| +--ro leaf-out opendevice-t:optical-power-dBm

​

+--ro aggregate-power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

+--rw leaf-ports* [port]

| +--rw port device-dependent-port-type

| +--rw description? string

+--rw line {hw-line-9}?

| +--rw output-voa? attenuation-type <0.0>

| +--ro osc

| +--ro tx-power? opendevice-t:optical-power-dBm

| +--ro rx-power? opendevice-t:optical-power-dBm

+--rw spectrum-scan!

+--rw period union

+--rw pin-source? union {pre-wss-ocm}?

+--ro common-in

| +--ro lowest-frequency frequency-ghz

| +--ro step frequency-ghz

| +--ro p? anydata

+--ro common-out

+--ro lowest-frequency frequency-ghz

+--ro step frequency-ghz

+--ro p? anydata

26 of 70

#2: activate these channels�(they become monitored)

27 of 70

#2: Activate Channels

module: czechlight-roadm-device

+--rw channel-plan

| +--rw channel* [name]

| +--rw name string

| +--rw lower-frequency opendevice-t:dwdm-frequency-mhz

| +--rw upper-frequency opendevice-t:dwdm-frequency-mhz

+--rw media-channels* [channel]

| +--rw channel -> /channel-plan/channel/name

| +--rw description? string

| +--rw add!

| | +--rw port device-dependent-port-type

| | +--rw (mode)

| | +--:(attenuation)

| | | +--rw attenuation attenuation-type

| | +--:(power)

| | +--rw power opendevice-t:optical-power-dBm

| +--rw drop!

| | ...

| +--ro power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

| +--ro leaf-in opendevice-t:optical-power-dBm

| +--ro leaf-out opendevice-t:optical-power-dBm

​

+--ro aggregate-power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

+--rw leaf-ports* [port]

| +--rw port device-dependent-port-type

| +--rw description? string

+--rw line {hw-line-9}?

| +--rw output-voa? attenuation-type <0.0>

| +--ro osc

| +--ro tx-power? opendevice-t:optical-power-dBm

| +--ro rx-power? opendevice-t:optical-power-dBm

+--rw spectrum-scan!

+--rw period union

+--rw pin-source? union {pre-wss-ocm}?

+--ro common-in

| +--ro lowest-frequency frequency-ghz

| +--ro step frequency-ghz

| +--ro p? anydata

+--ro common-out

+--ro lowest-frequency frequency-ghz

+--ro step frequency-ghz

+--ro p? anydata

28 of 70

#3: setup routing�(might be asymmetrical, attenuation vs. power level,...)

29 of 70

#3: Routing

module: czechlight-roadm-device

+--rw channel-plan

| +--rw channel* [name]

| +--rw name string

| +--rw lower-frequency opendevice-t:dwdm-frequency-mhz

| +--rw upper-frequency opendevice-t:dwdm-frequency-mhz

+--rw media-channels* [channel]

| +--rw channel -> /channel-plan/channel/name

| +--rw description? string

| +--rw add!

| | +--rw port device-dependent-port-type

| | +--rw (mode)

| | +--:(attenuation)

| | | +--rw attenuation attenuation-type

| | +--:(power)

| | +--rw power opendevice-t:optical-power-dBm

| +--rw drop!

| | ...

| +--ro power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

| +--ro leaf-in opendevice-t:optical-power-dBm

| +--ro leaf-out opendevice-t:optical-power-dBm

​

+--ro aggregate-power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

+--rw leaf-ports* [port]

| +--rw port device-dependent-port-type

| +--rw description? string

+--rw line {hw-line-9}?

| +--rw output-voa? attenuation-type <0.0>

| +--ro osc

| +--ro tx-power? opendevice-t:optical-power-dBm

| +--ro rx-power? opendevice-t:optical-power-dBm

+--rw spectrum-scan!

+--rw period union

+--rw pin-source? union {pre-wss-ocm}?

+--ro common-in

| +--ro lowest-frequency frequency-ghz

| +--ro step frequency-ghz

| +--ro p? anydata

+--ro common-out

+--ro lowest-frequency frequency-ghz

+--ro step frequency-ghz

+--ro p? anydata

30 of 70

#4: fancy monitoring options

31 of 70

#4: Monitoring

module: czechlight-roadm-device

+--rw channel-plan

| +--rw channel* [name]

| +--rw name string

| +--rw lower-frequency opendevice-t:dwdm-frequency-mhz

| +--rw upper-frequency opendevice-t:dwdm-frequency-mhz

+--rw media-channels* [channel]

| +--rw channel -> /channel-plan/channel/name

| +--rw description? string

| +--rw add!

| | +--rw port device-dependent-port-type

| | +--rw (mode)

| | +--:(attenuation)

| | | +--rw attenuation attenuation-type

| | +--:(power)

| | +--rw power opendevice-t:optical-power-dBm

| +--rw drop!

| | ...

| +--ro power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

| +--ro leaf-in opendevice-t:optical-power-dBm

| +--ro leaf-out opendevice-t:optical-power-dBm

​

+--ro aggregate-power

| +--ro common-in opendevice-t:optical-power-dBm

| +--ro common-out opendevice-t:optical-power-dBm

+--rw leaf-ports* [port]

| +--rw port device-dependent-port-type

| +--rw description? string

+--rw line {hw-line-9}?

| +--rw output-voa? attenuation-type <0.0>

| +--ro osc

| +--ro tx-power? opendevice-t:optical-power-dBm

| +--ro rx-power? opendevice-t:optical-power-dBm

+--rw spectrum-scan!

+--rw period union

+--rw pin-source? union {pre-wss-ocm}?

+--ro common-in

| +--ro lowest-frequency frequency-ghz

| +--ro step frequency-ghz

| +--ro p? anydata

+--ro common-out

+--ro lowest-frequency frequency-ghz

+--ro step frequency-ghz

+--ro p? anydata

32 of 70

live spectrum scans, 2.5pm resolution, 1Hz refresh rate

33 of 70

Monitoring & Telemetry

  • Streaming Telemetry
    • All the metrics
    • Sub-second latencies
  • I/O Formats
    • IETF YANG-push
    • OpenMetrics�(Prometheus)
    • Grafana

​

34 of 70

Time-Series Database

35 of 70

Grafana: On-Device Channel Details

36 of 70

But Grafana can do more:

37 of 70

Physical topology with live per-channel power metering

38 of 70

Grafana: Topology

39 of 70

Can we deploy this

without an NMS?

40 of 70

Store complete config of all channels of all devices in Ansible

41 of 70

Configuration via Ansible

- hosts: roadm-c0

# Ansible’s built-in NETCONF support

connection: ansible.netcommon.netconf

tasks:

​

# Spectrum Definition

- CzL_channel_plan:

name: ciena

state: present

lower_frequency: '193950000'

upper_frequency: '194150000'

​

# Spectrum Routing

- CzL_media_channel:

name: ciena

state: present

leaf_port: 1

attenuation_add: 0

attenuation_drop: 10

description: Ciena

42 of 70

(this is how it’s implemented behind the scenes)

43 of 70

Ansible’s NETCONF

def config_string(CHname, min_f, max_f, state) -> str:

'''Turn spectrum channel definition to an XML string'''

​

root = ET.Element('{urn:ietf:params:xml:ns:netconf:base:1.0}config')

cl = '{http://czechlight.cesnet.cz/yang/czechlight-roadm-device}'

ET.register_namespace('cl', cl[1:-1])

​

channel_plan = ET.SubElement(root, f'{cl}channel-plan')

channel = ET.SubElement(channel_plan, f'{cl}channel')

if (state == 'absent'):

channel.attrib['ns0:operation'] = 'delete'

ET.SubElement(channel, f'{cl}name').text = CHname

else:

ET.SubElement(channel, f'{cl}name').text = CHname

ET.SubElement(channel, f'{cl}lower-frequency').text = min_f

ET.SubElement(channel, f'{cl}upper-frequency').text = max_f

​

return ET.tostring(root, encoding='unicode', xml_declaration=True)

44 of 70

demo: re-route a channel

&

watch it in Grafana

45 of 70

Demo recording

youtu.be/lviKYlL0EYA

46 of 70

sensing setup

47 of 70

Sensing

  • Measure small changes of the optical path
    • …or (indirectly) in the environment
    • Phase of light
    • Polarization
    • Reflections
  • Demo
    • Remotely activated vibrations
    • Live telemetry of�“event intensity”
    • Optically transparent,�inline detection

​

48 of 70

the motor induces�vibrations to the fiber

49 of 70

Real-time Data

50 of 70

“it shakes before it breaks”

51 of 70

52 of 70

Also, it’s just telemetry.

53 of 70

What’s Next?

  • Plug-in Transponders
    • NETCONF/RESTCONF
  • More ROADMs
    • New vendors
  • OLS Controller
    • Active power control
    • Optimize with GNPy
  • Alarms and Alerts
    • Event-driven architecture
    • Fancy UI
  • Better APIs
    • TIP’s MUST

54 of 70

Some details

about the ROADMs

55 of 70

Modular system

  • 1U pizza boxes
  • Scalability
    • Up to 8-degree ROADM
    • Redundant Add/Drop
    • Limit: 10 boxes per site
  • Element-level control
    • SDN northbound APIs
    • One NETCONF server per box

56 of 70

Czech Light ™ SDN ROADM

57 of 70

Line Degree

  • Long-haul connection
    • Internal amplification,�up to 25 dB spans (VOA)
    • 0 dBm/ch typ. TX power
    • Integrated OSC
    • OTDR option via SFP
  • Flexgrid
    • 12.5 GHz granularity
    • min. 50 GHz MCs
  • 9 Express ports
    • -12 dBm/ch TX
    • -15..+5 dBm/ch RX

58 of 70

Add/Drop Options

  • Passive
    • Direct-connection to Express ports
    • 2-deg only
    • 8 client ports
    • Spectrum control via Line Degrees
    • -12 dBm/ch

​

59 of 70

Add/Drop Options

  • Coherent Signals
    • 8ch/1U
    • Per-port power monitoring
    • Power equalization via Line Degree ROADM

60 of 70

Add/Drop Options

  • WSS-based
    • 20ch/1U
    • 12.5 GHz routing�(min. 37.5 GHz per MC)
    • 12.5/6.25 GHz monitoring
    • Indirect client-port monitoring

61 of 70

Add/Drop Options

  • Alien Wavelength
    • 20ch/1U
    • 3.25 GHz routing
    • 312.5 MHz monitoring (2.5 pm)
    • Direct client-port spectrum monitoring

​

62 of 70

In-Line Amplifiers

  • Dual EDFA
    • Flat gain, 27 dB
    • NF ≤ 5.5
    • No tilt control
    • Up to 25 dB spans, output VOA in both directions
    • Class-1M product
  • OTS
    • Power monitoring
    • Integrated OSC
    • OTDR option via SFP

63 of 70

BiDi EDFAs

  • BiDi Amplification
    • Mainly for time & frequency transfer
  • Multiband
    • Various options, talk to us

64 of 70

SDN at L0

65 of 70

SDN

  • Everything is remotely configurable
    • NETCONF
    • RESTCONF YANG-push
    • Web dashboard (R/O)
    • Local console
      • SSH
      • microUSB
  • Optics (DWDM)
    • Spectrum routing�and monitoring
  • System management
    • HW monitoring

​

66 of 70

Operating System

  • Read-only Linux rootfs via buildroot
    • Stateless system
      • Except the YANG database (and some bits for early boot)
  • Userland based on systemd
  • A/B software slots via rauc
    • Integrated with HW watchdog
    • Atomic system updates

Image source: © Enrico Jörns, Pengutronix.

Open Source

67 of 70

Source Code + CI/CD

https://gerrit.cesnet.cz/

br2-external docs ⎘

68 of 70

Example Integration:

TIP Summit 2019

69 of 70

ONOS + GNPy + OLS

70 of 70

Questions, comments?

jan.kundrat@cesnet.cz