1 of 66

Advanced TimeCard and SyncModule for Datacenter & ORAN Synchronization

01/06/22

Nir Laufer nlaufer@adva.com , Oscilloquartz/ADVA

OCP-TAP Project

1

2 of 66

Agenda

Intro to OSA 5400 TimeCard and SyncModule
GNSS antenna installtion and operation challenges
GNSS assurance
Syncjack – Sync probing and assurance
Jamming and spoofing detection
Holdover challenges

2

3 of 66

Introduction to OSA 5400 TimeCard

3

4 of 66

OSA 5400 TimeCard^TM

IEEE 1588 PTP grandmaster/boundary/slave clock

Up to 64 unicast clients at 128pps
Multiple PTP profiles
PTP profiles conversion

GNSS receiver (L1 , L1+L5 coming soon)
NTP server
PTP input as backup to GNSS (APTS)
Sync probe (Syncjack^TM)
GNSS assurance
Sync-E In/Out
Different variants of oscillators

Comprehensive sync capabilities

Extended holdover, connectors on the front panel and PCIe

SyncModule

4

5 of 66

OSA 5400 TimeCard^TM for accurately�synchronizing servers

Open compute server featuring PCIe card slots

OSA 5400 TimeCard^TM

Server

Interface

Timing appliance

Interface

PCIe bus

GNSS receiver

Oscillator (OCXO/DOCXO/Rb )

PCIe interface

PTP master

PTP probe�PTP slave

PTP BC�NTP server

Server

Storage

Eth (copper)

Eth (fiber )

PPS/CLK

PPS+ToD

5

6 of 66

OSA 5400 TimeCard^TM

100M/1G copper +PoE Output

GNSS antenna input

PPS+TOD

CH1/CH2 I/O (PPS/CLK)

1G Fiber

Oscillator options: OCXO Qz/OCXO HQ+/DOCXO HQ++/Rb

OSA 5400 SyncModule

6

7 of 66

OSA 5400 SyncModule^TM Capabilities

IEEE 1588 PTP grandmaster/boundary/slave clock

Up to 64 unicast clients at 128pps
Multiple PTP profiles
PTP profiles conversion

GNSS receiver (GPS/GLONASS/BEIDOU/GALILEO/SBAS/QZSS)
NTP server (500K TPS)
PTP input as backup to GNSS (APTS)
Sync probe (Syncjack^TM)
Sync-E In/Out
OCXO based holdover

Comprehensive sync capabilities

Low-power solution <2.6W
Easily integrated into systems due to M.2 interface
Extended temperature range

42mm

22mm

AMC4 RF connector

GNSS Antenna

AMC4 RF connector

PPS/CLK

HDMI-D

1GbE copper

w/o magnetics

M.2 Interface

7

8 of 66

OSA5400 TimeCard variants

Power consumption:

SyncModule: 2.6W
TimeCard Quartz: 6.5W
TimeCard Quartz HQ+: 8.5W
TimeCard Quartz HQ++:10.5W
TimeCard Rubidium: 12.5W

8

9 of 66

Why to use PTP directly from TimeCard?

Simple and accurate Sync connectivity to external smart antenna (OSA 5405)
Accurate way to deliver timing to the application (e.g. ORAN LLS-C3)
PTP can be used a local interface to carry time and frequency (instead PPS+ToD)
Enable direct Sync probing of PTP traffic by the TimeCard

Copper port

Fiber port

9

10 of 66

Precisely synchronizing a wide range of applications

PTP�G.8265.1

PTP/NTP�Enterprise

PTP�G.8275.2

PTP�G.8275.1

2G/3G/4G/5G�FDD

DOCSIS 3.1�Small Cells�APTS

Datacenters

5G/LTE TDD�LTE-A �Small Cells

PTP�broadcast

Professional�broadcast

Power utilities

PTP�Power

Managing and operating transport and synchronization networks

10

11 of 66

Supported PTP Profiles

802.1AS-2011
AES67 Media
SMPTE ST 2059-2

IEC-61850-9-3
C37.238 2011
C37.238 2017

Telecom

Broadcast/Automotive

G.8275.1
G.8275.2
G.8265.1
Telecom-2008
IEEE1588-2008 Annex F
Enterprise

Power

Master/Slave

11

12 of 66

Example - 5G Open Radio Access Network

GM

PRTC/ePRTC

DU

BC

CU

Midhaul

Fronthaul

O-RU

C-RAN

O-RAN

12

13 of 66

Open-RAN Architecture

LLS-C3 Configuration

T-GM implemented in the fronthaul network to distribute timing to DU and CU

DU

RU

Fronthaul Switches

BC/TC

Sync

O-RAN LLS-C3 Config

T-GM

LLS - Lower Layer Split

Local or remote PRTC source

13

14 of 66

OSA5400 TimeCard Open RAN

O-RAN C3 Config

Fronthaul

RU

Sync

DU

Sync

ITU-T G.8275.1
DU GM

PTP Packets

T-GM

DU

RU

Fronthaul Switches

BC/TC

Sync

O-RAN C3 Config

BC/TC

BC

1G Copper/fiber

OSA 5400 Timecard

14

15 of 66

PTP Features

Full featured GrandMaster, Boundary Clock and Slave

PTP input to backup GNSS outage over network with partial/no timing support

VLAN (IEEE 802.1Q) or untagged

conversion between PTP profiles
Sync-E input to PTP output (frequency) conversion

IEEE1588 PTP

APTS

Conversion

VLAN support

Fixed and dynamic asymmetry compensation

Asymmetry compensation

Hardware base protection

DoS protection

15

16 of 66

NTP Server summary

Smallest NTP server formfactor
Security-hardened NTP server with Hardware-based responder
Stratum 1 NTP server when locked to GNSS
NTP v1, v2, v3 ,v4 and SNTP over IPv4 /IPv6
Hardware-based timestamping
Within +/–100nsec from UTC
Hardware base DoS protection using NTP responder
Up to 500,000 (TPS) Transactions Per Second
Support PTP and NTP on same port
PTP to NTP translation
PTP backup in case of GNSS outage
Stationary or moving platforms

16

17 of 66

GNSS Installation

17

18 of 66

GNSS installtion challenges

Installing GNSS RF antenna is expensive and complicated

Most of the solutions (including TAP TimeCard) are connecting the GNSS receiver over RF cable (e.g. LRM 400)
While this may seem simple , installing RF cable can be challenging:

The Cable in expansive
Cable is thick and hard to maneuver inside the building
Cable delay must be known and compensated in the receiver
The distance between the GNSS receiver and the antenna is limited (typically needs to be under 150m)
Installing GNSS RF cable is complicated and requires expertise
Separation of the GNSS receiver from the TimeCard:

Optimize GNSS receiver to application (Single vs Multi band , L1+L2 vs L1+L5 etc’)

18

19 of 66

Synchronizing sites and applications

Requirements

Simpler installation

Integrated solution

Higher accuracy

GNSS antenna

Coaxial cable

Standard TimeCard

Good enough ?

Optimized solution

OSA 5405-MB

Integrated antenna
Robust & accurate multi-band receiver
Ruggedized design
PTP/NTP directly from the smart antenna

PTP (over copper/fiber)

Extended range

OSA 5400 TimeCard

19

20 of 66

Simplifying GNSS antenna installation with Smart antenna

COTS (e.g. DU)

PTP Over Copper/Fiber

OSA 5405 Outdoor

NIC

PPS/CLK/ToD

OSA 5400 TimeCard as PTP slave

Recover clock from OSA 5405 and provide it to NIC via PPS/CLK/ToD

PTP+PoE

PTP (G.8275.1) is used a local time transfer between smart antenna and the TimeCard

20

21 of 66

OSA 5405 Outdoor Installation

21

22 of 66

GNSS Assurance

22

23 of 66

GNSS vulnerabilities and threats

GNSS for�timing

Jamming and �spoofing

Obstruction

Interference with�transmitters at adjacent bands

ionospheric disturbance, solar activity

GNSS segment errors

23

24 of 66

What is the resilient PNT mandate/standard?

Driven by US federal gov’s executive order 13905 of Feb 2020

Protect critical gov & industry infrastructure against PNT disruptions from GPS/GNSS jamming/spoofing & other cyberattacks

Define critical infrastructure under national security threats

Power grid
Finance
Transportation
Communications (5G, broadcast, defense, etc)
Data centers

Use published resilient PNT guidelines & standard in progress

DHS Resilient PNT Conformance Framework
NIST Cybersecurity Framework for PNT Profile
IEEE P1952 Resilient PNT UE Standard working group

24

25 of 66

DHS PNT Resilience Levels

End-user select a level based on risk tolerance, budget, application critically

Resilience levels

Level 1

Support full system recovery
Include ability to securely update firmware

Level 2

Identify compromised PNT sources
Support Automatic recovery

Level 3

Must ensure that corrupted data from one PNT source cannot corrupt data from another PNT source
Cross verify between PNT solutions all PNT sources

Level 4

Ability to operate through any compromising events without degradation to the PNT solution
Diversity of PNT source technology to mitigate common mode threats

PNT Sources examples:

GNSS receivers, networked and local clocks, inertial navigation systems, timing services provided over wired or wireless connection

Upper-level resilience behavior requires all resilience behavior from lower levels

Verification

PNT source

PNT system

PNT source 2

PNT source 1

PNT system

Verification

Resilience processing

Automatic recovery

PNT source N

PNT source 2

PNT source 1

Verification

Advanced resilience processing

Automatic recovery

PNT system

Architecture requires future development to ensure inclusion of next generation PNT systems

Level 1

Level 2

Level 3

Level 4

Low level receiver is not necessarily better or worse. It simply reflects a level that meets the user’s particular needs

25

26 of 66

Multilayer Detection

4: Network Management

3: Device

2: GNSS Receiver

1: GNSS Antenna

Layer 1

Layer 3

Layer 4

Layer 1: GNSS Antenna

Use anti-jam/spoof antennas, with threat alarms

Layer 2: GNSS Receiver

Use multi-constellation/-band receivers, with jam/spoof & satellite count monitoring, integrated spoof detection and threat alarms
Use advanced spoofing detection

Layer 3: Device Level

SyncJack to mitigate spoofed GNSS vs PTP signals

Layer 4: Network Management

Manage/monitor/compare/verify all network clocks (GNSS/PTP/ etc.) in real-time, with performance threat alarms/analytics

Layer 2 Advanced spoofing detection

26

27 of 66

Spoofing methods

Low budget attack

SDR (Software defined radio)

100$ attack i.e. HackRF+Github SW

Hard-take-over

Spoofing signal is not aligned with prototype in terms of time and Doppler
High-power (>30dB) brute-force attack recognized as strong interference at first and forcing receiver to switch to Signal Acquisition Mode

After that receiver lock to the stronger spoof signal

High cost of the attack

High accuracy signal necessary (offset<500ns)

Smooth-take-over attack

Generates signal identical to genuine GNSS (time, location, almanac, ephemeris)
Increase the power smoothly and receiver lock to spoofer when power is higher

Spoof signal is ~4dB higher than prototype

Receiver identify synchronous signal as genuine
Spoofer may smoothly shift time when device took-over

Asynchronous attack

Synchronous attack

GNSS Spoofer

with GNSS receiver

27

28 of 66

Layer 2+ – Device level detection

Detect spoofing even if device started under spoofing conditions

Detect spoofing attack to the device location

Generate dedicated alarm when advanced spoofing detected

Detect spoofing attacks with no time jump

Stateless detection

Position discrepancy

Time discrepancy

Alarm

Detect multi-constellation and multi-band spoofing attacks

Multi-constellation & Multi-band inconsistencies

Detect spoofing attacks started with high power jamming attack

Jamming+spoofing

Jamming / Basic Spoofing detection/ Advanced Spoofing Detection

Spoofing mitigation algorithm is able to distinguish signals from satellites and from SDR

Advanced algorithm

28

29 of 66

TimeCard PNT Resilience Levels

Level 4 device with ENC Firewall Support

Level 1

Level 2

Level 3

Level 4

Ability to operate through compromising events without degradation to the PNT solution
Diversity of PNT source technology to mitigate common mode threats

Level 4

PTP

SyncE

GNSS

29

30 of 66

Centralized GNSS monitoring and assurance

GNSS Receiver

GNSS monitoring and assurance

30

31 of 66

Centralized GNSS monitoring and assurance

GNSS Receiver

GNSS monitoring and assurance

Use ML and AI

Instructions are sent back to the relevant devices (e. go to holdover or switch to backup)

31

32 of 66

What data can we get for a GNSS receivers ?

The following data is available from most of the commercial GNSS receivers via API

Can be collected remotely over secured interfaces (e.g. CLI-SSH/SNMPv3)

Location :

Latitude , Longitude, Altitude

Satellites related data

SV , Carrier to Noise , Health , Azimuth, Elevation ,AGC

32

33 of 66

Blind spot example – customer site

Poor signal from the tower direction

High tower

33

34 of 66

Useful data displays

C/No heatmap (indication of directional GNSS signal strength).

Satellites usage heatmap

34

35 of 66

Site Analysis – “Bad” Site

Help identify local reception issue

35

36 of 66

Site Analysis – “Good ” Site

36

37 of 66

GNSS Assurance – GNSS firewall - AI/ML React

Intelligent automated solution for sync quality issues prevention and correction

Key features
Utilize AI for automated detection of potential GNSS vulnerabilities Use predictive maintenance to safeguard GNSS signal reception A SW based solution, no additional hardware Allows user to define preventive actions to automatically stop using the GNSS when Alarm (either from NE or from ML/AI on ENC) is raised; the “clear” action can be defined to use GNSS again as Time Clock reference when Alarm is cleared

Benefits
Provides a key solution to fight against GNSS cyber attacks Utilize ML/AI for automated optimization and security protection of GNSS Simplifies manual operations in reaction to device and/or ENC reported synchronization alarms

Firewall reference switch triggers

Device Level

AI/ML Level

37

38 of 66

Sync Assurance

38

39 of 66

Why „In Service“ Sync Assurance is needed ?

In service monitoring sync critical component in NG networks

Large number of application are highly dependent on accurate synchronization
Making sure synchronization is working as designed is not trivial task
Networks are dynamic- PDV, asymmetry and environmental conditions can affect the Synchronization quality
Ways to ensure proper synchronization should be integrated into Sync distribution/delivery functions or accompanied by cost effective Sync assurance tools
Lab test equipment is too expansive for “in service” installation in multiple locations
Other aspects such as power consumption and OSS should also be taken into consideration

Challenges

39

40 of 66

SyncJack

Collect RAW measurement data
Monitor:

TIE/TE
Const TE
TIE/TE (LPF)
TDEV
MTIE

RAW data.txt file

FTP Server

40

41 of 66

Sync probe “Tester” in TimeCard

GNSS/PTP/SYNC-E/PPS/CLK as source

GNSS/PTP/SYNC-E/PPS/CLK as reference

Sync “tester’

TE hardware counters (nsec accuracy)

41

42 of 66

Centralized in service Sync monitoring and assurance

TimeCard�Probe

Centralized monitoring and assurance

Raw measurements

42

43 of 66

Probing a boundary clock

GM

BC

TimeCard �Probe

Slave

NMS

PTP/Sync-E

TE/TIE/MTIE/TDEV

PTP

Raw measurement

43

44 of 66

Probing a slave clock

GM

BC

TimeCard�Probe

Slave

NMS

SUT:

PPS/CLK/BITS/Sync-E

TE/TIE/MTIE/TDEV

Raw measurement

44

45 of 66

Probing the network

GM

TimeCard probe

NMS

PTP

Asymmetry / pktSelected2wayTE

PTP

Raw measurement

Network

PTP

45

46 of 66

Oscillators and holdover

The “good” the “bad” and the“ugly”

46

47 of 66

It can get ugly…

GNSS failure

Jamming
Antenna breakdown (lightning , cable cut)

Equipment failure

Hardware failure

47

48 of 66

Long-term Holdover

Required in case of long GNSS outage e.g., antenna failure due to lightning
Duration : Few hours – 3 days
Holdover performance depends on oscillator type

48

49 of 66

Will temperature be constant during long holdover?

49

50 of 66

Oscillator Types – Aging and Temperature Stability

Temperature Stability

over temp. range

Aging/Day

≤10ppb

≤1ppb

≤0.1ppb

≤0.01ppb

≤0.001ppb

≤1ppb

≤0.1ppb

≤0.01ppb

≤0.001ppb

≤0.0001ppb

OCXO

Rubidium

x2-x10 better Aging Vs Super DOCXO

≤0.0001ppb

Super

DOCXO

X5 better Temperature Stability Vs Rb

Super

DOCXO

Aging compensation using GNSS

50

51 of 66

Oscillator Types - Who is the “good” and who is the “bad” ?

Clock Type #	Cost	Temperature range	Temp Stability	Aging/Day
OCXO	Low (10%)	-40 to 85 C	1-10 ppb	1ppb
Super DOCXO	Medium (100%)	-40 to 85 C	0.01 ppb	0.05 ppb
Rubidium A	High (300%)	-10 to 75 C	0.05 ppb	0.025ppb
Rubidium B	High (300%)	-10 to 60C	0.4ppb	0.005ppb

Rb High cost , limited operational temperature and instability under temperature variation

51

52 of 66

Testing Holdover in Realistic Conditions

The objectives of the test were to measure the real performance of the Super DOCXO Vs Rubidium in lock and in holdover modes
Two PTP grand masters were placed in the same oven under same temperature conditions
Vendor A with Super DOCXO and Vendor B with Rubidium (Rb market leader)
Both devices were connected to the same GPS simulator using a GNSS splitter
Both device were powered at the same time and have been locked to GPS for 72 hours before holdover initiated
The temperature in the oven varied between 25C to 35C

52

53 of 66

Test Setup – The Duel

Super DOCXO

GNSS

Splitter

GNSS RF

GPS

Simulator

Cs Clock

10MHz

Rubidium

Tester

PPS REF

PPS

Oven

Vendor A

Vendor B

53

54 of 66

Temperature Profile

…

The oven was running the following temperature profile during the entire measurement:

25C for 4 hours
Ramp from 25C to 35C at 1.5C/min
35C for 4 hours
Drop from 35C to 25C at 1.5C/min
Back to #1

25C

4 Hours

35C

25C

35C

1.5C/Min

4 Hours

1.5C/Min

54

55 of 66

TE Test Results – Locked to GPS (25-35C)

Rb output “peaks” during temperature changes

1 Days

Super DOCXO is agnostic to temperature variation

Green – Super DOCXO

Red – Rubidium

Blue – Temperature Profile

35C

25C

35C

1.5C/Min

25C

TE

0nsec

30nsec

40nsec

55

56 of 66

TE Test Results – Holdover (25-35C)

Rubidium “drift” during temperature changes

300ns

800nse

2 Days

GPS

disconnected

Green – Super DOCXO

Red – Rubidium

TE

0nsec

100nsec

200nsec

800nsec

600nsec

Super DOCXO

is agnostic to temperature variation

56

57 of 66

Conclusions

Select the type of oscillator based the expected thermal condition on site

No good or bad – finding the best fit to the application

The Rubidium clock tested was very sensitive to temperature variations Vs Super DOCXO
The Rubidium clock holdover performance under temperature variations were significantly degraded Vs constant temperature
Actual performance under temperature variation is not listed in the datasheet !
Test your clocks in realistic holdover scenarios which include temperature variation! Testing in constant temperature is not sufficient!
Consider worse case scenarios such as air condition failure or AC hysteresis

57

58 of 66

OSA 5400 TimeCard enhancements

Taking the TimeCard to the next level!

PTP and NTP directly from the card
Simplify GNSS antenna installtion with smart antenna (OSA 5405)
Advanced jamming and spoofing detection
GNSS assurance
Syncjack – sync monitoring and assurance
Multiple oscillator/holdover options

58

59 of 66

Additional info:

OSA 5400 TimeCard product page:

https://www.oscilloquartz.com/en/products-and-services/embedded-timing-solutions/osa-5400-timecard

Datasheet : https://www.oscilloquartz.com/en/resources/downloads/data-sheets/osa-5400-timecard

OSA 5400 SyncModule product page:

https://www.oscilloquartz.com/en/products-and-services/embedded-timing-solutions/osa-5400-syncmodule

Datasheet:

https://www.oscilloquartz.com/en/resources/downloads/data-sheets/osa-5400-syncmodule

59

60 of 66

nlaufer@adva.com

Thank you

IMPORTANT NOTICE

The content of this presentation is strictly confidential. ADVA is the exclusive owner or licensee of the content, material, and information in this presentation. Any reproduction, publication or reprint, in whole or in part, is strictly prohibited. �The information in this presentation may not be accurate, complete or up to date, and is provided without warranties or representations of any kind, either express or implied. ADVA shall not be responsible for and disclaims any liability for any loss or damages, including without limitation, direct, indirect, incidental, consequential and special damages, alleged to have been caused by or in connection with using and/or relying on the information contained in this presentation.�Copyright © for the entire content of this presentation: ADVA.

60

61 of 66

OSA5400 SyncModule

Coax

GNSS

GPIOs 1-11, Eth2, Eth3

Timing Diagram

Eth1

M.2 Interface

61

62 of 66

OSA5400 TimeCard – Power options

Power – three alternatives

PCI Express 225W/300W High Power Connector

External AC/DC power Converter

PCI-e

All TimeCard variants can be powered-up through PCI-e interface only, if PoE is disabled

62

63 of 66

OSA5400 SyncModule Evaluation

Evaluation Board

Evaluate SyncModule without hosting device and M.2 interface integration
TimeCard Quartz variant as hosting device
Use External AC/DC power converter

Input 90-264VAC
Output 12VDC, max 3.3A
AC cable ordered separately

AC/DC power converter

63

64 of 66

Management

64

65 of 66

Management

Access

Remote auth RADIUS
Access Control Lists

Secured

Backup
Config Restore

ENC

SNMP v2, v3
Alarms, Inventory and traps reporting to NMS

Secure Access

IPv4 IPv6

ENC

Logs

CLI

Backup & Restore

Logs

Syslog
Alarm log

In-band management

IPv4 & IPv6 supported
Separate MGMT & PTP IP
VLAN support

Communication

CLI: Telnet, SSH
SW Download: TFTP, SCP

65

66 of 66

OSA5400 TimeCard – Holdover (constant temperature!)

	200ns	400ns	1.1us	1.5us
Quartz	2 hours	3 hours	6 hours	6.5 hours
Quartz HQ+	5 hours	8 hours	16 hours	18.5 hours
Quartz HQ++	14.5 hours	16.5 hours	1.4 days	4.8 days
Rubidium	15 hours	26 hours	2 days	2.3 days

66