Prepared for the 27^th ICCRTS

25-27 October 2022

An integrated framework and taxonomy for cyber damage assessment

Dr. Maxwell Dondo and Dr. Natalie Nakhla

Director General R&D Science and Engineering

Outline

• The problem and proposed solution
• Key Performance Indicators (KPIs) for cyber damage
• Cyber damage taxonomy
• CDA framework
• Sample reports and visualisations
• Next steps

2

The problem and proposed solution

• Problem: Measure damage following a cyber breach

CAF needs an intuitive way to demonstrate and benchmark military Cyber Intelligence performance and Return On Investment (ROI) on defensive cyber operations (DCO) so that decision makers are constantly refreshed on their DCO value proposition.

​

• Solution: Metrics for damage following a cyber breach

Develop a Cyber Damage Assessment (CDA) reporting capability for metrics and measures of damages and losses following a cyber breach. Metrics to be driven by key performance indicators (KPIs) for cyber damage.

3

Solution approach

• KPI Taxonomy:

Develop a taxonomy of KPIs for cyber damage by identifying:

• strategic, operational, and tactical KPIs for CDA
• associated, context-specific metrics and measures for each KPI

​

• Framework for CDA:

Create a framework for CDA reporting by developing:

• Algorithms for CDA metrics calculations
• Approaches for cyber data
• collection and aggregation
• storage
• sharing
• Reporting and visualisations portal

4

Key Performance Indicators (KPIs) for cyber damage

• Remediation performance:
• Recovery effectiveness
• Direct business losses:
• Direct losses and payments
• Proprietary information losses:
• IP, PII, process, software data, etc.
• Opportunity Cost losses:
• Costs of lost opportunities
• Reputational losses:
• Loss in trust/partnerships, damages to the brand name
• Collateral damage losses:
• Losses incurred by non-combatants
• Wellness losses:
• Losses of life, health, wellbeing, etc.

5

* PII= Personally Identifiable Information

The CDA Taxonomy

6

The CDA Framework

• Monitoring
• Environment, such as ongoing missions, are monitored
• Data collection
• Heterogeneous data is collected
• Data is formatted and stored
• Analysis
• Algorithms analyse data and calculate CDA metrics
• Reporting
• Web-based metrics and measures are periodically reported and presented to decision makers

7

Sample proof of concept reports�and visualisations

9

Executive Dashboard- KPI: Sample service outages and productivity losses

​

Outages due to event and remediation

Total:

$76,264

Weekly Loss=Percentage capability usage x Outage hours x Number of users x FTE hourly rate

* Actual data ~ Sample data

An operational application

• Map to damage to stages of Cyber Kill Chain® [1]
• Infer cyber capabilities from damage reports
• Map damages to the five mission functions to show:
• operational impact
• benchmark CyInt capabilities
• compare remediation performance
• decision making support

10

[1] https://www.lockheedmartin.com/en-us/capabilities/cyber/cyber-kill-chain.html

Preliminary CDA challenges and suggested courses of action

11

1. Lack of awareness of CAF Cyber terrain

2. Insufficient defender logs to determine data exfiltration across all networks

3. Cost of Cyber protection operations are not captured as an activity

4. No outage logs available to generate productivity losses across all networks

5. Not currently tracking or analysing the adverse effects on CAF reputation following cyber breaches

Acquire Network Mapping tools

Acquire/develop tracking tools to tabulate data flows (coord with SSC?)

Establish Fin Codes for Cyber protection operations (or all DCO)

Link with NOCs to tabulate baseline outages to contrast with cyber-attacks

Synch with Public Affairs and open source reputational analytics to derive Cyber attack impact

Next steps

• Identifying and automating CDA data collection
• Completion of algorithms for CDA metrics
• Collaboration with partners to develop CDA metrics
• Develop a software tool for application by DND/CAF
• Deploy tool on DND/CAF network for access by stakeholders

13

Questions?

Key Performance Indicators (KPIs) for cyber damage

14