UNCLASSIFIED

1

UNCLASSIFIED

Model-Driven Wargaming and �Cyber Risk to Mission Assessment

​

Mr. John Garstka, Director

Cyber Warfare Directorate

ODASD Platform and Weapons Portfolio Management

OUSD Acquisition and Sustainment

​

September 25, 2024

29^th International Command and Control Research and Technology Symposium

RUSI, London, UK

Agenda

• Challenge of Operating in a Contested Cyberspace Environment

​

​

• Our Approach to Assessing Cyber Risk to Mission

​

​

• Summary and Way Ahead

​

​

Cyber and Space Excerpts from Office of the Director of National Intelligence’s Annual Threat Assessment for 2023

UNCLASSIFIED

3

UNCLASSIFIED

People’s Republic of China

• “Counterspace operations will be integral to potential PLA military campaigns…intended to target U.S. and allied satellites.”
• “The PLA will continue to integrate space services…and satellite communications into its weapons and command-and-control systems in an effort to erode the U.S. military’s information advantage.”
• “If Beijing feared that a major conflict with the United States were imminent, it almost certainly would consider undertaking aggressive cyber operations against U.S. homeland critical infrastructure and military assets worldwide.”

Russia

• “Russia continues to train its military space elements, and field new antisatellite weapons to disrupt and degrade U.S. and allied space capabilities.”
• “Russia will remain a top cyber threat as it refines and employs its espionage, influence, and attack capabilities. Russia views cyber disruptions as a foreign policy lever to shape other countries’ decisions.”
• “Russia is particularly focused on improving its ability to target critical infrastructure…because compromising such infrastructure improves and demonstrates its ability to damage infrastructure during a crisis.”

Available to public on dni.gov

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

Cyberspace Enables Missions

and is a Contested Operational Domain

UNCLASSIFIED

4

UNCLASSIFIED

Cyberspace

Operating� environment

Adversary Cyber Forces Operating in Cyberspace

Adversary Kinetic Forces Dependent Upon Cyberspace

Enabling �environment

�

Blue Kinetic Forces Dependent Upon Cyberspace

Enabling environment

Operating environment

Blue Cyber Forces Operating in Cyberspace

kinetic v kinetic

cyber v cyber

Cyberspace is a Contested Operational Domain

UNCLASSIFIED

5

UNCLASSIFIED

Tier

Description

IV

Advanced – Have the capacity to conduct complex, long term cyber attack operations that combine multiple intelligence disciplines to obtain access to high-value networks

I

Moderate – Able to use customized malware with OPSEC practices to conduct wider-range intelligence

III collection operations, gain access to more isolated networks, and create short duration effects against critical infrastructure networks.

Limited – Able to identify and target for espionage or

II attack easily accessible unencrypted networks running common operating systems using publicly available tools.

Nascent – Little to no organized cyber capabilities, with no knowledge of a networks underlying systems or industry beyond publicly connected open-source information.

Existential

Nuisance

Tier

IV

― $Bs ―

Tier

III

― $Ms ―

Tiers

I - II

― $10s ―

Creates vulnerabilities using full spectrum

Discovers unknown vulnerabilities

Exploits pre-existing known vulnerabilities

DoD Forces must be able to operate in a contested cyber environment

(U) Cybersecurity as an Element of National Security

UNCLASSIFIED

6

UNCLASSIFIED

(U) National Cybersecurity Strategy

• (U) “Defending the systems and assets that constitute our critical infrastructure is vital to our national security, public safety, and economic prosperity. The American people must have confidence in the availability and resilience of this infrastructure and the essential services it provides”
• (U) “Software and systems are growing more complex, providing value to companies and consumers but also increasing our collective insecurity. Too often, we are layering new functionality and technology onto already intricate and brittle systems at the expense of security and resilience”

(U) Summary of the DoD Cyber Strategy

• (U) “The Department will enhance the cyber resilience of the Joint Force and ensure its ability to fight in and through contested and congested cyberspace.”
• (U) “As cyber threats grow and intensify, every soldier, sailor, airman, marine, guardian, coast guardsman, DoD civilian, and contractor is responsible for exercising cyber awareness and helping to manage the risk of the Department.”
• (U) “The United States is challenged by malicious cyber actors who seek to exploit our technological vulnerabilities and undermine our military's competitive edge. They target our critical infrastructure and endanger the American people.”

(U) “Defending the nation is paramount among our missions. It means defending our military systems, networks and the critical infrastructure that enable national security”

–(U) GEN Paul Nakasone, Commander, USCYBERCOM 2023 Posture Statement

FIGURE IS UNCLASSIFIED

FIGURE IS UNCLASSIFIED

Parallels in Historical and Modern Capital Asset Development

UNCLASSIFIED

7

UNCLASSIFIED

The majority of operational DoD weapon systems were conceived and developed prior to the evolution of current cyber threats. The DoD must ensure its weapon systems do not become the “battleships” of the 21^st century.

You are never as invincible as you believe.

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

Parallels in Historical and Modern Capital Asset Development

UNCLASSIFIED

8

UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

March 1914 Construction commences

October 1916 Commissioned at Brooklyn Yard

1940

Significant upgrades completed

1929

Major modernization completed, anti- air and torpedo defenses added

December 7, 1941

USS Arizona sunk

27 Years between program development and when USS Arizona was sunk

Risk to Surface Combatants From Air Threats

Bi-planes capable with early bombs and entered into service

RAF bi-planes destroy 3 Italian Surface Warships

GRAPHIC IS UNCLASSIFIED

December 7, 1941 USS Arizona sunk in Pearl Harbor

Parallels in Historical and Modern Capital Asset Development

UNCLASSIFIED

9

UNCLASSIFIED

1981

Program Initiated

1993

Program Developed

2010

27 years after program developed

1994

DFS-1

Launch

$5 billion spent

Increasing capability of malicious cyber actors

NotPetya ransomware attacks

Yahoo account compromises

Colonial Pipeline attacks

The PRC and Russia…are already using non- kinetic means against our defense industrial base and mobilization systems, as well as deploying counterspace capabilities that can target our Global Positioning System and other space-based capabilities that support military power and daily civilian life

2022 National Defense Strategy

NASA Satellites targeted

GRAPHIC IS UNCLASSIFIED

(U) Assessing Cyber Risk-to-Missions/Operations

UNCLASSIFIED

10

UNCLASSIFIED

THE MISSION STACK

FIGURE IS UNCLASSIFIED

FIGURE IS UNCLASSIFIED

(U) Cyber Key Terrain Landscape: Examples

UNCLASSIFIED

11

UNCLASSIFIED

(U) Global Impact of the 2017 “Tactical Cyber Attack” in Ukraine

UNCLASSIFIED

12

UNCLASSIFIED

[Jun 2017] ‘Fancy Bear’ hackers release malware ‘NotPetya’ in Ukraine

• “It was the equivalent of using a nuclear bomb to achieve a small tactical victory”
• “To date, it was the fastest propagating piece of malware we’ve ever seen” [Cisco]

– Within hours, the worm spread around the world and crippled numerous multinational companies

• Total cost: $10B
• Merck: $870M; FedEx (TNT Express): $400M; Saint-Gobain: $384M; Maersk: $300M; Nabisco and Cadbury: $188M
• Impact to Maersk operations of NotPetya Cyber Attack:

− Created chaos at 17 of 76 ports worldwide causing tens of thousands of shipping trucks to be turned away

− Effectively took down entire global corporate network (4,000 servers, 45,000 PCs, etc.)

− Simultaneously wiped out nearly all of the domain controller servers, which are needed to map its global network and set basic rules for access, except for one in Ghana (because of a local blackout which prevented NotPetya from spreading)

Source: https://www.wired.com/story/notpetya-cyberattack-ukraine-russia-code-crashed-the-world/

“Almost everyone who has studied NotPetya, however, agrees on one point: that it could happen again or even reoccur on a larger scale…Global corporations are simply too interconnected, information security too complex, attack surfaces too broad to protect against state-trained hackers bend on releasing the next world-shaking worm.” -Andy Greenberg, Wired

FIGURE IS UNCLASSIFIED

FIGURE IS UNCLASSIFIED

(U) Example – Cyber Risk: Impact to Maersk Business Operations from 2017 Cyber-Attack

UNCLASSIFIED

13

UNCLASSIFIED

(U) Maersk CEO’s Perspective: “It is time to stop being naive when it comes to cybersecurity.

I think many companies will be caught if they are naive. Even size doesn’t help you.”

​

Platforms

IT/Network/Applications

Operational Technology

Business Operations

Commercial Infrastructure

Impact to Operations: 20% drop in shipping volume – managed 80% percent of volume manually – with help from customers

Impact to Earnings: $200M - $300M

Business Applications Impacted: E-mail, invoicing, systems for sharing system rates, online track and trace, and customer support phone lines that transport and logistics operations depend on

IT Infrastructure Rebuild: 4000 new servers, 45,000 new PCs, 2,500 applications

FIGURE IS UNCLASSIFIED

FIGURE IS UNCLASSIFIED

(U) Real World Degradation to DoD Global Logistics Mission� /Commercial Airlines: Notice to Air Missions (NOTAMs)

UNCLASSIFIED

14

UNCLASSIFIED

FIGURE IS UNCLASSIFIED

Commercial Critical Infrastructure

DIB, FAA NOTAMs system, communications, transportation sector

Priority DoD Missions (NIST Tier 2)

Global Logistics

DoD Critical Infrastructure

​

Military airport, DoD installations

DoD Networks (NIST Tier 3)

Defense Internet NOTAM Service (DINS)

Civil Reserve Air Fleet, Supply Chain Risk Management, Commercial NOTAM Management Tool

DoD Weapons Systems / Platforms

USTRANSCOM AOC

DoD Organizations (NIST Tier 1)

USTRANSCOM, USAF

Incident 1

Nov 2022

1

2

3

4

Cyber-attack degrades data integrity for commercial NOTAM management tool; company takes tool offline in response

CRAF carrier does not possess trusted NOTAMs data

CRAF carrier delays, cancels flights while NOTAMs tool offline

PAXs delayed

USTRANSCOM

Global Logistics

FAA NOTAM System

Omni Air

Jeppesen NOTAM Management Tool

1

2

3

4

1

2

Incident 2

Jan 2023

1

Poor database management allowed an inadvertent input error to degrade FAA NOTAM System data integrity; FAA issues ground stop until it can ensure data integrity

2

DoD airlift unaffected as DoD possesses its own NOTAMs system: DINS

System offline for ~14 hours,

~1,100 PAX delayed

​

~10,000 flights delayed

FIGURE IS UNCLASSIFIED

First ground stop since September 11, 2001

~1,300 flights cancelled,

(U) Real World Degradation to DoD Global Logistics Mission/Commercial Airlines: Notice to Air Missions (NOTAMs)

UNCLASSIFIED

15

UNCLASSIFIED

(U) The U.S. Government failed to address and appreciate the latent risk-to-mission that remained present after the November 2022

cyber-attack against Boeing’s Jeppesen NOTAM Management Tool that degraded confidence in NOTAMs data. Omni Air operations remain dependent on this Jeppesen-provided service, which enables users to build and edit NOTAMs.

FIGURE IS UNCLASSIFIED

FIGURE IS UNCLASSIFIED

Cyber Risk- to-Mission

Cyber-attack against Boeing’s Jeppesen degraded confidence in NOTAMs;

system offline 14 hours,

~1,100 PAXs delayed

Manifest Risk

The (realized) impacts that cyber-attacks or an exploited vulnerability caused

Latent Risk

The (unrealized) impact a cyber-attack that exploits an unmitigated vulnerability could cause

Database management errors degraded confidence in NOTAMs’ integrity;

~1,300 flights cancelled,

~10,000 flights delayed

Incident 1

Nov 2022

Incident 2

Jan 2023

Adversarial Cyberattacks Degrade Capabilities

in Pre-kinetic Phases of Conflict

​

Agenda

• Challenge of Operating in a Contested Cyberspace Environment

​

• Our Approach to Assessing Cyber Risk to Mission
• Maturing the Cyber Warfare Analysis Ecosystem
• CWD Mission Resilience Wargame Series (2017 to the present)
• Critical Success Factors for Wargame Execution
• Significance for the Acquisition and Operational Communities
• Mission Resilience Analysis Process and Outputs
• Mission Level Wargame Execution
• Cyber Risk to Mission (CRM) Metrics
• Wargame Models, Simulations, and Visualizations
• Wargaming Infrastructure
• Wargame Models and Simulations
• Wargame Visualizations

​

• Summary and Way Ahead

​

​

Agenda

• Challenge of Operating in a Contested Cyberspace Environment

​

• Our Approach to Assessing Cyber Risk to Mission

​

• Summary and Way Ahead

​

​

Cyber Warfare Analytic Ecosystem

UNCLASSIFIED

19

UNCLASSIFIED

Body of Evidence

Domain Knowledge

Analyst Work Force

Analysis Community Infrastructure

Conceptual Framework Measures of Effectiveness and Performance

Assessment Methodology

Models and Tools Community Fora

…

Body of Literature Testbeds

Wargame M&S Environment

Simulation Environments Weapons Ranges

…

Education - Training Credentials

Mentors

Codes of Best Practice

…

Lessons Learned Case Studies Experimental Results

Wargames

…

Operational Concepts, Doctrine, Tactics

Operational Effectiveness Force Posture and Structure

Weapon Capabilities and Performance

…

Threats and Scenarios Asset Vulnerabilities

Task-Mission Dependencies

…

“Physics” of the Cyber Domain Cyber Systems Dynamics

CWD Mission Resilience Wargame Series

Critical Success Factors for Wargame Execution

UNCLASSIFIED

21

UNCLASSIFIED

Over the course of the 7 MLCRAs executed by OUSD(A&S), a number of key factors have proven critical for successful execution.

Customer Engagement

Senior Leader Advocacy

AO-Level Support

Disseminating Findings

Quick Look Report Release ~ 1 Month

AAR release ~ 6 months

Analysis

Operationally relevant impacts

Qualitative and Quantitative

Recommendations and Mitigations

Prioritize based on importance and sensitivity

Quick wins when possible

Stay engaged with relevant participants and sponsors

Administrative

Reach out early and often for participants

Practice makes perfect

Design

Build in possibility for flexibility and creativity

Balance between SMEs and wargame practitioners

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

Significance for the Acquisition and Operational Communities

22

UNCLASSIFIED

• Cyber Risk to Mission, starting with the highest priority DoD missions, must be captured, known, and understood by decision-makers in both the Operational and Acquisition communities.
• This improved understanding can guide and prioritize funding decisions to mitigate cyber risks to key-enabling systems, platforms, and installations.
• At senior levels, many commanders are aware that they have blind spots with respect to risk and want to incorporate CRM into their decision calculus.
• MLCRA (wargames) can help so commanders justify using money to fund the right personnel, inform Integrated Priority Lists, Joint Urgent Operational Needs (JUON), Joint Emergent Operational Needs (JEON), and Issue Papers.
• MLCRA results and visualizations are designed for commander and J3 consumption, unlike other highly technical cyber risk efforts across DoD.

​

UNCLASSIFIED

23

UNCLASSIFIED

Research

(7 Months)

• Understand combatant command missions
• Model systems of interest and cyber terrain
• Coordinate with SMEs

Plan

(4 Months)

• Develop red campaign based on identified cyber vulnerabilities (red cell)
• Organize blue defense of key assets (blue cell)

Execute

(1 week)

• Hold wargame capstone event
• Observe Red and Blue player interactions
• Simulate cyber effects

Assess

(6 Months)

• Examine mission degradation
• Characterize cyber risk-to-mission

20%

0%

100%

80%

60%

40%

1 2 3 4 5 6 7 8 9 10

Ability to Conduct Operations

Planned vs. Actual Operational Effectiveness

Days Since Cyber-Attack Occured

​

Actual Planned

70

50

60

80

60 60

10

0

20

30

100

90

80

70

60

50

40

30

20

10

0

Asset 1 Asset 2 Asset 3 Asset 4 Asset 5

% Assets Available

Functionality at Beginning and End of Assessment

Starting Availability Ending Availability

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

Note: Data is notional and does not represent MLCRA wargame findings

Example Analysis Outputs

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

D- 60

D- 50

D- 0

SYSTEM AVAILABILITY

D- 40 D- 30 D- 20 D- 10

DAYS PRIOR TO KINETIC CONFLICT

MISSION EFFECTIVENESS

Mission Resilience Analysis Process and Outputs

Mission Level Cyber Wargame Execution

UNCLASSIFIED

24

UNCLASSIFIED

Operations & CONOPS

Scenario & Threat

Mission Mapping Models

Blue Tactical Team:

• System Operators, Engineers and Network Defenders
• Respond to attacks and scenario injects

Blue C2/Policy Team:

• Higher Headquarters for Operations and Cyber Defense
• Policies, authorities and Mission Assurance

White/Control Team:

• Sponsors, System and Policy SMEs
• Regulate Gameplay
• Adjudicate actions

Red Team:

• IC, System SMEs and Attack/Red Team SMEs
• Campaign Plan
• Attack planning and execution

Architecture

& Vulnerabilities

Cyber Risk to Mission Assessment

GRAPHIC IS UNCLASSIFIED

GRAPHIC IS UNCLASSIFIED

Model-Driven Wargaming Infrastructure

25

UNCLASSIFIED

• CWD’s Model-Driven Wargaming Infrastructure is currently built around two major JHU APL-developed software components, Dagger and Cyber AWARE.
• Dagger is a modeling and visualization tool suite that depicts how losses of cyber and cyber-enabled capabilities (e.g., system failures) impact mission tasks, plans, and risk.
• Dagger supports analysis by facilitating:

- Interactive creation and visualization of hypothetical scenarios such as

“What happens if this system, server, or facility goes down?”

- Sensitivity analysis computation that shows the most important items supporting a

target mission, helping to identify critical infrastructure and key terrain

• Cyber AWARE is a wargame management and cyber situational awareness software suite that enables visualization of player actions and effects, enables Blue Cell defenders to inject responses, and allows the Red Cell to dynamically create and inject new attacks during the game.  
• Cyber AWARE uses Dagger as both the playing board (the attackable terrain) and as a scoring mechanism that calculates the impact to mission

​

Wargaming Models and Simulations

26

UNCLASSIFIED

These Models and Simulations were created to:

• Represent the chain of Risk Management Activities
• Calculate Values for the CRM-related Metrics
• Provide ‘real-time’ inputs to other models and simulations
• Feed visualizations to provide Situation Awareness
• Create an integrated transaction log to support further

​

The Set of Models include:

• Mission Stack Dependencies
• Cyber Terrain
• Engagement
• Capability

​

C2 and Decision Making provided by Humans in the Loop

​

Visualizations are feed by the M&S Outputs

​

​

Role of Mission Resilience Models and Simulation

27

UNCLASSIFIED

The MR suite of models represents an understanding of the relationships between cyber and cyber-enabled capabilities, mission tasks, and mission outcomes. This understanding constitutes the Domain Knowledge pillar of the Cyber Warfare Analytic Ecosystem. Collectively, these models and simulations:

​

• Capture and synthesize what is known about parametric values and relationships from multiple sources
• Generate a transaction log that facilitates CRM assessment and analysis
• Diagnose problems and answer questions at different levels of analysis
• Provide limited support for sensitivity analyses to answer ‘what if’ related to changes in people, processes, technologies, resources, and/or adversary capabilities and behaviors
• Serve as a hands-on discovery tool to learn about and facilitate discussion of cyber warfare, Multi-Domain Operations, and operating in a CCE
• Highlight the most needed data collection, analysis, and research

​

Mission Stack Dependencies and Visualization

28

UNCLASSIFIED

This is an example of a

Dagger generated display

fed by the set of M&S

developed and instantiated

for a given War Game

CRM Visualizations: CRMT

29

UNCLASSIFIED

This Sankey Chart driven by the Cyber Risk Management Tool (CRMT).

It can also be used to provide visualizations to wargame players or relevant Combatant Commands to demonstrate how risk and cyber capability flows from adverse impacts on installation capabilities to Combatant Commands.

Wargaming Contributions to Cyber Warfare Analytic Ecosystem

UNCLASSIFIED

30

UNCLASSIFIED

Body of Evidence

Domain Knowledge

Analyst Work Force

Analysis Community Infrastructure

Conceptual Framework Measures of Effectiveness and Performance

Assessment Methodology

Models and Tools Community Fora

…

Body of Literature Testbeds

Wargame M&S Environment

Simulation Environments Weapons Ranges

…

Education - Training Credentials

Mentors

Codes of Best Practice

…

Lessons Learned Case Studies Experimental Results

Wargames

…

Operational Concepts, Doctrine, Tactics

Operational Effectiveness Force Posture and Structure

Weapon Capabilities and Performance

…

Threats and Scenarios Asset Vulnerabilities

Task-Mission Dependencies

…

“Physics” of the Cyber Domain Cyber Systems Dynamics

Cyber Risk to Mission (CRM) Metrics

31

UNCLASSIFIED

Minimum Set

of CRM-related

Effective-ness Metrics

​

*Agility

includes the impact of the following:

Responsiveness

Resilience

Flexibility

Versatility

Adaptiveness

Innovativeness

Mission Planning and Execution

Functional Capability Consequence Mitigation

Restoration

Asset Hardness - Remediation

Deterrence, Prevention, Suppression, Maintenance

Potential Threats and Hazards

Events

Assets Damaged/Degraded

Degraded Assets not Restored in Mission Time

Mission-relevant Functional Capability

Cyber Risk to Mission

Agility*

Effectiveness of Functional�Consequence Mitigation

Effectiveness of Restoration

Effectiveness of Remediation –

Hardening -- Defense

Effectiveness of Deterrence and Prevention

CRM-related MOEs

**Events

include

Cyberattacks

Accidents

Hazards

CRM Assessment Ecosystem and Metrics

32

UNCLASSIFIED

​

• Provides a basis for a consistent cyber warfare lexicon
• Highlights key roles of the physical-kinetic, cyber, cognitive, and social domains
• Measures Impact of C2 on cyberspace operations and supported kinetic missions in the context of a contested cyberspace environment
• Identifies potential dependencies between and among cyber warfare-related processes and metrics
• ‘Metricizes’ Mission Threads
• Identifies key value drivers (variables whose values potentially have a large impact on MOPs / MoEs
• Provides a method for generating cyber warfare-related hypotheses and a basis for testing them

​

CRM Visualizations

33

UNCLASSIFIED

This is an example of a

display that depicts the levels of capability over time have changed for

- Cyber Assets

- Mission Essential Tasks

- Joint Functionality

- OPLAN execution

​

Agenda

• Challenge of Operating in a Contested Cyberspace Environment

​

​

• Our Approach to Assessing Cyber Risk to Mission

​

​

• Summary and Way Ahead

​

​

Summary and Way Ahead

Summary

• Cybersecurity is National Security; the cyber threat is a clear and present danger
• Cybersecurity for weapon systems/platforms/infrastructure creates unique challenges for �Multi-Domain C2
• Model-Driven Wargames contribute to our understanding of Cyber Risk to Mission and generate potential material and non-material solutions that need to be investigated further

Way Ahead

• Augmenting CWD’s Wargaming capabilities with a CRM Parametric Model-Driven Exploratory Analysis capability to explore “what-If” questions and the expected impact of potential solutions
• Development of an Agent-Based Model for C2 in a Contested Cyberspace Environment that can allow replaying wargames without humans in the loop

​

UNCLASSIFIED

35

UNCLASSIFIED

ICCRTS is a key Cyber and C2 Research Forum whose participants can make significance contributions to to the State of the Art of CRM Assessment and Model-Driven Wargames

​