Loyal Servant or Loose Cannon? �Capabilities and Concerns with Autonomous AI for C2 and Crisis Management

Bjørn T. Bakken, Inger Lund-Kordahl, Kristoffer Lie Eide

University of Inland Norway (INN)

• Autonomous AI has the potential to revolutionize C2 and crisis management, as the pivotal capability to support the decision-making cycle in conflict, crisis and warfare.
• However, adding an autonomous capability to generative AI systems comes with an inherent risk. Such systems will be impossible to test fully against unforeseen, hybrid threats.

Research Problem

Research Question

• How can autonomous, generative AI be used efficiently, effectively, and safely within civilian and military (C2) crisis management?

Automation: Cognitive Effort

“When the system functions with no/little human operator involvement: however, the system performance is limited to the specific actions it has been designed to do.”

Autonomy: Cognitive Control

“The degree to which the system has the capability to achieve mission goals independently, performing well under significant uncertainties, for extended periods of time, with limited or non-existent communication, and with the ability to compensate for system failures, all without external intervention.”

​

Automation vs. Autonomy in AI

Endsley (2015, p. 3). https://doi.org/10.13140/RG.2.1.1164.2003

Traditional AI: Rule-based

• Explicit instructions
• Predefined rules
• Specific tasks
• Expert knowledge

Generative AI: Data-driven

• Machine learning
• (Deep) neural networks
• Patterns and structures
• Generates new content

​

Traditional vs. Generative AI

www.upgrad.com/blog/generative-ai-vs-traditional-ai/

• Risk Assessment
• Mapping critical events by probability and consequence
• Qualitative judgements of threats and vulnerabilities
• Identify dependencies and cascade effects
• Preparedness Planning
• Propose and analyze measures for prevention and mitigation
• Analyze demand for resources and capabilities, relative to availability
• Compute optimal strategies, and courses of action (COA)
• Training and Exercises
• Assessment of training needs in relation to requirements
• Exercise program design and scheduling
• Scenario generation and event management
• Evaluative feedback: lessons identified, learned and implemented

​

Roles and Functions for AI in C2

• H1.1: A decision-maker will tend to use more supervised AI when time pressure (TP) is low compared to when TP is high.
• H1.2: A decision-maker will tend to use more autonomous AI when TP is high compared to when TP is low.
• H2.1: A decision-maker will tend to use more classical AI when VUCA is low compared to when VUCA is high.
• H2.2: A decision-maker will tend to use more generative AI when VUCA is high compared to when VUCA is low.

​

Hypotheses

Research Model, Experimental Method

Thanks for your attention!

This study is (partially) �financed by the EU/Interreg, �Sweden-Norway program �SWE: 20363842�NOR: IRSN2023-0026

​

References in original paper