1 of 19

A challenging case for Artificial Intelligence: evaluation of the benefits of AI-enabled socio-technical concept solutions for Operational-level planning (050)

Patrick Turner, Holly Roberts & Amy Jones – QinetiQ

Andrew Leggatt & Simon Attfield - Trimetis

Richard Ellis – RKE Consulting

Martin McMillan, Rachel Asquith & Albert Forsey – Faculty AI

25 September 2024

Figure © Faculty AI

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

2 of 19

2

Opportunity Knocks

(043)

The Machine Speed C2 HAC

(041)

A Challenging case for AI

(050)

Assessing novel C2 socio-technical AI enabled planning concepts

(057)

A set of guidelines for designing Decisive Conditions in military Operational Planning

(044)

Modelling Arguments and Battlespace Entities in C2 planning

(008)

AI techniques in Gaming for C2

(023)

Underpinning C2 approach

Evaluation methods

Concept development

C2 concepts

Low TRL AI opportunities

Data concepts

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

3 of 19

Share two ideas for how Operational-level planning could be improved

Summarise progress in bringing these ideas to life as AI-enabled solutions

Describe the Defence benefits of implementing these ideas

3

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

4 of 19

Contents

Research Drivers & Perspectives
Concepts & Concept Solutions
Auto-Piggery Concept & the Stakeholder Mapper Concept Solution
The Systems Approach Concept & Support to Operational Design Concept Solution
Evaluations
Benefits Maps
Conclusions & Reflections
Next Steps

4

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

5 of 19

Research Drivers & Perspectives

Drivers

Desire to explore the potential benefits of Artificial Intelligence (AI)

Realising the promise of machine-speed processing, greater access to data and the potential to address human biases

Address a range of C2 challenges, e.g.

Grappling with complexity
Exploiting AI and data

Perspectives

C2 is a socio-technical system
Complexity demands new approaches to Operational-level planning
Operational-level planning is a ‘challenging case’ for AI
The C2 HAC is a design pattern for future C2 capability
Operational-level planning should strive for utility rather than accuracy
AI provides a supporting role in human-agent interaction within Operational-level planning

5

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

6 of 19

Concepts and Concept Solutions

Each concept solution was brought to life as a socio-technical system, through the design, development and integration of:

Human planners
A proof-of-concept (PoC) functional AI agent
An interaction AI agent to broker/manage interactions between humans and PoC functional agents
Activity workflows for humans and agents, as appropriate
Human-agent interactions, supported by human-machine interfaces (HMI)

All concepts focused upon Operational-level planning

A concept is an idea, expressing how something might be done or accomplished

A concept solution provides a functional design for a concept – and can be evaluated

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

7 of 19

7

CONCEPTS

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

8 of 19

Auto-Piggery Concept

Problem/challenge:

Planners need to develop understanding of stakeholders in complex Operating Environments – including counterfactual thinking / “what if..?”

Operational-level Planning Context:

Mission Analysis

Concept:

Interaction between planners and an AI agent – to aid planners’ understanding of how stakeholders perceive, and may respond to, critical issues
Example critical issues: blue intervention, key ‘plays’ by other stakeholders, key events e.g. elections

8

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

9 of 19

Stakeholder Mapper Concept Solution

A socio-technical system in the Operational-level planning domain to develop understanding of multiple stakeholders’ interests in, and potential reactions to, a critical issue – through human-agent interaction, supported by data
Planners ‘in the lead’, agent in a supporting role

9

Stakeholder Mapper Proof-of-Concept Agent functionality
Based upon a choice of a closed-system, locally hosted Large Language Model (LLM) (Falcon 40B) or proprietary LLM accessible via an API (GPT-3.5)
Provided with a small corpus of relevant open-source information and a transcript of planners’ Mission Analysis dialogue
Includes a graphical / text-based Human-Machine Interface (HMI) through which a critical issue is entered and outputs are presented.
Exploits Named Entity Recognition to identify stakeholders with an interest in the Critical Issue. Such interests are produced using the LLM with Retrieval Augmented Generation (RAG) using the information described above. This both reduces hallucinations and controls what information is used to form responses.
Isolates the specific parts of the source documentation that were used as context for producing stakeholder interests (explainability)

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

10 of 19

Stakeholder Mapper Concept Solution

10

Figures © Faculty AI

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

11 of 19

The Systems Approach Concept

Problem/challenge:

Planners need to develop deep understanding of complex Operating Environments and design effective Operations

Operational-level Planning Context:

Mission Analysis, Evaluation of Factors, Centre of Gravity Analysis, Operational Design

Concept:

The application of thinking about the operating environment as a Complex Adaptive System, supported by AI and data.
Supports a richer and deeper understanding of the Operating Environment
Fosters an outcome-focused and systemic approach to Operational Design

11

End-State

Operational Centre of Gravity (CoG)

Decisive Condition (DC)

Example Operational Design

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

12 of 19

Support to Operational Design Concept Solution

A socio-technical system in the Operational-level planning domain to develop an Operational Design, including the generation, description, analysis and representation of a set of DCs that address the Operational CoG and realise the Operational End-State, through human-agent interactions, supported by data
Planners ‘in the lead’, agent in a supporting role

12

Support to Operational Design Proof-of-Concept Agent functionality
Based upon a closed-system pre-trained Large Language Model (LLM).
Provided with a small corpus of relevant open-source information and previous planning outputs (Mission Analysis, CoG Analysis)
Includes a text-based Human-Machine Interface (HMI) through which outputs are presented.
Exploits Retrieval Augmented Generation (RAG) to generate a long list of proposals for DCs, including title and description.
Isolates the specific parts of the source documentation and planning data that were used as context for producing DCs (explainability).
Analyses DCs it has produced by applying guidelines developed through expert elicitation.

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

13 of 19

Support to Operational Design�Concept Solution

13

Figures © Faculty AI

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

14 of 19

Evaluations

Purposes: develop evidence for C2 benefits and guide further development
Formative (not summative) evaluation methods
Participants: two ex-Operational Level planners
Input data: based upon Op HUSKY (Allied invasion of Sicily, 1943)

Audio/transcripts and planning products from five days of Operational-level planning
Selected Op HUSKY historical analysis documents
Guidelines on the development of Decisive Conditions

Concept solutions ‘brought to life’ and ‘put to use’

PoC Agents, activity workflows, interactions supported by HMI

Laboratory settings

Realistic enough but not real-world
Appropriate given low maturity of concept solutions

Evaluation did not impose tight controls on human activity or human-agent interaction

Flexibility in execution of workflows and management of interactions

Data were collected through both observation of planning activity and structured dialogue with the planners, post-activity

Shaped by a set of research questions (next slide).

14

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

15 of 19

Research Questions – C2 Benefit

In what ways does the Stakeholder Mapper concept solution develop understanding of stakeholders’ interests in critical issues?
In what ways does the Support to Operational Design concept solution generate Decisive Conditions that have utility to Operational-level planning?
Given that DC generation, as a human activity, existed before the PoC agent was developed, how did the effectiveness of the Support to Operational Design concept solution change when the PoC agent was introduced?

15

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

16 of 19

Stakeholder Mapper:�Benefits Map

16

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

17 of 19

Support to Operational Design:�Benefits Map

17

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

18 of 19

Conclusions & Reflections

C2 benefits

Both concept solutions offer benefits to Operational-level planning
Planners and agents were, collectively, more effective than either would have been alone

Agents added value to human activities as carried out by experts

Value comes from:

Rapid processing of large datasets
Novelty of agent output rather than accuracy – when used by expert planners
Explainability
Relevant input data

AI and data

Large Language Models / generative AI can be exploited to support Operational-level planning
The type of C2 data used are currently not readily available

Planning dialogue
Planners’ cognitive approaches

18

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY

19 of 19

Next Steps

User-centred co-design and co-development – including with current military planners and their parent HQs
Generation and curation of relevant C2 data
Evaluation in more ecologically valid settings with methods that generate higher levels of evidence:

Introduce a novel scenario
Work with participants who have had no previous exposure to the scenario
Work with participants who have differing levels of expertise in Operational-level planning
Introduce real-world constraints such as time-pressure and sparse data

19

29^TH INTERNATIONAL COMMAND AND CONTROL RESEARCH AND TECHNOLOGY SYMPOSIUM

QINETIQ CONTROLLED PROPRIETARY