Augmented C2 Assessment Using Digital Data

Niels Alston, Dr Alexander Kalloniatis and Dr Cameron Bowles

Human & Decision Sciences Division

Defence Science & Technology Group

​

Presented by Dr Alexander Kalloniatis (Agile C2 STaR Shot)

• Observational notes, interviews and surveys demand lots of resources, both in collection and analysis

​

• Footprint of participants of C2 systems in the digital domain has vastly increased but remains untapped in analytical studies

​

• Manual data collection needs to be timely

2

Introduction

• Digital C2 metrics cannot be developed in isolation from human factors impacting on C2

​

• Focus on planning at the headquarters level

​

• Planning presents itself as a nonlinear, “Wicked Problem”

3

Problem Setting

Rittel & Webber 1973 Wicked problems +

Conklin 2005, Lambert & Scholz 2005

• Identify assessment goals to narrow focus on digital systems

​

• Focus on using existing systems and data

​

• Iterative process, identify what exists and how it might contribute to a measure

​

• Use automatic methods for extracting data

​

• Combine with traditional human factors analysis

4

Methodology

5

Methodology

<Metadata>

<API Requests>

C2 Application Server

Analyst

• Plan MURA -> IOTs
• Assess the impact of new organisational structure
• Assessed within confines of a CPX
• Well defined planning process (JMAP) used

6

Application to the Operational Planning Process

• What digital data can we extract?
• Documents themselves can be analysed
• Document Audit Events:
• Creation
• Opening
• Version Creation (Locking)
• Document Release (Unlocking)
• Publishing
• We can analyse the quality (content) and the usage (behaviour)

​

7

Data Collection – Document Management System

• Dynatrace – A tool for measuring web application performance, can also be used to roughly capture user behaviour
• What digital data can we extract from Dynatrace?
• Number of active users
• Number of actions
• Action duration
• Error rates
• Session duration
• Database connections
• Many more but the prime focus has been on data that may indicate user behaviour
• Not implemented currently, custom user actions (specific URL visits or strings of actions)
• We can analyse broadly how and when the Command Support System is being used over time eg in relation to Battle-Rhythm

8

Data Collection – Command Support System

• NOT real data, shows information access and flow by users based on existing audit metadata trails.
• If mapping exists, can group users to see between group interactions

9

Application to the Operational Planning Process

User to User Interaction

Team to Team Interaction

10

Rittel & Webber 1973 Wicked problems +

Conklin 2005, Lambert & Scholz 2005

Application to the Operational Planning Process

Indicative data showing sequencing of planning artefact development

• Again, NOT real data
• Can track number of active users of specific applications
• See trends in data, higher periods of operational tempo, unusual spikes in activity

11

Application to the Operational Planning Process

• Difficulties in tracing the digital thread of activity between applications
• Differing user ids, metadata formats in different C2 systems

​

• Challenging to combine datasets across classification levels, users may simultaneously operate on separated networks

​

• Ultimate goal is near real or even real-time C2 assessment capability

​

​

• Future Data Platform including HQ observational data, thematic analysis (Nvivo) history of events – build comprehensive picture of performance of the Socio-Technical system

​

• Incorporate results into mathematical models and simulations of C2 systems

​

12

Challenges and Future Work

13

END