1 of 21

Censeo: A Scenario-based Approach to C2 Capability Assessment and Experimentation

November 30, 2023

Brian Krisler, PhD

Raytheon BBN

Brian.Krisler@rtx.com

This document does not contain technology or Technical Data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

 Approved for public release per AFRL Case Number AFRL-2023-5773

RTX Corporation (Corporate) - Unrestricted Content

Align cover image to grid lines

  • Click to insert photo
  • Delete existing photo to select a new one.
  • Contact Advanced Media for assistance with alternative cover images.

© 2023 Raytheon BBN Technologies.�All rights reserved.

2 of 21

2

Identification, evaluation and incorporation of new C2 functionality can be an expensive and time-consuming process

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

3 of 21

3

For decades, the DoD has consistently faced challenges in software acquisition, with certain issues recurring repeatedly

  • Poor cost tracking
  • Poor schedule tracking
  • Ill-defined requirements
  • Limited understanding of the problem space

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

4 of 21

What is our Objective?

4

To develop a methodology that promotes and aids the transition from large, monolithic applications to a more agile microservice based acquisition strategy

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

5 of 21

Agile Software Development

DevOps Loop

  • Development functions on the left
  • Operational functions on the right
  • “Infinity” loop illustrates continuous development and integration
  • Evolution
    • DevSecOps: Moves “security” left
    • Censeo: Move “assessment” left

Censeo is an DevOps approach to move assessment ”left” into development

5

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

6 of 21

Development of Censeo for the C2 Acquisition Process

  • We developed a process to reduce costs associated with the
    • Identification
    • Specification
    • Development
    • Adoption
  • Of new functionality within a Joint All-Domain Command and Control (JADC2) context

6

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

7 of 21

The Censeo Process

7

  • Collect Statistics
  • Present Metrics
  • Map Services to Sources
  • Exercise the system
  • Capture Communications
  • Identify Constraints
  • Define Scope
  • Specify Requirements

Data Modeling

Process Modeling

Procurement

Execution

Evaluation

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

8 of 21

Process Modeling

  • Supports the capturing of high-level communications
  • Doesn’t get caught-up in the low-level operational details
  • Provides insight that is well suited for identifying bottlenecks

Process models direct the focus of interaction at agent coordination and communication.

8

Evaluate Target

Process Modeling

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

9 of 21

Data Modeling

  • Data models define and scope the data requirements
  • Specify the requisite inputs and outputs for tasks
  • Declare the message data types and sources for the interaction

Data models specify what information should flow between each task

9

Evaluate Target: Data Model

Data Modeling

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

10 of 21

Procurement

  • Focus on small “microservice” based solutions
  • Short development life-cycle (i.e., 6-9 weeks)
  • Provide requisite inputs and outputs (data model)
  • Delivered as a container for evaluation

Procurement should focus on small, microservice based solutions

10

Procurement

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

11 of 21

Process Instantiation and Execution

  • Instantiation
    • Map services and message sources to a process
    • Map services to essential assets and resources
  • Execution
    • Start up all the services in the workflow
    • Construct the virtual network to support communications
    • Execute the workflow with notional data

An instantiation and execution framework supports the continuous evaluation of new services

11

Execution

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

12 of 21

Evaluation

  • Collect execution statistics and metrics data
  • Generate evaluator-centric analytics
    • Details of the current executions
    • Comparison against prior executions
    • Highlighting of issues encountered

Comprehensive analytics are essential to proper and efficient service evaluation.

12

Evaluation

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

13 of 21

Case Study: No Strike List

  • Multi-day evaluation
    • 805th Combat Training Squadron at Nellis Air Force Base
    • Subject matter experts in areas critical to Command and Control
    • Core set of developers tasked with developing new C2 technologies
  • Study Objective
    • Identification and transition from manual to automated workflow tasks
    • Do not remove the operator, instead remove the menial tasks from the operator
    • Design and develop a solution over the course of 1-2 days
  • Dynamic Targeting Workflow
    • Target adjudication requires a series of C2 decisions
    • Requires human interactions with external systems
    • No Strike List – Essential to the process; Ripe for automation

​

​

​

13

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

14 of 21

Process Modeling : Extracting the Process Models

  • Met with a select group of squadron members
    • Dynamic targeting operators from Combined Air Operations Center Operations Directorate
    • Developers from the Innovation Directorate
    • Infrastructure experts from the Sustainment Directorate
  • Conducted an informal task-based analysis
  • Conducted a guided classroom exercise
  • Interviewed the operators about daily tasks
  • Developed a Business Process Modeling Notation (BPMN) workflow

Working with Subject Matter Experts is essential for accurate process modeling

14

Process Modeling

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

15 of 21

Data Modeling: Identifying the Data Model

  • Whiteboard exercise with the subject matter experts
  • Essential Input
    • KML file containing all items on the No Strike List
    • Latitude of the potential target
    • Longitude of the potential target
    • Altitude of the potential target
  • Essential Output
    • Yes/No indicating if the coordinates are within a predefined distance of an entity on the No Strike List

Determining the necessary inputs and outputs is essential to success.

15

Data Modeling

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

16 of 21

Procurement: Service Implementation

  • Provided two teams with
    • The service goal (evaluate a region for no strike items)
    • Data contracts
  • Each team had 6-8 hours to develop a solution
  • Two different containerized services developed
    • One in Go, one in Java
  • A third “control” service was developed for human-in-the-loop
    • Forwarded the NSL request via UI alert to a human operator
    • Human performed task as currently done
    • Submitted results back into the process flow

​

Working from data contracts and objective, the teams were free to implement their own design

16

Procurement

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

17 of 21

Execution: Service Instantiation and Execution

  • Each service was tested individually within the process
  • When executed and a 4 Line was selected
    • The newly created microservice was incorporated into the workflow with the input
      • KML
      • Target (Lat/Long/Alt)
    • The service produced
      • Strike/No Strike

17

Execution

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

18 of 21

Evaluation: Service Evaluation

  • Each service was evaluated within the actual process
  • Execution times collected
  • Success/fail tracked
  • Runs selectable for�details

Evaluation provides both high-level and detailed analytics for each execution

18

Human in the Loop

Red: Failed run

Green: Success

Evaluation

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

19 of 21

Case Study Results

  • Goal: Create a new information technology-based solution from scratch for an identified problem
  • In just one day of development
    • Defined an operational workflow
    • Developed new software to fulfill gaps
    • Tested and deployed the software in a notional environment
    • Compared results against a human operator

​

19

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

20 of 21

Conclusion

  • Leveraging concepts from Agile Software is effective for C2 system acquisition
  • Working with SMEs, high-level processes can be extracted from everyday tasks
  • We demonstrated that
    • Higher-level models successfully map to modern software architectures
    • Transitioning from large monolithic solutions towards smaller, more agile microservices speeds up development

​

20

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1

21 of 21

21

Questions?

RTX Corporation (Corporate) - Unrestricted Content

Non-Export Controlled – See Slide 1