1 of 24

SPRITE: A Plan Template Learning Framework

Melinda Gervasio^*, Karen Myers^*, Laura Tam^*, Christian Szatkowski^†

^* SRI International

^† NIWC Pacific

30 November 2023

28th International Command and Control Research and Technology Symposium (2023 ICCRTS)

This material is based upon work supported by the Office of Naval Research (ONR) under Contracts N00014-18-C-7005 and N00014-21-C-2034. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of ONR.

[1]

2 of 24

Motivation

Plans are documented post hoc in PowerPoint, Excel, or Word and the lack of machine-readable representations precludes the use of advanced analytics or AI technologies for improving plan quality and planning efficiency

Previous COA planning tools:

Require users to learn idiosyncratic interfaces

Limit the types of content that could be documented

Disrupt the flow of planning processes

rigid templates

restricted data types

proprietary formats

COA generation is complex and time-consuming: much effort is devoted to routine tasks that could be better spent addressing operational challenges

[2]

3 of 24

AMPT: Advanced Multi-echelon Planning Tool

[3]

4 of 24

AMPT

Benefits

Codifies planning products into digital artifacts with shared representations
Provides effective interactive tools to let planners develop and adjust plans
Facilitates rapid production of common planning products
Paves way for advanced AI and ML tools to provide intelligent assistance

However

Taking full advantage of framework’s power and flexibility requires knowledge of and experience with the tools
Does not reduce the need for planning expertise: operators still need to know what plans require, what events are involved, and the constraints around them
Codifying plans remains time-consuming, even with powerful tools like AMPT

[4]

5 of 24

PAL-CPOF: Lessons from End-User Automation in CPOF

CPOF

A collaborative system for sharing and visualizing data
Derived from SAGE and VISAGE UI research platforms
DARPA Research Program active 1998-2003
Widely used Army system of record

SIGACT Management

SA Monitoring

Asset Tracking

Storyboard Management

Examples of end-user automated procedures in CPOF

Challenge: Powerful, flexible C2 tool but ‘click-intensive’—users engage in many repetitive, time-consuming processes

Idea: Enable users to program by demonstration to extend and customize operation and automate repetitive tasks in the field

[5]

6 of 24

Plan Templates for AMPT: Concept

Generalize plan fragments into reusable, parameterized plan templates that capture related events for a particular task

Create new plans by combining and instantiating plan templates in various ways

plan template = building block

[6]

7 of 24

Understanding User Needs

Templates can positively impact operational planning

Reduce cold start problem
Avoid inadvertent omission of tasks, increase efficiency
Leverage others’ experience/expertise
Applicable to various tasks: Ballistic missile defense, medical, firefighting, training exercises, …

Improving usability

Standardization of terminology
Ability to edit templates, semi-structured search
Context awareness, real-world constraints
Proactive recommendation, automatic adaptation to usage

User Engagement Sessions

December 2020

Objective: Explore concept of plan templates

April 2022

Objective: Obtain feedback on initial design and implementation, new concepts

September 2022

Objective: Iterate on design and implementation, explore new directions

[7]

8 of 24

SPRITE: Specialized Plan Recommendation through Intelligent Template Extraction

Original plan

Template creation UI

Created template

[8]

9 of 24

Plan Generalization: Unification and Variablization

AttendICCRTS Plan

Resources: UA251, Hertz83, Homewood

Location: JHU-APL

Time: 11/27/23–12/1/23

Events:

Fly(UA251, SFO, BWI, 11/27/23 1000–1600)

Drive(Hertz83, BWI, Homewood, 11/27/23 1645–1730)

Stay(Homewood, 11/27/23–12/1/23)

Drive(Hertz83, Homewood, JHU-APL, 11/27/23 0730-0745)

…

AttendConference Template

Resources: ?Flight0, ?Car0, ?Hotel0

Location: ?Loc0

Time: ?TimeInt0

Events:

Fly(?Flight0, ?Airport1, ?Airport2, ?TimeInt1)

Drive(?Car0, ?Airport2, ?Hotel0, ?TimeInt2)

Stay(?Hotel0, ?TimeInt3)

Drive(?Car, ?Hotel0, ?Loc0, ?TimeInt4)

…

The hotel I drive to from the airport is the same hotel I stay at and drive from to get to the conference

Basic Idea: Arguments (assets, times, and locations) that are the same should probably remain the same

The airport I fly to is the closest airport to the city in which the conference venue is located

?City

is-located-in

is-closest-airport

[9]

10 of 24

Resource Generalization

AttendICCRTS Plan

Resources:

UA251: Flight

Hertz83: RentalCar

Homewood: Hotel

Location:

…

AttendConference Template

Resources: ?Rsrc1, ?Rsrc2, ?Rsrc3

ResourceConstraints:

type(?Rsrc1) = Flight

type(?Rsrc2) = RentalCar

type(?Rsrc3) = Hotel

…

In practice, resources are selected based on

Capabilities
Accessibility
Availability

Simplifying assumption: Accessibility and availability determined prior to planning

Often determined by organizational and command structure

Heuristic: Preserve the types of assets used

[10]

11 of 24

Temporal Generalization

0800 0900 1000 1100 1200 1300 1400 1500 1600 1700

(?p0,T9H30M)

?w=(?p0+T1H,T1H)

?a1=(?p0+T2H,T1H30M)

?r=(?p,T1H)

?b1=(?p0+T2H,T1H30M)

?a2=(?p0+T4H,T1H30M)

?b2=(?p0+T4H,T1H30M)

?a3=(?p0+T6H,T1H30M)

?b3=(?p0+T6H,T1H30M)

?a3=(?p0+T8H,T1H30M)

?b3=(?p0+T8H,T1H30M)

?b1=(?p0+T3H30M,T30M)

?l=(?p0+T5H30M,T1H)

?b=(?p0+T7H30M,T30M)

Heuristic: Preserve the relative timing (start, duration) between events

In practice, additional background knowledge often informs the relative timing between events, e.g.,

Preconditions and effects
Simultaneous events
Conflicting events
Additional events
Environmental factors

Conference Day Plan

Conference Day Plan Template

[11]

12 of 24

Location Generalization

Generalized RAS Location

Generalized Plan Location

Circle with centroid = ?v17

offset by (459.348801, 49.677662)

with radius 300

Polygon with centroid = ?v17 offset by (0.0,0.0) and relative coordinates (distance/bearing offset from polygon centroid): [

(301.494935, 9.253117)

(265.107778, 110.104818)

(314.934792, 199.102301)

(250.819953, 276.963065)

(301.494935, 9.253117)]

Point at ?v17

RAS Location

Plan Location

Circle at (4.464888,-122.493439) with radius 300

Point at (0.169234,-127.572784)

Polygon with coordinates [

(4.475874,-126.869659)

(-1.149126,-123.969269)

(-4.137407,-129.066925)

(0.608687,-131.176300)

(4.475874,-126.869659)]

Heuristic: Preserve the relative locations between events

In practice, the choice of location often requires consideration of vulnerabilities and determining decisive points

Broad support is a longer-term endeavor, but identifying and generalizing specific types of decisive points may be feasible

E.g., Identifying chokepoints based on readily available geographic info

[12]

13 of 24

Relationship Discovery

Basic Idea: (Implicit) Relations between plan arguments are often meaningful

is-located-in

is-closest-airport

is-located-in

is-near

is-located-in

ICCRTS 2023

[13]

14 of 24

Relationship Discovery: Inferring task resources

Discovering that the carrier, destroyer squadron, and carrier air wing are all part of CSG9

ShowOfForce(TheodoreRoosevelt, …)

ShowOfForce(DESRON-23, …)

ShowOfForce(CVW-11, …)

Carrier(CSG9) = TheodoreRoosevelt

DestroyerSquadron(CSG9) = DESRON-23

CarrierAirWing(CSG9) = CVW-11

plan fragment

discovered relations

ShowOfForce(Carrier(?csg), …)

ShowOfForce(DestroyerSquadron(?csg), …)

ShowOfForce(CarrierAirWing(?csg), …)

Carrier(CSG2) = DwightDEisenhower

?csg = CSG2

ShowOfForce(DwightDEisenhower, …)

ShowOfForce(DESRON-22, …)

ShowOfForce(CVW-3, …)

DestroyerSquadron(CSG2) = DESRON-22

CarrierAirWing(CSG2) = CVW-3

learned template fragment

new instantiation with inferred values

enables SPRITE to automatically infer the resources to assign to the ShowOfForce tasks

[14]

15 of 24

Relationship Discovery: Generalization over Repeated Events

RAS(Russell, …)

RAS(PaulHamilton, …)

RAS(Preble, …)

RAS(Pinckney, …)

RAS(Kidd, …)

RAS(JohnSMcCain, …)

SPRITE recognizes that the RAS task is performed over every member of the group Destroyer Squadron DESRON-23

DESRON-23 = [

Russell, PaulHamilton, Preble, Pinckney, Kidd, JohnSMcCain

]

IteratedTask(RAS, ?group:DestroyerSquadron, …):

FOREACH ?member in ?group

RAS(?member, …)

It generalizes this into an IteratedTask that can be instantiated for every member of any DestroyerSquadron group

RAS(Gridley, …)

RAS(Sterett, …)

RAS(Dewey, …)

RAS(WayneEMeyer, …)

RAS(Mitscher, …)

RAS(Laboon, …)

RAS(Mahan, …)

RAS(ThomasHudner, …)

?group = DESRON-1

?group = DESRON-22

Members(DESRON-1) = [Gridley, Sterett, Dewey, WayneEMeyer]

Members(DESRON-22) = [Mitscher, Labon, Mahan, ThomasHudner]

[15]

16 of 24

Template Library Browser

[16]

17 of 24

Template Instantiation

Instantiating a plan

Newly instantiated plan

Plan rendered in AMPT

[17]

18 of 24

Template Editing

Change event start and/or duration

z

Change (add/remove) resources assigned to task

Original plan template

[18]

19 of 24

Empirical Evaluation: User Study

9 participants

2 independent variables → 4 conditions

plan complexity (simple, complex)
construction method (manual, template)

within-subjects design (counterbalanced)

2 dependent variables

#mouse clicks
time (seconds)

Experimental Setup

Research Question: Do templates reduce the time and effort needed to create plans?

One-on-one sessions

Participants tasked with creating 4 plans
Training session on AMPT (5–10min) prior to manual condition
Training session on SPRITE (5–10min) prior to template condition

[19]

20 of 24

Experimental Setup

Participants were relative novices ⇒ Simplified planning task to data entry given plan specification

Simple plan requires calculating event start times relative to other events
Complex plan adds event locations and transits between events

Mimics real world TTPs

simple plan spec

complex plan spec

Simple: PHOTOEX

Event: Take Picture�Time: 1600�Duration: 30 Minutes�Participants: CVN, CG, DDG 1, HELO
Event: Maneuver into position�Time: 1 hour 45minutes before Take Picture�Duration: 1 hour�Participants: CVN, CG, DDG1
Event: Join Event�Time: 1 hour before Maneuver into position�Duration: 1 hour�Participants: CVN, CG, DDG1
Event: Comms Check�Time: 15 minutes before Join�Duration: 15 minutes�Participants: CVN, CG, DDG1, HELO
Event: Flight OPS�Time: 1300�Duration: 4 hours�Participants: HELO
Event: Commence Exercise�Time: 30 Min Before Take Picture�Duration: 15 minutes�Participants: CVN, CG, DDG 1, HELO
Event: Finish Exercise�Time: 30 minutes after Take Picture�Duration: 30 Minutes�Participants: CVN, CG, DDG 1, HELO

Complex: Live Fire

Event: Drop off Target at Range �Time: 1400�Duration: 30 Minutes�Participants: Missile Range Facility
Event: Tow Target to Range �Arrival Time: 1400�Speed: 12knots�Participants: Missile Range Facility
Event: Return to Port �Arrival Time: 1400�Speed: 12knots�Participants: Missile Range Facility
Event: Maneuver for Multi-Axis Strike�Time: 1400�Duration: 30 Minutes�Participants: Missile Range Facility, CVN, CG, DDG 1, SURV AIRCRAFT, STRIKE AIRCRAFT
Event: COMMS CHECK�Time: 1400�Duration: 30 Minutes�Participants: TOW SHIP CVN, CG, DDG 1, SURV AIRCRAFT, STRIKE AIRCRAFT
Event: SAFETY CHECK�Time: 1400�Duration: 30 Minutes�Participants: TOW SHIP CVN, CG, DDG 1, SURV AIRCRAFT, STRIKE AIRCRAFT
Event: STRIKE FLIGHT OPS�Time: 1400�Duration: 30 Minutes�Participants: CVN, STRIKE AIRCRAFT
Event: Observation FLIGHT OPS�Time: 1400�Duration: 30 Minutes�Participants: SURV AIRCRAFT
Event: Maneuver for Multi-Axis Strike�Time: 1400�Speed: 16knots�Participants: CVN, STRIKE AIRCRAFT
Event: Maneuver for Multi-Axis Strike�Time: 1400�Speed: 16knots�Participants: CG, DDG 1
Event: Launch Missile�Time: 1400�Duration: 30 Minutes�Participants: STRIKE AIRCRAFT
Event: Launch Missile�Time: 1400�Duration: 30 Minutes�Participants: CG, DDG 1

[20]

21 of 24

Live Fire (complex plan)

Initial AMPT setup

After adding specified events

[21]

22 of 24

Experimental Results

manual

with templates

Planning with templates requires significantly less time and effort than planning manually.

#mouse clicks

time (secs)

Average #clicks and time (sec) to create plan

simple

complex

simple

complex

simple

complex

simple

complex

With templates, creating a complex plan also incurs no more effort than creating a simple plan

manual

with templates

[22]

23 of 24

Experimental Results

Planning with templates also results in fewer errors

All the errors were in plans created without templates

Errors in plans created by participants

[23]

24 of 24

Summary and Conclusions

Plan template learning is a powerful concept for operational planning
Value of plan templates successfully demonstrated via unclassified scenarios
Classified experimentation with fleet operators will continue during FY24
Transition of concepts being explored with Maritime Tactical Command And Control POR

Directions for future work

Full template lifecycle

Proactive recommendation
Automatic adaptation

Multimodal, natural language interfaces for planning

LLM

Dialog Manager

Planning System

[24]