Advanced Multi-Agent Management of Diverse Assets using iCOSMOS-Swarm

(Presented September 13, 2023

Revised January 9, 2024)

Presenters:

Trevor C. Sorensen

Eric J. Pilger

THE PROBLEM

2

• Tyranny of Distance
• Need for improved long distance situational awareness
• Need for increased range of swarm effectiveness
• Primary Initial Users:
• USARPAC Data Officers
• UAV Operators
• INDOPACOM
• Tyranny of Information Overload
• Management of swarms is complex and labor intensive for ground controllers and analysts
• Accurate and timely intelligence is crucial for effective military operations (ISR)
• Current satellite constellations are limited in their effectiveness to aid the warfighter due to their lack of mobility and adaptability

“Senior DOD leaders have stated that access to data and information will be critical in the future operating environment… a multidomain approach is required (where U.S. forces would use ground, air, naval, space, and cyber forces to challenge an adversary’s targeting calculus).”

Hoehn, J. R. (2022, January 11). Joint All-Domain Command and Control (JADC2) https://crsreports.congress.gov/product/pdf/IF/IF11493

​

“One main challenge in artificial swarming is the design of systems that, while maintaining decentralized control, have agents capable of (i) acquiring local information through sensing, (ii) communicating with at least some subset of agents, and (iii) making decisions based on the dynamically gathered sensed data.”

Uppal, R. (n.d.). Satellite swarms and Satellite formation flying missions and technology. International Defense Security & Technology. https://idstch.com/space/satellite-swarms-and-satellite-formation-flying-missions-and-technology/

“The Army needs the capability of intelligent autonomous swarms. We have not yet even achieved that with UAVs.”

Army Scientist, USARPAC 2023

​

​

THE SOLUTION: iCOSMOS-SWARM

iCOSMOS-Swarm is an ISR software framework that is efficient, adaptable, and effective in the battlefield of the future

• Efficiency
• Simplify control of swarms by using coordination of semi-autonomous nodes
• Local problem solving reduces decision-making latency
• Adaptability
• Modular “plug-and-play” architecture for integration with existing sensors and software
• Ability to integrate with systems of heterogeneous assets
• Effectiveness
• Goal-oriented rather than control-oriented
• The agility and responsiveness provided by space swarms increase the warfighter’s effectiveness
• This is a force multiplier for the warfighter - increasing battlefield situational awareness
• Global coverage can be achieved by inclusion of satellites to swarm ecosystem
• Swarm operational life extended by on-orbit refuelling
• Autonomous targeting designed to “fill gaps” in desired observation space - critical issue with ISR

​

3

BASE TECHNOLOGY DESCRIPTION

Interstel’s Comprehensive Open-architecture Solution for Mission Operations Systems (iCOSMOS)

4

Mission Operations Support Tool (MOST)

iCOSMOS Executive Operator (CEO)

Other Tools

• Mission Planning & Scheduling Tool (MPST)
• Data Management Tool (DMT)
• Ground Segment Control Tool (GSCT)

Encompasses and integrates all functions of mission operations (including flight software) into single ecosystem

• Mission Design Tool (MDT)
• Flight Dynamics Tool (FDT)
• Test Bed Control Tool (TBCT)

Legacy of Usage

• 2 satellites launched (HiakaSat, Neutron-1), 2 ready (HyTI, Artemis cubesat)
• UH ground station, Mission Ops Center, I&T cleanroom operations
• Autonomous Electric Vehicle System for campus parcel delivery
• BalloonSats with autonomous ocean recovery vehicle
• UAVs (quadcopters)
• Lunar rover (Phase 0 study for Canadian Space Agency)

TRL 5 to 7

POTENTIAL FOR IMPACT

A fully deployed iCOSMOS-Swarm ecosystem for the U.S. Army will consist of local swarms organized as a global network:

​

​

Local Swarms: Enhance the ability of the warfighter to control and interface with a swarm of diverse autonomous assets during surveillance and combat missions into enemy territory

• Goal-oriented swarm behavior is very resistant to counter-measures due to number of nodes, autonomous interconnectivity and reconfigurability
• Rapid integration of additional or replacement nodes into the swarm
• Reduced number of ground controllers
• Coordinated targeting events
• Track specific targets (changing, dynamic, and specific targets)
• Distribute the information to other nodes
• Autonomous updates and coordination of the pointing campaigns

​

Global Network: Multiple interoperating local swarms to enhance coverage on a global scale with assets on water, ground, air and space facilitated by swarms of satellites in multiple orbits

5

Satellite Swarm Changing Formations with Sensors Pointed at Multiple Targets (STTR iCOSMOS-Swarm simulation)

TECHNOLOGY DESCRIPTION - System Architecture

Our software will monitor/control:

• Satellite communications network (SatComm)
• DSCS, Iridium, TDRS
• Constant communication with space segment
• Network of multiple satellite swarms (Swarm)
• Spaced 3 or 4 hours apart in different polar orbits
• Multiple sensors (eg. VIS, IR, SAR)
• Communicate with/through SatComm and HAPs
• Autonomously adjust formation for maximum effect
• Provide updated intelligence
• High Altitude Platforms (HAPs)
• Communicate with space and ground swarms
• Autonomously adjust formation
• Provide updated intelligence
• Ground units in Area of Operations
• Receive commands
• Provide updated status and tasking
• Mission Operations
• Monitor intelligence
• Assess situation
• Command assets

6

TECHNOLOGY DESCRIPTION - Approach

7

NASA STTR: TRL 2 to 6

Every component in iCOSMOS is reduced to its key attributes, and each independent system, or node, is a collection of these components

• iCOSMOS-Swarm gathers groups of nodes into assets and targets
• Assets operate semi-autonomously through the use of simple behavioral rules relating each node to the other, and to their targets
• The concept is symmetrical; targets can become assets and assets can become targets
• This semi-autonomy reduces the need for complex ground operations and decision latency
• Swarm tactics allow for pooling of knowledge, sharing of resources, and resiliency of communication
• Autonomous multi-agent coordination with inherent collision avoidance using Smoothed Particle Hydrodynamics (SPH) and Multi-Agent Coordination (MAC)
• Scalable swarm control for heterogeneous assets in dynamic, hostile environments using robust position estimation using built-in Hierarchical Cooperative Localization (HCL)

SUPERIORITY OF SOLUTION

Why iCOSMOS-Swarm is better than current solutions:

​

Built on software with Flight Heritage that has been field-tested and mission-tried

• iCOSMOS-Swarm tools designed to aid the warfighter in making a complex system manageable (based on space mission operations and computer gaming experience)

Space segment is an active dynamic participant in ecosystem with air, ground, and sea segments

• Over the horizon intelligence extends the reach of the warfighter
• Swarm flexibility optimizes data quality for ISR
• Space-located communications provides resistance to enemy interference
• Protocols developed for intermittent space communications are resilient to loss of communication

iCOSMOS-Swarm is agile and provides a responsive solution for the warfighters’ needs

• Designed as web of information flow with each warfighter operator included as a node
• Individual nodes can operate autonomously, but work cooperatively as elements of the swarm to complete larger swarm missions
• Requires fewer operators
• Modular “plug-and-play” architecture allows direct integration with current Army Ground legacy systems

8

“In a conflict, satellite operations are critical to the command and control (C2), movement and maneuver, protection, and sustainment of space capabilities.”

-Joint Publication 3-14 Space Operations

COMMERCIALIZATION POTENTIAL

​

• Government and Commercial Applications and Academic Research
• NASA, NOAA, Academic Institutions
• Climate science, Lunar and Martian exploration, Earth remote sensing, disaster monitoring
• Shipping, transportation, computer gaming (Interstel forming computer gaming division in 2023)
• Wildfire Management
• The Governor of California’s proposed budget for 2023-24 includes a total of $4 billion to support CalFire
• CalFire’s Office of Wildfire Technology Research & Development (OWTRD) was formed to locate and test technologies to improve the detection and response to wildfires
• According to OWTRD, “public, private, and non-profit organizations and companies work closely with us to advance the application of technology to wildland firefighting.”
• Tribal Forestry Management
• Various Native American Tribes have large swaths of forest that they would like to manage sustainably using their allocated federal funding
• Swarm based monitoring can guide usage through analyzing forest health and potential resiliency
• Agriculture Carbon Sequestration
• In the coming years, agriculture will have a compelling role in the decarbonization of the atmosphere - by trapping carbon in the soil, farmers can improve its quality without the use of harmful fertilizers while also offsetting emissions
• Satellite-based monitoring of this process would be cheaper, more transparent, and a huge asset to farmers looking to adopt carbon-reducing practices

​

9

Interstel’s recent win of $16,000 in Phase I of NASA’s Entrepreneur Challenge indicates a strong potential for the utilization of iCOSMOS-Swarm in climate science data generation.

​

SBIR WORK & MILESTONE SCHEDULE

10

Q1

Q2

Q3

Q4

Q6

Q5

NASA STTR Phase II

Q7

Q8

Develop System Requirements

Develop Top-Level Ops Concept & System Architecture

Develop Design (Space, HAP, and Ground Segments)

Develop System Prototype

Test Prototype System

SBIR Phase III

Seek out Venture Capital for Funding Prototype System (~$1-2M)

Explore Partnerships and Contracts with NOAA, CalFire, etc., and with Army Primes (or Sale to Prime) with help of NSIN or OTAs

Work with Army Experts to Align Technology with Army Needs and for Testing Validation

SBIR Phase II

ΔSystem Requirements Review (SRR)

ΔPreliminary Design Review (PDR)

ΔCritical Design Review (CDR)

ΔTest Readiness Review (TRR)

Final Report Δ

Build and fly prototype satellite swarm with ground integration

Top 3 Phase II Objectives

SBIR Phase II Goal is to develop iCOSMOS-Swarm software & tools then test them with prototype and simulated swarms

​

DRAFT

SBIR

PHASE II WBS

11

TRANSITION RISK MANAGEMENT

• Technical viability of basic swarm system validated in NASA STTR project
• Interstel has contractual experience with:
• DOD (Navy)
• NASA
• Major risk for transition to Army use is obtaining sufficient funding to complete project and sustain company into Phase III. Mitigations include:
• Access to NSIN can help us connect with end users and funding sources
• Sell/license Interstel products to Government or commercial customers (Interstel is NASA Entrepreneur competition finalist for using technology to improve collection of climate science data)
• Obtain investment funding
• Team with or sell company to Army prime contractor

12

TECHNICAL RISK MANAGEMENT

• Technical viability of basic system validated in NASA STTR project
• Major risk is adapting iCOSMOS-Swarm to Army’s needs in support of the warfighter
• Army Phase II SBIR will provide means to adapt our system to Army requirements, thus reducing the major risks
• Army Phase II SBIR deliverables:
• Develop system architecture
• Develop utility simulations
• Modify key software tools for Army applications

13

BENEFITS OF iCOSMOS-SWARM

14

“The power of swarming is more than just numbers—swarms of unmanned vehicles could communicate with each other and adjust their tactics and targets as circumstances change.”

(The Future of the Battlefield, NIC-2021-02493, National Intelligence Council’s Strategic Futures Group, April 2021)

INTERSTEL TEAM (current)

15

Dr. Trevor Sorensen CEO

• 50 yrs experience (NASA, DOD, commercial, academia); 60+ publications
• Expert in mission operations (author)
• Space shuttle guidance & control engineer; assistant flight director; software manager
• Lunar Mission Manager for DOD’s Clementine mission to the Moon
• Program Manager, NRL Space R&D contract
• Technical lead for Honeywell’s commercial satellite ground network
• Observation Manager for DOD’s LACE satellite
• Selected awards:
• NASA Medal for Exceptional Scientific Achievement
• International SpaceOps Organization Distinguished Service Medal
• NRL Certificate of Appreciation
• Fellow, American Institute of Aeronautics and Astronautics (AIAA)
• Fellow, American Astronautical Society (AAS)
• Director, AIAA Space and Missiles Group (2008-2014)

Eric Pilger VP, Technology

• MA in Astronomy
• Lead software engineer of HSFL (Hawaii Space Flight Laboratory)
• 30 yrs experience in planetary science instrument development
• Systems engineer for HIGP (Hawaii Institute of Geophysics & Planetology)
• Architect of low-level structure of COSMOS
• Co-inventor of COSMOS multi-agent software

Dr. Miguel Nunes VP, Engineering

• PhD in Mechanical Engineering
• Aerospace Engineer
• Lead Engineer for HSFL satellites
• Co-inventor of COSMOS multi-agent software
• Developed Multi-Agent Robotic System & Guidance, Navigation, and Control algorithms for space missions
• Systems/Attitude Control Engineer for various SmallSat missions

Dr. Zhuoyuan Song, Research Scientist

• PhD, Mechanical Engineering
• Former director of Robot Autonomy and Navigation Laboratory at UH Manoa
• Research areas: Autonomous navigation, GPS-denied localization, multi-robot systems, marine robotics
• Principal investigator of 10+ extramurally funded research projects with >$2 million share
• Publications in areas including robotics, control, and ocean engineering
• Subcontractor on NASA STTR project

​

Other Team Members:

• 4 Software Developers
• 4 Programming Interns
• 4 Business Interns
• 1 Engineering Intern

Other Service Contributors:

• Accounting/Financial Firm
• Attorney
• University of Hawaii Office of Technology Transfer
• Hawaii Space Flight Laboratory

16

Interstel Technologies, Inc.

Dr. Trevor Sorensen, CEO. trevor.sorensen@interstel.tech / 808-772-2851

Problem Statement - Army

• Management of swarms is complex and labor intensive for ground controllers (“Tyranny of Information Overload”)
• Loss of interconnectivity between heterogeneous assets in hostile environments
• Need for increased range of swarm effectiveness (“Tyranny of Distance”)
• USARPAC Data Officers
• UAV Operators
• Need for improved long distance situational awareness
• Current satellite constellations are limited in their effectiveness to aid the warfighter due to their lack of mobility and adaptability

Solution (iCOSMOS-Swarm)

Dual-Use Opportunity

Why Us?

• Simplify control of swarms by using coordination of semi-autonomous nodes - goal oriented rather than control oriented
• Scalable swarm control for heterogeneous assets in dynamic, hostile environments using robust position estimation with built-in cooperative localization
• Local problem solving reduces decision making latency
• Swarms able to interact operationally on scales from local to global
• Integrates space swarms for global reach and improved responsiveness - more agile than constellations

• Government (NASA, NOAA, State, etc.)
• Climate science data collection for data fusion
• Lunar and Mars exploration (stated applications of STTR swarm technology)
• Earth remote sensing, especially for dynamic or high priority events (e.g., volcanic eruptions, wildfires, earthquakes, cyclones, etc.)
• Cooperative data collection from diverse assets to monitor atmospheric and ocean events
• Law enforcement/border security
• Wildfire management & prevention (e.g., CalFire)
• Academia
• Basic research using cooperative heterogeneous assets
• Commercial Clients
• Operators of satellite constellations interested in cooperative data collection, (e.g., agriculture, forestry, emergency services, environmental & carbon management)
• Operators of shipping, transportation, networked sensors
• Carbon sequestration monitoring (e.g., Carbon Farmer Africa)

​

• iCOSMOS (base technology)
• 2 satellites launched, 2 ready for launch
• Satellite mission ops, ground stations
• Autonomous electric vehicle system, BalloonSat ocean recovery, UAVs
• iCOSMOS-Swarm
• Technology developed and proven in NASA STTR Phase I & II (TRL 2 --> 6)
• NASA Entrepreneurs Challenge Finalist (Technology proposed to improve climate science data collection)
• Time to transition ROM: ~48 months (SBIR Phase II through Phase III)
• $$$ to transition ROM: ~$50M

Army Focus Areas: Network CCCI

USARPAC Data Officers

UAV Operators

INDOPACOM

​

Tech Banner Statement: New mission operations software allows for easier management of swarm resources

​

System architecture diagram depicting interactions between the space and ground swarms in response to long-distance target discovery or changes using the iCOSMOS-Swarm software solution

CONTACT

Dr. Trevor C. Sorensen

CEO, Interstel Technologies, Inc.

trevor.sorensen@interstel.tech

Eric Pilger

VP Technology, Interstel Technologies, Inc.

eric.pilger@interstel.tech

www.interstel.tech�

​

Marina Cleavenger

Business Development, Interstel Technologies, Inc.

marina.cleavenger@interstel.tech

17