1 of 9

Assurance in Context and Rigor for Employment of Autonomous Systems

Douglas S. Lange, Ph.D.

Distinguished Scientist for Machine Learning/Artificial Intelligence

24 September 2024

Distro Statement A. Approved for public release: distribution is unlimited.

1

Braulio Coronado�Eric Gustafson�Bruce Nagy, Ph.D.

Naval Information Warfare Center Pacific

2 of 9

Commanders will determine whether models developed through machine learning are employed

Distro Statement A. Approved for public release: distribution is unlimited.

2

  • Maintain Alignment
  • Provide Situational Awareness
  • Advance the Plan
  • Comply with Procedure
  • Counter the Enemy
  • Adjust Apportionment

Willard, R., “Rediscover the Art of Command and Control”, Proceedings Magazine, October 2002.

The Contribution of a Commander

3 of 9

Why Machine Learning is Different

Distro Statement A. Approved for public release: distribution is unlimited.

3

  • Models tied to the distribution of their training data
  • Models can be opaque and non-linear
  • Behavioral models suffer from alignment problems
  • ML is brittle with respect to novelty
  • Test and Evaluation, Verification and Validation (TEVV) of ML is not settled science

4 of 9

Assurance of Machine Learning for Autonomous Systems

Distro Statement A. Approved for public release: distribution is unlimited.

4

Figure 1AMLAS Phases [from 5, used with permission of the author]

5 of 9

AMLAS

Distro Statement A. Approved for public release: distribution is unlimited.

5

Figure 1AMLAS Argumentation [from 5, with permission of the author]

6 of 9

Level of Rigor

Distro Statement A. Approved for public release: distribution is unlimited.

6

Table 2 Levels of Confidence [From 6]

7 of 9

LOR Tasks

Distro Statement F: No further distribution unless directed by NIWC Pacific CO/ED or higher DoD authority.

7

Table 1LOR Tasks [From 6]

Distro Statement A. Approved for public release: distribution is unlimited.

8 of 9

What can we tell decision makers using AMLAS + LoR

Distro Statement A. Approved for public release: distribution is unlimited.

8

  • Using context of the operations, we can determine if the arguments for safe, mission-critical, legal, and ethical use remain valid.
    • Is the environment within the distribution used for training?
    • In ways is it different? Are there other arguments that allow for these differences?
    • Is there some additional data collection that could validate use in this environment?
  • Do the models and systems provide the level of confidence required for the mission?
    • Has the development been conducted with the appropriate level of rigor?
    • Are there additional steps that could be taken to gain the confidence necessary? Is there additional monitoring that can be done?
  • How does the confidence level through LoR affect the arguments from AMLAS?
    • Can we argue that given the level of confidence LoR provides, that use in the environment is sound?
    • Does weak rigor undermine any AMLAS argumentation?

9 of 9

9

Distro Statement A. Approved for public release: distribution is unlimited.