Human–AI Teaming in Aviation Safety

Using Simulation to Strengthen Human-AI Teaming in Aviation Safety Critical Decision Making

Elita HUYNH

SKYLAB Innovation Manager, Thales

Director, Simulation Australasia

Philip SWADLING

Avionics Technical Director, Thales

Former Chair, Simulation Australasia

Safeskies Conference | May 2026

"What an incredible burden it is to be a

moral creature."

In aviation, we do not merely operate machines — we carry responsibility for lives.

The question is no longer "Can the machine decide?" but "Which decisions must remain human?"

Moral Responsibility

Humans carry the burden of accountability

AI Enabled Systems

Systems operate within ethical bounds

Governance

Authority allocation is the key

What Makes a Team?

Teams are not built on equality — they are built on differentiated duties

The key to effective teaming lies in understanding that responsibility is asymmetric — a safety feature, not a flaw.

Captain ↔ First Officer

Ultimate authority

Support & execution

Clear command hierarchy ensures safety

Handler ↔ K9

Strategic control

Sensory capability

Asymmetric responsibility is normal

Human ↔ AI System

Moral accountability

Operational execution

Governance of judgment

Asymmetry of responsibility is a safety feature

Team Dynamics

Human Factors Risks in AI-Assisted Decision Making

Identifying concrete vulnerabilities when AI supports pilots in safety-critical contexts

Automation Bias

Over-trust in AI advisories, confirmation bias, and significantly reduced active vigilance.

Skill Degradation

Decline in manual flying proficiency causing slower recovery in reversionary modes.

Mode Confusion

Uncertainty about system state or lack of clarity regarding active authority transitions.

Complacency

Delayed anomaly detection when the AI performs nominally for extended periods.

Design Implication

Mitigate via clear mode annunciation, authority-transition cues, trust-calibration training, and scenario-based simulation.

Human Factors

Cognitive Load and Situational Awareness

Maintaining the optimal band of human engagement in highly automated environments

COGNITIVE RESILIENCE

AUTOMATION LEVEL

Too Low

Overload

Human error

Optimal Band

Shared control

High Situational Awareness

Too High

Underload

Skill fade

Key Risks of Underload

Loss of Situational Awareness

Slower Anomaly Recognition

Brittle Authority Hand-backs

Remedies:

Deliberate Cognitive Training

Rapid Handover Practice

Active Explainability Use

Latency & Workload Metrics

Cognitive Resilience

Low

High

Simulation — Strengthening Human–AI Teaming

In Human–AI teaming, simulation becomes the primary environment for AI assurance evidence

Simulation has always been central to aviation training and safety validation. But in the context of human–AI teaming, its role expands significantly.

1. Role Allocation Rehearsal

Validate who decides/advises/overrides, including at the boundary conditions where AI confidence is low or the system is operating outside its nominal envelope.

2. Quantitative Evaluation

Capture compliance rates, response latency, workload indices, and decision accuracy across systematically varied scenarios. This is the evidence base for certification arguments.

3. Qualitative Evaluation

CRM training must evolve. Simulation-based debriefs and cognitive load assessments provide the qualitative dimension for understanding human-AI interaction.

4. Boundary Condition Testing

The conditions most likely to produce safety incidents — degraded sensors, conflicting advisories, authority transition failures — cannot be safely explored in live operations.

5. Trust Calibration

Over-reliance and under-reliance are equally hazardous. Scenario design can deliberately calibrate the trust relationship — building in moments where following the AI is correct.

6. Safety Governance Evidence

Under NPA 2025-07 and the EU AI Act, high-risk AI systems require documented human oversight assurance. Simulation produces the structured, repeatable evidence trail needed.

Simulation Framework

Simulation Scenario Design Principles

Using simulation to strengthen Human-AI teaming in aviation safety-critical decision making by designing scenarios that expose and manage real risks.

1. Nominal Operations

Establish trust-calibration baselines, verify AI advisory quality and explainability instrumentation under standard operating conditions.

2. Boundary Conditions

Test AI confidence thresholds, degraded sensors, conflicting advisories, data dropouts, and critical edge-of-ODD scenarios.

3. Authority Transitions

Rehearse hand-off and take-over protocols, mode changes, and annunciation/timing/acknowledgment loops between human and AI.

4. Off-Nominal Events

Emergencies, system faults, MEL constraints, and automation disengagements; ensure retained human accountability under extreme pressure.

5. Cognitive Challenge Scenarios

Time pressure, ambiguity, and workload spikes; monitor situational awareness and decision quality to prevent cognitive overload.

Design Considerations

Required fidelity (aircraft/AI), scenario-coverage matrix across phases of flight, progressive difficulty (simple → complex), repeatability, and embedded explainability cues.

Scenario Design

Simulation Fidelity Requirements

Strategic allocation of simulation resources to capture high-validity human-AI teaming evidence.

High-Fidelity Imperatives

AI Activity Realism

Simulations must run the exact AI logic, latency, and confidence algorithms intended for actual flight operations.

Sensor Degradation Modeling

Accurate replication of noise, signal spoofing, and edge-cases that trigger AI boundaries and prompt authority transitions.

Crew Environment (HMI)

Displays, auditory alerts, and physical controls must be physically identical to capture valid cognitive load and reaction times.

Acceptable Medium-Fidelity

Aircraft Dynamics

Standard aerodynamic models are generally sufficient for testing cognitive processes, unless flight anomalies directly couple to the AI.

Motion Platforms

Fixed-base simulators are often adequate for evaluating cognitive load, decision-making quality, and supervisory teaming tasks.

Out-the-Window Visuals

Can be simplified in scope and detail, provided visual-meteorological conditions are not a primary input parameter for the AI system.

Cost-Benefit Analysis Guidance

Focus heavy capital investment on the cognitive and digital interface layer (AI models, edge-case sensors, and precise HMI). Optimize training ROI by utilizing fixed-base or mixed-reality simulators instead of expensive full-flight physical motion systems where possible.

Simulation Strategy

Measuring Human-AI Teaming Performance

Comprehensive measurement framework connecting simulation results to certification evidence (EASA NPA 2025-07, EU AI Act)

Quantitative Metrics

Procedures compliance rates

Response latency

Decision accuracy

Workload indices (NASA-TLX)

Eye-tracking/fixation patterns

Authority-transition success rates

Intervention timing

Error rates

Qualitative Assessments

Structured debrief interviews

Cognitive-load self-reports

Situational-awareness probes (SAGAT-style)

Trust-calibration surveys

Narrative justifications for key decisions

Behavioral Observations

Monitoring discipline

Override frequency and rationale

Consultation-to-execution ratios

Manual take-over smoothness

Communication quality in crew/AI interactions

Certification Mapping

Aggregate into a structured safety case with traceability from Scenario → Evidence → Assurance Claim (Trustworthiness, Oversight, Accountability, Explainability).

Training Progression and Trust Calibration

Pathway for using simulation to build reliable Human-AI teaming in safety-critical decision making while AI systems remain ethically bounded and humans retain accountability.

Phase 1

Introduction

Understand AI capabilities/limits, ODD, explainability cues, authority rules.

Phase 2

Guided Practice

Instructor-led scenarios, practice consultation/cross-checking, feedback on justification.

Phase 3

Progressive Autonomy

Increase complexity and AI authority within boundaries, monitor reliance.

Phase 4

Boundary Testing

Deliberate failures, degraded sensors, authority-transition challenges, rehearse rapid take-over.

Phase 5

Proficiency Validation

Certification-level scenarios with behavioral checks, documented assurance evidence.

Trust Calibration Checkpoints

After Phases 2-4, adjust thresholds, cues, and procedures to correct over- or under-reliance. Set gating criteria, recency requirements, and SMS integration for operational deployment.

Training Pathway

CLOSING VISION

Aviation has always been a discipline of delegated trust.

Human–AI teaming introduces a new kind of trust:

not trust in the machine, but trust in the boundaries we place around it.

AI systems do not carry moral burden. Humans do.

Simulation is where we prove that burden remains governable.