The Last Mile Index

A public benchmark of agent experience on 30 airline websites

30 AIRLINES · 24 TASKS · 720 RUNS · CLAUDE OPUS 5.5 · SEP–OCT 2026

A companion benchmark to the Last Mile methodology.

AGENDA

What's inside this report

Last Mile · Airline Agent Experience

02 / 17

01

Summary

The headline findings at a glance

02

Methodology

How the Last Mile Index works — scoring, dimensions, the loop, and why it works

03

Results

What agents complete today, and where they stall

04

Diagnosis

Why journeys break — the five core failure modes

05

The field

How 30 airlines rank, top to bottom

06

Next steps

How to get your scorecard and work with us

SUMMARY

Agents can read airline sites, but they struggle with interaction.

66%

of task runs successfully

3 / 30

airlines clear the 80% mark

83 → 35%

pass rate: public reads vs booking funnel

1 in 7

runs gated at a bot or consent wall

Interaction is the challenge. Public information tasks pass ~83%. The moment a task enters the booking flow, the pass rate falls to 35%. Anything behind a sign-in or payment wall is closed to the agent entirely.

Agents get stuck in long, sometimes infinite, loops. 100 of 144 failures ran out of the 40-step budget on long, brittle funnels. The agent doesn’t proactively decide when to give up.

Agents are realistic about task completion. Only 3 self-reported “found” answers failed the deterministic check.

Last Mile · Airline Agent Experience

03 / 17

METHOD

How the Index is scored

THE RUN

30 airlines, 24 tasks

• 720 agent runs, Sep–Oct 2026
• Claude Opus 5.5 in a real browser
• 40 step budget per task
• Tasks span the full customer journey

DEFINING COMPLETION

Three possible outcomes

1. Completed — satisfies a success check written before the run
2. Failed — no verified answer
3. Gated — an honest access wall (sign-in, payment)

THE LENS

Signals & dimensions

• Six friction signals read off each page
• Four dimensions: discoverability, understanding, interaction, transaction
• The agent doesn’t grade itself. Rather, a deterministic check judges task success.

SCORE = COMPLETED ÷ 24 TASKS

Failed and blocked tasks both count against the site; only completed tasks score.

Last Mile · Airline Agent Experience

04 / 17

METHODOLOGY · THE MATURITY MODEL

Decomposing agent experience into four dimensions

DIMENSION

THE QUESTION

WHAT WE TEST

D

Discoverability

Can an agent find you?

Reach the site — and the capability it needs

U

Understanding

Can an agent read you?

Navigate the page and extract the right information

I

Interaction

Can an agent operate you?

Operate forms and inputs; recover from errors

T

Transaction

Can an agent pay?

Checkout and payment — not yet tested

MONEY LINE

This report measures the three dimensions below the money line. Transaction — paying on the customer's behalf — is left for when the technology and user trust mature.

Last Mile · Airline Agent Experience

05 / 17

METHODOLOGY · THE LAST MILE LOOP

A continuous loop to improve the Agent Experience

STAGE 01

Assessment

Can agents complete the task?

STAGE 02

Diagnosis

Where and why do they fail?

STAGE 03

Optimization

How do we improve it?

Note: This report covers Assessment and Diagnosis across 30 airlines. Optimization is where each diagnosed failure becomes a concrete, re-testable fix.

Last Mile · Airline Agent Experience

06 / 17

METHODOLOGY · THE TASK CATALOG

From real customer journeys to measurable tasks

01

Inspiration

e.g. Suggest destinations in budget

→

02

Flight search

e.g. Find the lowest fare

→

03

Compare & select

e.g. Total cost of a trip

→

04

Book

e.g. Review booking before pay

05

Prepare

e.g. Checked-bag allowance

→

06

Check-in

e.g. Provide the check-in window

→

07

Disruption

e.g. Flight status by route

→

08

Support

e.g. File & track a case

24 TASKS span the journey. Each pairs a realistic prompt with a success check written before the run. During the run, it resolves to completed, failed, or gated.

Last Mile · Airline Agent Experience

07 / 17

METHODOLOGY · WHY IT WORKS

Continuous, outcome-driven optimization

01 · OUTCOME-DRIVEN

Outcome-driven

• Measures real task completion rather than static features.
• A deterministic check defines the outcome of each run, before it starts.

02 · REALISTIC

Realistic

• Built from industry-specific customer journeys.
• The path to value differs in every domain.

03 · ACTIONABLE

Actionable

• Every failure maps to a concrete, re-testable fix.
• The next step is always clear, allowing for structured, predictable iteration.

Last Mile · Airline Agent Experience

08 / 17

ASSESSMENT RESULTS

Agents struggle with most airline websites.

Passes 3 airlines

clear ≥80% of 24 tasks — Delta, United, Air Canada

Struggles 23 airlines

complete 50–79% — the broad middle of the field

Last Mile · Airline Agent Experience

09 / 17

Fails 4 airlines

complete under 50% — IndiGo, American, SWISS, All Nippon

ASSESSMENT RESULTS

Success rates per task

LEGEND

≥80% (mostly information tasks)

50–79% (comparison, status)

<50% (booking and checkout)

Last Mile · Airline Agent Experience

11 / 17

ASSESSMENT RESULTS

Interaction across long funnels is the main challenge.

WHY

Public information often lives on the surface of the website. Booking is a long, custom funnel. In addition, anything behind a sign-in or payment wall is not accessible for the agent.

* Two booking tasks were tested:

• “Review booking before pay”: the agent searches a specific flight, proceeds through fare selection into the booking flow, and has to reach the Review & Pay / order-summary screen and read the final all-in total — fare plus taxes/fees, with its currency — that the site shows before any payment.
• “Smart checkout validation”: the agents need to get to checkout and confirm the amount due is shown before payment, explicitly stopping before entering passenger/personal data. It tests whether the agent can validate the final price at the checkout edge without signing in or submitting PII.

Last Mile · Airline Agent Experience

10 / 17

Information tasks

Booking tasks*

DIAGNOSIS · WHERE RUNS BREAK

Agent journeys break in three places

Last Mile · Airline Agent Experience

12 / 17

① THE DOOR

Can it get in?

99

runs blocked

Bot checks and consent walls turn the agent away before it even starts.

→

② THE RUN

Can it do the task?

44

runs fail

It gets in, but can't read, operate, find, or recover. The agent gives up or produces a wrong result.

→

③ THE FINISH

Does it ever end?

100

runs time out

The agent gets into a long loop and burns the 40-step budget without completing the task.

DIAGNOSIS

One in seven runs never gets in

99 runs

gated before the journey could begin

WHAT BLOCKS THEM

Bot walls — “press & hold” / “verify you’re human” checks (48 runs)

Consent overlays — cookie / region walls that trap focus (51 runs)

Last Mile · Airline Agent Experience

13 / 17

① THE DOOR → ② THE RUN → ③ THE FINISH

DIAGNOSIS

Once inside, agents fail in four ways

Last Mile · Airline Agent Experience

14 / 17

The 44 runs that failed mid-task break in four ways:

Can't read the answer

UNDERSTANDING

Results that can’t be scored by our deterministic success check, e.g. a fare quoted in QAR, a duration written “1h30”, etc.

Can't operate a control

INTERACTION

A bespoke date-picker or custom form the agent cannot drive.

Can't find the capability

DISCOVERABILITY

No findable path to check-in, flight status, or a support case.

Fails silently

INTERACTION

An error surfaces no signal — the agent assumes success and moves on.

① THE DOOR → ② THE RUN → ③ THE FINISH

DIAGNOSIS

The biggest failure: agent journeys that don’t end

100 / 144

runs hit the 40-step budget without ever finishing

Last Mile · Airline Agent Experience

15 / 17

THE PATTERN

The agent errs around instead of giving up. Successful journeys finish in ~19 steps (median); failed ones burn all 40. The fix is not in letting the user wait longer but in providing a clear, shorter path to task completion.

① THE DOOR → ② THE RUN → ③ THE FINISH

THE FIELD

Last Mile Index for Airlines

TOP PERFORMERS

Delta 92% United 92% Air Canada 83%

Clean reads, operable funnels, prices visible across steps.

HOW TO IMPROVE

Clear the walls — give the agent access to the website.

Shorten the funnel — operable inputs and fewer steps to a price keep runs inside a reasonable budget and timeframe.

Expose key capabilities — a typed tool (e.g. WebMCP) removes the guesswork on the hardest tasks.

Re-measure to validate — every fix should be confirmed by re-running the same task.

Last Mile · Airline Agent Experience

16 / 17

NEXT STEPS

FOR AIRLINES & TRAVEL BRANDS

Get a scorecard and optimize your AX.

We’ll benchmark your full site, give you a prioritized, re-testable fix list, and run the optimization loop with you, turning agent visits into booked revenue.

FOR PARTNERS & THE ECOSYSTEM

Build the benchmark with us.

We’re opening this up: collaborators, feedback and partners to extend the benchmark beyond into new industries, task catalogs and metrics.

lastmileproject.ai

Jd Fiscus | Dr. Janna Lipenkova

Contact: hello@lastmileproject.ai

Last Mile · Airline Agent Experience

17 / 17