1 of 1

[1] Chaudhuri et al. (2025). TripCraft: A benchmark for spatio-temporally fine-grained travel planning. ACL.

​

[2] Xie et al. (2024). TravelPlanner: A benchmark for real-world planning with language agents. ICML.

​

[3] Choi et al. (2026). ATLAS: Constraints-aware multi-agent collaboration for real-world travel planning. ICLR.

​

UTP-Bench: Uncertainty-aware Travel Planning Benchmark

Etcharla Revanth Rao1, Priyanshu Karmakar1, Shubhojit Mallick2 , Manish Gupta2, Shreya Ghosh1, Abhik Jana1

IIT Bhubaneswar1, Microsoft2

Why UTP-Bench?

Code & resources

(UTP-Bench)

A valid itinerary can still

be a fragile itinerary.

  • Travel delays can disrupt planned connections.
  • Crowds can increase visit and waiting times.
  • Different travellers need different safety margins.

Research question

Can LLMs plan appropriate time buffers to handle real-world travel uncertainty?

Real uncertainty signals

Historical transport delays and hourly crowd patterns.

Human reference plans

1,000 queries with itineraries refined by 14 trained annotators.

Robustness evaluation

BAS, CATS and TDAS reveal failures hidden by static pass rates.

What the benchmark adds

Coverage

504

Indian cities

33

states / UTs

1,000

travel queries

20

uncertainty-data cities

  • 2,990 attractions
  • 3,433 restaurants
  • 3,670 accommodations
  • Multimodal transport connections

Traveler risk profiles

Risk-tolerant

Tighter schedules, smaller buffers

Risk-optimized

Moderate buffer allocation

Risk-averse

Larger safety margins

UTP-Bench

Travel databases + empirical uncertainty + traveler risk profile

Three Metrics for Robust Travel Plans

BAS

Buffer Adequacy Score

Does activity slack match visit duration, crowd conditions and the traveler’s risk profile?

CATS

Crowd-Aware Timing Score

Do attraction visits avoid peak crowd windows and favor quieter periods?

TDAS

Transport Delay Absorption Score

Do transport buffers absorb expected delays without excessive slack?

Hourly crowd-level distributions across attractions and restaurants.

Robustness across Trip Duration

Final Pass Rate

References

Agentic Baseline

​

  • UTP-Bench evaluates uncertainty-aware travel planning
  • Crowd avoidance is stronger than temporal robustness
  • Longer trips are challenging
  • ATLAS improves shorter trip pass-rate but can not consistently improve uncertainty handling suggesting that agentic planning alone is insufficient

​

Conclusion