1 of 14

ARIA-AT:�Interoperability

Test Suite

TPAC 2022 • 09.14.2022

2 of 14

Agenda

What is ARIA-AT?

  • Overview
  • Example

Status Update

What’s Next?

  • Expanded coverage
  • AT Automation
  • Continuously-updated reports

ARIA spec process

  • Integrating ARIA-AT into ARIA process

3 of 14

What is ARIA-AT?

  • A project to test the interoperability of screen readers and other assistive tech (ATs) using the ARIA Practices Guide design patterns as examples

Tests

Evaluation criteria

for AT rendering

Automation

Headless driving of

ATs to collect results

Infrastructure

Run tests regularly on cloud servers

Reporting

Share results with

ATs & developers

4 of 14

The ARIA-AT homepage at aria-at.w3.org.

5 of 14

Why is AT interoperability important?

  • Ensures AT users can reliably and consistently browse the web and closes the gap between those who use ATs and those who don’t
  • Helps AT and browser vendors find, prioritize, and fix bugs
  • Simplifies AX testing for web developers and helps them make informed decisions about support with interop data

6 of 14

Running an ARIA-AT test plan and recording results.

7 of 14

ARIA-AT Example

Test Plan: Modal Dialog

  • Test: “Close a modal dialog”
  • Command: “Press the ESCAPE key”
  • Expected Behavior: The dialog should close and the AT should announce the role and name of the button that toggles the modal dialog
  • Results: VoiceOver for Safari, JAWS / Chrome, NVDA / Chrome all have FULL support

8 of 14

Status Update

9 of 14

ARIA-AT in 2022

  • Test Suite: Coverage of most impactful APG patterns
    • Draft test plans for 45 of 65 APG examples for NVDA, JAWS,and macOS VoiceOver
  • Test Running: Manual results collection and review
    • First 8 design patterns have been tested across VoiceOver, JAWS, and NVDA, with 2 or more independent manual testers confirming output and results
  • Reporting: First round of reports
    • Candidate reports at aria-at.w3.org/reports
  • Automation:
    • Built generic AT Automation driver & drafted AT Automation API standard

10 of 14

What’s Next?

11 of 14

ARIA-AT in 2023 and beyond…

  • Test Suite: Coverage of APG patterns, ARIA Roles, ARIA States & Properties, HTML
  • Test Running: Automated test running with manual review
  • Reporting: Continuously-updated report, tracks new AT versions and new browser versions
  • Syndication: Embedded tests results in ARIA Practices Guide and elsewhere, e.g. MDN Web Docs
  • Standards: ARIA spec-writing process includes ARIA-AT tests and AT Automation API is standardized and implemented
  • Automation: Multiple ATs can be programatically automated out-of-the-box

12 of 14

ARIA Spec Process

13 of 14

ARIA Spec Process

Proposal: start writing AT tests early in the ARIA spec-writing process

  • Potential benefits:
    • Ensure more complete understanding of intent of ARIA features
    • Help check that ARIA features are capable of providing intended user benefit
    • Simplify implementation by AT vendors
    • Day 0 interoperability would improve author adoption

14 of 14

Please join the ARIA-AT Community Group to get involved!