Anatomy of a testing report

Sample AUL report

https://docs.google.com/document/d/1Dw0Yeo8ZiK-xD1t_Shj_TG2uj1IXPIsFrNWe3qg40l0/edit

Description of element and its behavior

When the user submits the form and the input is incorrect, a list of error messages is displayed in a modal window that is timed to disappear after five seconds.

Description of behavior with AT

The modal window does not receive screen reader focus, cannot be found with a screen reader, and is completely inaccessible to screen reader users.

Screen reader and OS in use

NVDA 2017-4

Mozilla Firefox 58.0.2

OS X 10.11.6

Tester details

Blind / low vision

Highly experienced with screen readers

Highly experienced with IT in general

Native in NVDA

Not familiar with OS X

Reference to a standard

Corresponding WCAG requirement:

2.1.1 Keyboard: All functionality of the content is operable through a keyboard interface without requiring specific timings for individual keystrokes, except where the underlying function requires input that depends on the path of the user's movement and not just the endpoints. (Level A)

​

Classification of problem

Severe

​

1. Severe issues: problems that violate accessibility standards and make core parts of application inaccessible.
2. Significant issues: problems that violate accessibility standards and make the application difficult although not impossible to use.
3. Usability issues: problems that don’t violate standards but that introduce user interface problems that make the application less intuitive and easy to use.

Technical analysis of why the problem occurs

The JavaScript that renders the modal window….

Location of problem

The code causing the error is on lines 123-125 in the file badcode.js

Remediation recommendation

The modal window should receive screen reader focus when it opens, should be findable in the DOM, and should not be on a timeout but have a close button that the user can control when they want.

Location of solution

The problem can be resolved by removing lines 123-125 in the file badcode.js and replacing it with

[some good code]

Description of the user experience

When a user tries to submit the contents on one page, a list of errors comes up, but there is no indication that the pop up is present and there is no way to access the list.

User experience in context

As long as there are errors, a user cannot continue to the next part of the form. As a result, the user does not make any progress, but does not know why.

​

User emotions

The convoluted way of reporting errors is frustrating and confusing for the user.

User quotes

“I want my dog to eat it”

Challenges

Amount of detail

Level of technical information

Recommendation depth

Amount of experience description

Emotional tone of report