Non-Visual Access to Information:�Turning Space into Time

Sile O’Modhrain

School of Information

School of Music, Theater, and Dance

University of Michigan

sileo@umich.edu

Overview

• What is a Screen Reader?
• A Design Dilemma
• A Word on Designing Web Pages
• Encountering Web Pages in Time
• Accessibility Guidelines Reconsidered
• Beyond Existing Guidelines – the Impact of AI

​

​

A Design Dilemma

• The output of screen-reading software is synthetic speech 

​

• Consequently,   the information presented in static form on a screen is only present for as long as it takes to be spoken. 

​

A Design Dilemma

The design challenge is therefore to convert information structured in space, for a 2D screen, into information that is structured in time, to be spoken.  �

A Thought Experiment

How do you imagine someone approaches reading a screen using screen-reading software?

Example

Live screen-reader demo

A Word on Designing Web Pages

What are the priorities for the designer of a website?

A Word on Designing Web Pages

• Web pages are laid out to capture and guide a user's visual attention to particular icons, images or text.  
• Assumes that, on landing on a page, users will quickly skim its content before focusing their attention on the item they are looking for. 

�

A Word on Designing Web Pages

If web designers and advertisers can capture a user's attention, they can "funnel" the user to particular content.

​

In short, websites rely on a model best described as "context before focus”. �

Encountering Web Pages in Time

• When a screen-reader user lands on a web page, they are aware of only one element, the one they happened to land on.
• This element can be anywhere – in the frame, in the content area, ...
• Often, but not always, it is the first link on the page, and it may or may not be labeled.
• It depends on the HTML code, and how this code is translated into the off-screen model that the screen reading software draws upon.�

Encountering Web Pages in Time

•The user’s first task is to figure out where they are.

​

• they must work outwards from their current position to find out what is around their cursor.

​

• Their model is therefore opposite to that of sighted users – Focus before Context

​

• And SR users must also build a mental model of spatial layout from information picked up over time.

Encountering Web Pages in Time

• In using a screen-reader,  the names of web elements and the content of the page must be listened to, not simply glanced at.

​

• The spatial layout of the page is disassembled and element descriptions become interleaved with content in a single spoken stream.

Encountering Web Pages in Time

•The listener often has to selectively ignore verbose tag labels that interrupt sentences and take precious time to be read.

​

•All the relational information embedded within a page’s spatial structure is lost and must be reconstructed in the mind of the user, adding an additional cognitive burden.

​

(Demo)

Encountering Web Pages in Time

• Studies have shown that even straightforward tasks take longer for visually impaired people to accomplish
• This is not because they are less capable or less skilled at using their technologies,
• It is  because SR users  are constrained by an interaction paradigm  that is entirely temporal  (in contrast to the predominantly spatial paradigm available to sighted users).  (Bigham et al, 2007)

​

Encountering Web Pages in Time

In Sum:

​

•Making a website or application accessible for those using screen reading software is largely about structuring information to be efficiently consumed in time, as opposed to space. 

Accessibility Guidelines Reconsidered

​

WCAG stands for Web Content Accessibility Guidelines.

a set of internationally recognized standards that provide recommendations for making web content more accessible to people with disabilities.

​

These guidelines are developed by the World Wide Web Consortium (W3C). 

​

Question: What do you think website accessibility guidelines are for?

Accessibility Guidelines Reconsidered

• Accessibility guidelines for website designers endeavor to overcome the challenge of converting space into time
• By urging designers to use HTML tags such as heading structures, form elements and so on, they create a template for off-screen models for screen readers.
• Screen reading software can then access these templates to construct hierarchies of information.
• SR users can then navigate these hierarchies using shortcut keys, e.g. heading levels,  to gain an overview of a page's structure and content. 

Accessibility Guidelines Reconsidered

• Shortcuts for jumping to tables, buttons, check boxes, and so on; can be combined with SR specific text search functionality.
• If you know what you are looking for, this greatly speeds up the process of building an understanding of a website.

​

(Demo)

Accessibility Guidelines Reconsidered

•this assumes that website designers adhere to accessibility guidelines. 

•When they fail to do so, three things can happen:

​

1. Elements that are not labeled are effectively invisible

2) Cognitive load is greatly increased

3) Interaction takes more time

(Bigham et al, 2007)

​

Accessibility Guidelines Reconsidered

1) Elements that are not labeled are effectively invisible

• If a screen reader cannot access a label for a button, it is just labeled "button", and if the button is not rendered using standard code, it may not be visible to the screen reading software at all. 

•And the screen reader user will likely not even know that a button is missing.

Accessibility Guidelines Reconsidered

The result is that screen reader users may not be able to determine:

1)  Whether the information they need is present but not accessible

2) Whether it is just difficult to access an element or 

3) Whether the information is not present at all �(Bigham and Savage, 2017)�

Accessibility Guidelines Reconsidered

2) Cognitive load is greatly increased

• Auditory information does not persist in space like visual information, and is therefore not available to refer to (Giraud et al, 2018).
• In order to navigate websites, blind users must therefore hold both their structure and content in working memory
• This, Giraud et al argue, vastly increases the cognitive load associated with negotiating content on the web.

Accessibility Guidelines Reconsidered

3) Interaction takes more time

Bigham and colleagues compared the time taken by sighted and SR users to complete a number of tasks on the web

They found that:

• SR users took more than twice as long to generate a query from Google’s home page, and more than four times as long to locate and click on a result once the result page was loaded (Bigham et al, 2007)

Accessibility Guidelines Reconsidered

On the positive side:

​

When designers adhere to standards,

•users’ experiences are greatly improved

•they can benefit from much more useful strategies for navigating content.

Accessibility Guidelines Reconsidered

2021 WebAim Survey

​

• In 2021, the WebAim organization surveyed 1568 screen reader users as part of their continuing effort to track changes in preferences across a range of factors 
• The survey included questions relating to how people use accessibility features to navigate unfamiliar web pages.  

�(Note: WCAG, the Web Content Accessibility Guidelines, is the international standard with which websites must comply to be certified as being accessible for screen reader users.  More details can be found here: https://www.w3.org/WAI/standards-guidelines/wcag/ Accessed: 18 May 2022)

​

Accessibility Guidelines Reconsidered

Headings

When asked which features they use to find information on a lengthy webpage:

• 68% of users said that their primary method was to navigate through headings on the page,  (rising to 76 Per Cent for those who self-reported as being advanced screen reader users)
• 85% of users said that they found the presence of heading levels to be particularly helpful in gaining an overview of how information on a website is organized.  
• Only 8% of users reported reading through the website line-by-line (predominantly novices).

Accessibility and AI

• As you can imagine, AI solutions are being developed to address issues around accessing the web, application interfaces and documents.

​

• I can, for example, use an app to find out more about the FedEx and Xfinity websites:
• Demo (Be-my-eyes)

​

Accessibility and AI

In the fall, the makers of JAWS will introduce their own AI-based solution, FS Copilot.

​

It will provide information about:

• Structure – The AI will describe the structure of a page or document
• Content – the AI will provide a content summary of the page
• Navigation – the AI will provide hints about how to navigate the page using JAWS.

Conclusion

•People’s preference is to find time-efficient strategies for consuming web content and for avoiding having to listen to information that is not relevant to their current task goal.

• WCAG guidelines provide the design framework to support this time-based way of negotiating information on the web

• With the advent of AI, users may also have tools to analyze poorly-designed webpages and application interfaces themselves to overcome those instances where they currently get stuck.

•While AI can help, it will never be as good as a document that is properly designed in the first place.

​

Beyond Existing Guidelines

• The WebAIM study mainly addressed the navigation of information based on existing HTML tags.
•  Other studies  have considered how information might be better presented for speech-based reading.

​

Beyond Existing Guidelines

Sighted web users often employ strategies for skimming or scanning website content  to quickly get a gist of what is being communicated.  These may include:

�1) Looking through visually prominent content such as images and illustrations, along with their titles and captions.

2) Reading the first and last sentences of each paragraph.

3) Scanning through the entire content and picking out salient phrases, slowing down and accelerating depending on what they are looking for.

4) Diagonal reading, in which the reader glances diagonally across a page, picking out salient phrases as they go. (Ahmed et al, 2012)

​

Beyond Existing Guidelines

Whatever the chosen strategy, the aim is to absorb information quickly without compromising overall comprehension.  

Beyond Existing Guidelines

Ahmed et al (2012)  investigated the use of automatic techniques for generating content summaries.  They concluded that:�•A successful speech-oriented approach should be able to extract meaningful words and combinations of words from every sentence, while  preserving word order throughout the extracted text.  �

Beyond Existing Guidelines

While semantic parsing is not available in off-the-shelf screen reading software, some basic strategies for summarizing text have been implemented.  

�In Jaws, for example, users can configure a "skim" mode to read the first lines or sentences in each paragraph on a webpage or in a document, and can also export these summaries for later reading.

�(For a description of this functionality for Jaws screen reading software, see:

https://www.freedomscientific.com/SurfsUp/Skim_Reading.htm

Accessed: 20 May 2022)

Beyond Existing Guidelines

In short:

�•There is now a substantial body of work indicating that, given sufficient structural support within documents and web pages, screen-reader users employ similar strategies to organize the way they encounter information in time.  

�•But this is not the primary mode of interaction for sighted designers who encounter their information laid out and formatted in space, in a modality that provides continually available contextual framing for their actions.  

�

Beyond Existing Guidelines

In short:

�•For all but the most conscientious designers, the process of making documents or websites accessible is an exercise in rule-following, in making sure that certain tags are in place for screen-readers. �

Returning to the Design Challenge

•Designers of web pages who are not themselves screen reader users lack the lived experience of encountering information primarily in time.

�•In addition, accessible solutions always lag behind current mainstream technology adoption because they are often implemented after the mainstream technology has been released. 

Returning to the Design Challenge

• This implementation lag, often measured in years, is accompanied by a conceptual lag where solving problems relies on techniques that mainstream HCI has long abandoned:

�•the checklist-style engineering approach that underpins compliance with WCAG standards for web design  represents a throwback to first and second wave HCI methodologies. 

�•To disrupt this approach, we will have to decouple the implementation lag from this conceptual lag 

(Power et al, 2018). 

A Proposed Design Approach

•Their proposal is to adopt a third wave approach to inclusive design by recognizing the wide variety of lived experiences of users with disabilities.  

�•They suggest providing users with the tools they need to customize their own environments, to shape their own user experience.  

�•For web browsing, this might involve:

1 combining tools for automatically parsing information in a document or on a website,

2) providing ways for users to customize what and how they read according to their current task goal.

Take Home

•A Third-wave design approach for web accessibility is more respectful of user's time and attention.

•It also prioritizes the fluid browsing experience that sighted users take for granted.

•But it will require web designers to learn to present information in time, not space.

Acknowledgements

• Hrishikesh Rao
• Alexa Seu
• Dan Fan
• Sean Follmer
• Emelia Piane�

References

• Ahmed, F.  Borodin, Y., Puzis, Y.,  and IRamakrishnan, I. V. Why Read if You Can Skim: Towards Enabling Faster Screen Reading
• Jeffrey P. Bigham, Anna C. Cavender, Jeremy T. Brudvik, Jacob O. Wobbrock, and Richard E. Ladner. 2007. WebinSitu: A Comparative Analysis of Blind and Sighted Browsing Behavior. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (Assets ’07). ACM, New York, NY, USA, 51–58. 
• Jeffrey P. Bigham, and  Irene Lin Saiph Savage (2017)  The Effects of “Not Knowing What You Don’t Know” on Web Accessibility for Blind Web Users ASSETS'17, Oct. 29–Nov. 1, 2017, Baltimore, MD, USA
• Faisal Ahmed Yevgen Borodin Yury Puzis I.V. Ramakrishnan
• Power, C., Cairns, P., Barlet, M. (2018). Inclusion in the Third Wave: Access to Experience. In: Filimowicz, M., Tzankova, V. (eds) New Directions in Third Wave Human-Computer Interaction: Volume 1 - Technologies. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-73356-2_10
• Silva, J., Freire, A.P., and Figueira Cardoso, C. (2022) When Headers Are Not There: Design and User Evaluation of an Automatic Topicalisation and Labelling Tool to Aid the Exploration of Web Documents by Blind Users. In 19th Web for All Conference (W4A’22), April 25–26, 2022, Lyon, France. ACM, New York, NY, USA, 11 pages.
• WebAim Survey #9, June-July 2021. https://webaim.org/projects/screenreadersurvey9/�