The Silver Files:

A Summary of Research

Intro

Who? The W3C Silver Task Force and Silver Community Group

What? Silver is the next major version that will be the successor to WCAG.

How? 12 months of research by the academic partners and by the members of Silver.

What’s Here? The summary of the key research findings and links to the original research if you want more information

Scope? This research is related to structure. Content research is ongoing.

Silver

Content

• Easier to use
• Easier to reference
• Easier to understand
• Expanded scope
• Inclusive of more disabilities
• Inclusive of more perspectives
• Inclusive of more technologies
• Useful for more people

Process & Structure

• Based on evidence and data
• Broadly communicate our efforts
• Clear project milestones
• Inclusive communication paths
• Inclusive of more perspectives
• Involve more stakeholders
• Broader international participation
• Easier to maintain and update
• Clearer update path

Research by Topic

Usability

Usability is a grading factor on the ease of use — specifically toward task completion or a goal.

​

Usability is defined by 5 quality components:��Learnability�Efficiency�Memorability�Errors�Satisfaction

Usability 101: Jakob Nielsen

Problem Statements on Usability

• Too Difficult to Read
• Difficult to get started
• Ambiguity in interpreting the success criteria
• Persuading Others to use WCAG

Usability Problem Statement #1�Too Difficult to Read

Undefined acronyms, specialized terms, pseudo-legalese, and complex sentence structure decrease user’s comprehension, especially for people in the development cycle who are less technical (project managers, designers, social media marketing leads), regulators, and international users who need translations.

Detailed Problem Statement for “Too Difficult to Read”

Research on Difficult to Read

• Silver needs to use plain language.
• “Legalese” is not needed for regulators, lawyers or policy makers.
• Other forms of presentation are needed other than a “wall of text”

Usability Problem Statement #2�Difficult to get started

WCAG is so complex in structure and content with documents, layers and resources that it is overwhelming to people who want to use it as a reference. Among other issues, it also difficult to search for – across multiple documents, and search within. This can be intimidating for people new to the topic who are genuinely interested in and / or tasked with supporting accessibility.

Detailed Problem Statement for “Difficult to get started"

Research on Difficult to Get Started

​

• Silver needs an on-ramp for beginners
• Silver needs to use plain language
• Silver needs better search capability
• Silver needs role-based organization

​

​

Usability Problem Statement #3 �Ambiguity in interpreting the success criteria

There isn’t a clear distinction on what is the “right answer”. Even accessibility experts disagree. As the technology and contexts that can make requests and output web content continues to expand, it becomes less clear over time, which contexts the guidelines apply to. A browser is now only one of many such contexts.

Detailed Problem Statement for “Ambiguity"

Research on Ambiguity

• It is difficult to get accessibility experts to agree
• Silver needs more updated examples across newer technologies
• Some success criteria are difficult to learn and use.
• The goal to make WCAG 2.0 “technology agnostic” resulted in a higher level of abstraction in guidance and testability.

​

Usability Problem Statement #4 �Persuading Others

Demonstrating that accessibility is not only important to people with disabilities, but that it also benefits the business as a whole can be a challenge. The fact that accessibility is required by law does not necessarily influence decision makers to invest in accessibility. There are many compelling reasons for this, but ultimately, it can be difficult to calculate or predict the business and human impact within any given industry.

Detailed Problem Statement for “Persuading Others"

Research on Convincing Others

• Silver needs materials to help with empathy
• Silver needs materials to help with the business case for accessibility
• Many of the material already exists, but is difficult to find.

Conformance

The structure of the guidance and how to measure if it is met.

“Conformance to a standard means that you meet or satisfy the ‘requirements’ of the standard. In WCAG 2.0 the ‘requirements’ are the Success Criteria. To conform to WCAG 2.0, you need to satisfy the Success Criteria, that is, there is no content which violates the Success Criteria.”

WCAG 2.0 Understanding Conformance�

Problem Statements on Conformance

• Strict Testability and Constraints on guidance
• Consistent Human Testable Results
• Accessibility Supported
• Evolving Technology

​

​

Conformance Problem Statement #1

Strictly testable

Certain success criteria are quite clear and measurable, like color contrast. Others, far less so. The entire principle of understandable is critical for people with cognitive disabilities, yet success criteria intended to support the principle are not easy to test for or clear on how to measure. Most of the existing criteria in support of the understandable principle are designated as AAA, which relatively few organizations attempt to conform with.

Detailed Problem Statement for “Strictly Testable"

​

​

Research on Strictly Testable

• The strict testability requirement of WCAG success criteria provides a structural barrier to including more needs of people with disabilities.
• Silver needs to decouple testable and conformance so that Silver can give guidance for more disability needs.
• Testable guidance is still needed, both for developers to know they “got it right” and for regulators, policy makers and lawyers to measure the accessibility of a website.
• There is still a use for levels of conformance to help people prioritize.
• Some types of content are a higher priority for conformance than others (e.g. login screens, emergency notifications).

Conformance Problem Statement #2

Human Testable

Regardless of proficiency, there is a significant gap in how any two human auditors will identify a success or fail of criteria. Various audiences have competing priorities when assessing the success criteria of any given digital property. Knowledge varies for accessibility standards and how people with disabilities use assistive technology tools. Ultimately, there is variance between: any 2 auditors; any 2 authors of test cases; and human bias. Some needs of people of disabilities are difficult to measure in a quantifiable way.

Detailed Problem Statement for “Human Testable"

​

​

Research on Human Testable

• Silver needs more test cases
• Silver needs a process where experts can agree on interpretation across evolving technologies.
• Silver needs a scalable method of certifying results of conformance claims.

Conformance Problem Statement #3

Accessibility Supported

‘Accessibility supported’ was never fully implemented in a way that was clear and useful. It requires a harmonious relationship and persistent interoperability between content technologies and requesting technologies that must be continuously evaluated as either is updated. It is poorly understood, even by experts.

Detailed Problem Statement for “Accessibility Supported"

​

​

​

​

Research on Accessibility Supported

• Accessibility Supported is poorly understood
• Accessibility Supported does not give adequate guidance to assistive technologies, particularly in non-native English-speaking countries.
• Accessibility Supported accepts all assistive technologies for conformance, even if they are not easily usable by people with disabilities.

Conformance Problem Statement #4

Evolving Technology

As content technology evolves, it must be re-evaluated against assistive technology for compatibility. Likewise, as assistive technology evolves or emerges, it must be evaluated against the backward compatibility of various content technology.

Detailed Problem Statement for “Evolving Technology"

​

​

​

​

Research on Evolving Technology

• Silver needs a flexible design that will allow for updates as new technologies emerge, assistive technologies improve, and changing technologies produce new barriers for people with disabilities.
• The implementation of IoT solutions should focus on the use of personal smartphones as the primary IoT interface device for students with disabilities (in post-secondary education).
• Universities need to ensure that IoT technologies are interoperable with student’s own technology, particularly AT and to address the challenges to privacy and security.

Maintenance

WCAG 2.0 was published in December 2008. It is getting its first update in 2018. In 10 years, the web has changed and the needs of people with disabilities have changed. More timely updates are needed.

Problem Statements on Maintenance

• Flexibility
• Scaling
• Governance

​

Note that there were few research conclusions on maintenance issues, but other research and experience of contributors to the Silver research inform these statements. �

Maintenance Problem Statement #1

Flexibility

Strict document structure, workflow and approval processes can sometimes ensure efficiencies and predictable outcomes. However, they may also stifle innovation and inclusion. Additionally, this process can be intimidating for some and limit participation by PwD.

Detailed Problem Statement for “Flexibility"

Comments on Flexibility

• Silver needs a flexible design that will allow for updates as new technologies emerge, assistive technologies improve, and changing technologies produce new barriers for people with disabilities.
• Silver needs a flexible process that will encourage more international contribution and contribution by people with disabilities.
• Silver needs the ability to update in a more scalable way, with greater inclusion of input.

Maintenance Problem Statement #2

Scaling

Updates and additions to WCAG end up proving difficult to include in while keeping the overall structure entirely intact (numbering & levels).

WCAG was not created with the anticipation of scaling in a way that would combine, absorb or incorporate the related specs for web content, user agent and authoring tools or any future accessibility guidelines.

Detailed Problem Statement for “Scaling"

​

​

​

Comments on Scaling

• Silver needs a more modular design that would allow inclusion of new platforms (hardware or software).
• The goal to make WCAG 2.0 “technology agnostic” resulted in a higher level of abstraction in guidance and testability. Silver could consider a different model.
• It is difficult for a working group to keep up with rapidly changing technologies and Silver could consider how to utilize a more “open source” model.

Maintenance Problem Statement #3

Governance

Accessibility guidelines have not keep up with trends, new directions, and expanding scope of technology. While WCAG 2.1 is relieving some of the pressure of built-up demand from people with disabilities for better guidance to meet their needs, many user requests (some more than 10 years old) could not be included in 2.1. Input of new ideas and implementation into accessibility guidance takes years of work using the current methods of standards development.

Detailed Problem Statement for “Governance"

​

​

​

​

​

Comments on Governance

• Silver needs a more modular design that would allow inclusion of new platforms (hardware or software).
• Silver should examine the processes used by other W3C Working Groups (like the HTML5 or CSS specs) for more efficient methods.
• Silver should have a regular public update schedule.
• Silver should consider a process where people can submit user needs or barriers encountered so they can be scheduled for consideration and inclusion.
• Silver could consider a more open source method of maintaining the spec and supplementary materials.

Research Sources

Silver Design Sprint Resources

�Job Stories

�Problem Statements

​

​

WCAG Use by UX Professionals

WCAG Usability - a survey of 121 UX and accessibility professionals by Peter McNally, Bentley University. In partnership with McNally, Silver did their own analysis of the anonymized data.

​

​

Silver Analysis of the McNally data

​

WCAG Success Criteria Usability Study

This survey asked participants to evaluate WCAG 2.0 success criteria by easy to learn, remember and teach. Results were analyzed to identify which SCs are easiest and hardest overall to learn, remember and teach.

The Internet of Things (IoT)

Scott Hollier et al conducted interviews with college students with disabilities.

Key findings:

• Universities need to seize the opportunities presented and develop plans to both engage with, and develop, these technologies in a learning and teaching environment.
• Universities need to ensure that these technologies are interoperable with student’s own technology, particularly AT and to address the challenges to the privacy and security for both students and staff presented by IoT technologies.
• The implementation of IoT solutions should focus on the use of personal smartphones as the primary IoT interface device for students with disabilities.
• The IoT equipment associated with learning such as a digital whiteboard should have the ability to provide its output to students via an LMS or app. This would ensure that students with disabilities can process the data with their preferred AT.
• The use of IoT to observe students and the lecturer to enhance the effectiveness of learning materials and facilitate the implementation of improvements.
• All IoT-related implementations will need to consider privacy, security and interoperability
• Any IoT solution must be accompanied by training to ensure that all staff and students are able to use it effectively.

Detailed Findings

​

Conformance Interviews

The results of hour long interviews with 6 people (lawyers, policy influencer, advocates and WCAG author) selected for their perspectives on WCAG. Key findings:

• A need for plain language in accessibility guidance.
• Preserve, but improve, testability of guidance with more flexible options including user testing. Decouple testability and conformance at level.
• The levels of WCAG (A, AA, and AAA) are useful but should be improved.
• No consensus on the scope of Silver.
• Advice to assistive technology products should be included.
• Adapt the model of LEEDS (energy-efficient building) certification program.
• Improve the usability of the accessibility guidelines themselves.
• Put user needs first.

Detailed analysis

​

​

Reimagining Accessibility Guidelines Feedback

Three presentations on Silver were held during the fall of 2017 where 4 questions were asked of the audience. Their answers provide insight into perceptions of WCAG 2.0 from technical professionals and accessibility professionals. Some common themes:

• Non-W3C sources of information on accessibility are used more than WCAG
• Silver needs an on-ramp for beginners
• Silver needs to use plain language
• Silver needs a better grading system (not a pass/fail system)
• Silver needs to expand the scope beyond the web
• People have trouble finding information in WCAG
• Need for more supplementary materials (code, design patterns, test cases, tutorials, videos, empathy-related) that are easily discovered.

Details of Audience Feedback

​

​

Conformance Survey

Among a professional audience that works with accessibility guidelines, there is an interest in changing conformance. These changes would include:

• Simpler language;
• Simpler criteria including examples; and explicit testing methods.
• More frequent updates would better support their needs.
• A method of claiming conformance that is more fluid to accommodate dynamic or regularly updated content and sites was also indicated.

Detailed analysis

Literature Review

The literature review was not completed in time for the Design Sprint.

Web Accessibility Perceptions

Student project from Worcester Polytechnic Institute - Supervised and approved by Eleanor Loiacono, PhD (Professor in Information Technology)

There were a total of 39 complete responses that included 16 graduate and 23 undergraduate students. Of these respondents 26 were male, 11 were female and 2 chose not to indicate.

• The majority of respondents were not familiar with the ADA, Section 508 or WCAG
• Most knew someone with a disability
• Most (85%) responded that they had a moderate to a great deal of exposure to web accessibility
• Half of the respondents stated they had discussed web accessibility in their courses
• Respondents seem to be overestimating their exposure to web accessibility when considering that 85% considered themselves to have had moderate or more exposure to web accessibility yet more than 65% were not familiar with any of the guidelines, standards or laws.
• Approximately 42% of those that self-identified as being familiar with the guidelines indicated that this information did not come from coursework or class discussions.

​

Mike Crabb Student papers

Prof Crabb assigned senior Computer Science majors to select a Silver research question, design a study and write a paper. He forwarded the grade “A” papers to us. There are some interesting results, although caution should be applied because the study sizes were small.

• Project Description
• W3C Accessibility Guidelines for Mobile Games by Amy Wilson
• What are the main reasons why accessibility is not seen as on the critical path for product success? by Arturas Krasendo
• How might we make accessibility guidelines easier to use by different stakeholder groups? by Duncan Pogson
• Conforming to accessibility guidelines: How can we make it more straightforward? by Graham Stead
• WHAT SHOULD BE THE SCOPE OF WEB ACCESSIBILITY GUIDELINES? by Hayley Reid
• How might we make the adoption of accessibility guidelines more straightforward (minimize disruption to process, practice and culture)? by Lisa Murray

​