Building WCAG 3 at TPAC

1

Background

2

TPAC Approach to Developing WCAG 3 (Bottom Up)

• Tests (TPAC Thursday)
• Issue Severity (TPAC Thursday)
• Conformance approach (TPAC Thursday/Friday)
• Levels (TPAC Friday?)
• We will discuss levels as they relate to solving a problem but are not likely to get to levels as a detailed conversation during TPAC
• Normative vs. Informative (After TPAC)
• Writing content (After TPAC)
• Organizing content (After TPAC)

3

Possible Conformance Approaches

• 100% passes or it fails
• Example: WCAG 2
• Current state: Tests pass/fail but we want to explore wider options for conformance
• Percentages (FWPD feedback indicates this is not a realistic option)
• Examples: WCAG 3 FPWD (partial) and Benchmark
• Current state: Issues and member feedback indicate the level of effort for this is too high
• Points
• Example: Scoring Tests Based on Functional Categories
• Current state: Possible area for exploration
• Adjectival scoring
• Example: Suggestions for a WCAG 3.0 Conformance model
• Current state: Possible area for exploration
• Evaluating severity in context -- minor errors may not fail, but critical errors do.
• Examples: WCAG 3 FPWD (partial)
• Current state: Possible area for exploration

4

Additional Factors in Conformance

• Prioritizing by functional needs
• Examples: WCAG 3 FPWD (partial)
• Current state: Possible area for exploration
• Minimum scores
• Examples: WCAG 3 FPWD (partial)
• Current state: Possible area for exploration
• Weighting
• Example: Weighting prototype
• Current state: Tested in Silver, agreed not viable in several group discussions
• Protocols
• Examples: Protocols and Assertions and Involving PwDs in digital authoring process

5

Issues WCAG 3 is Trying to Solve (Based on Research)

Usability

• Too difficult to read and translate
• Difficult to get started
• Ambiguity in interpreting the success criteria
• Too difficult to persuade others to implement

Conformance Model

• Strictly Testable Constraints
• Human Testable
• Accessibility Supported
• Evolving Technology

Maintenance

• Flexibility
• Scaling
• Governance

6

Opportunities section of Requirements doc are based on the Problem Statements from the WCAG 3.0 research of 2017-2018

Architecture

7

Structure for the WCAG 3.0

• Guidelines: high-level, plain-language version of the content for managers, policy makers, individuals who are new to accessibility
• How-To: describe the guideline, technology neutral
• Outcomes: testable criteria that include information on how to score the outcome in an optional Conformance Claim
• Outcomes are technology neutral
• Methods: detailed information on how to meet the outcome, code samples, working examples, resources, as well as information about testing and scoring the method
• Methods are technology specific

​

​

8

Example: Captions (Exploratory)

Guideline: Provide captions and associated metadata for audio content.

• Outcome: Translates speech and non-speech audio into alternative formats (e.g. captions) so media can be understood when sound is unavailable or limited. User agents and APIs support the display and control of captions.
• Method: Provides text equivalents of speech and key sound effect (XR)
• Method: Reflow of captions and other text in context (UAAG)
• Outcome: Conveys information about the sound in addition to the text of the sound (for example, sound source, duration, and direction) so users know the necessary information about the context of the sound in relation to the environment it is situated in.
• Method: Provides visual equivalent of speech and non-speech audio metadata

9

Method Example

• Introduction
• Background
• Examples
• Tests
• Glossary

10

How-to Example

• Get started
• Plan
• Design
• Develop
• Examples
• Resources

11

Outstanding Questions on Architecture

• The informative documentation needs to be moved and redesigned
• For example: The methods and how to’s look similar and cause confusion
• Form a group to improve CSS - Can member companies volunteer designer and UX time?
• How guidelines will be organized
• Tagging will be used to provide customizable organization
• The default organization will be agreed upon after more of the guidelines are written

12

Integrating Test Types & Severity

13

Issue Severity

Test Based:

• AG evaluate each test and identify ones which are (usually) critical issues.

Task Based:

• Post testing
• Process which evaluates issue severity based on task
• Process that uses existence within type of task to determine severity

Key question:

• What do we do with critical (and non-critical) issues?
• E.g: Require zero critical issues at Bronze but allow all (some?) non-critical issues? At Silver level require no issues.
• How do people using their own method to meet the normative text establish whether it is critical/non-critical?

14

Types of tests (Refine through survey)

15

Two types of test:

• Computational: Tests where results will not vary based on the tester or approach. Examples include testing whether certain properties exist in the content or if they match a value specified by the requirement.
• Qualitative: Tests that rely on a qualitative evaluation based on existing criteria. Test results may vary between testers who understand the criteria. Examples include evaluating the quality and applicability of certain properties of the content.

Four types of requirements:

• Prescriptive requirements: How to conform, and how to test for conformance is predetermined. If there is more than one option, all options are defined by the standard.
• Adaptive requirements: The testing process includes variations where the results rely on the external and/or user-specific context in which content is being tested. The tests are still computational or qualitative tests but the context dictates what results are expected, and which tests are relevant.
• Extensible requirements: Requirements that rely on selecting from multiple valid ways to measure the outcome (for example, there are multiple accepted specifications for measuring luminance, and the testers will need to determine which is most valid for their use case).
• Protocol-based requirements: Using accepted industry protocols that meet defined quality criteria (examples include Usability and Plain Language protocols) to improve accessibility.

Comparing Test Types

16

Exploratory: Test Types and Issue Severity

17

Critical error in test:

• Documented in WCAG 3 Method, based on test e.g., functional images without alt text

Critical error by task:

• New process based on task evaluation or dictated by type of task (e.g. Operation)

Example methods for Images have text alternatives (HTML)

18

Critical error in test:

• Functional image has non-empty accessible name

Critical error by task:

• If the image alt text is needed for operation than any failed test becomes critical

Example methods for text contrast (HTML)

19

Critical error in test:

• Text is below bare minimum threshold
• Text color cannot be adapted

Critical error by task:

• If the text is needed for operation than any failed test becomes critical

Ex: Provide captions and associated metadata for audio content.

20

Critical error in test:

• ​

Critical error by task:

• ​

Normative vs. Informative (Non-Normative)

21

Functional Needs, Guidelines, Outcomes

• Functional Needs - Statements that describe a specific gap in one’s ability, or a specific mismatch between ability and the designed environment or context.
• Guidelines - High-level, plain-language versions of content
• Guidelines support one or more Functional Needs
• Outcomes - written as testable criteria
• Outcomes support one or more Guidelines (ideally many to 1, but likely not possible)
• Outcomes have an “And” between them (must meet all the outcomes)

22

Methods & Test Sets

• Methods - technology specific ways to meet Outcomes (ideally many to 1)
• Each method is sufficient
• Methods include scope tested (component, view, user processes, aggregate)
• Methods have an “Or” between them (choose the appropriate method)
• Because different technologies are supported
• Different scope
• Test Sets - Unique combination of tests that is sufficient to meet the Outcome
• Test sets may include different types of tests
• Methods should be broken apart when:
• They have a different test set
• Anytime an “Or” occurs
• Different technologies
• Different test scope

23

Allocation of WCAG 3 Parts

Normative

• Outcomes
• Glossary Items
• Higher Level Functional Categories?
• Critical Error Definition or Criteria?

Informative

• Guidelines?
• Principles?
• Functional Needs
• How Tos
• Methods
• Title
• Test Set
• Description
• Example
• Critical error details?

24

• Should User Needs be Normative or Informative?

Outcome or User Need Statements?

25

Alternative to Outcomes in Normative

• One possible solution is to shift outcomes to the related user needs statement
• Note that these user needs would be more detailed than those captured in the Functional Needs Document

26

Error Prevention Example - context

Errors subgroup followed the WCAG3 Writing Process and enhanced the user needs analysis section, which made it ideal for this example. The WCAG3 writing process is:

1. Define Flows and User Need
2. Write Outcomes
3. Develop Tests
4. Write Methods
5. Write Guideline How-To

27

Errors Prevention Example: FAST - Main Functional Needs

Main Functional Categories

Essential

Sensory - Vision & Visual

Sensory - Sensory Intersections

Cognitive - Attention

Cognitive - Language & Literacy

Cognitive - Learning

Cognitive - Memory

Cognitive - Executive

Cognitive - Mental Health

Cognitive - Cognitive & Sensory Intersections

Independence

Main User Needs

Provide help and instructions

​

​

28

Example of Outcomes vs. User Needs – Errors

Guideline: Provide features that help users avoid errors.

User needs instructions that display at the source of input so they can access the instructions while focused on the input.

• Tests: G162, Easy Checks: Required fields and other instructions

User needs instructions that persist so they know how to enter data for as long as it takes to enter the data.

• Test: G184, Easy checks: Required fields and other instructions, WAT: Form Instructions

User needs examples for inputs that have formatting requirements so they can follow the example to provide the data in the correct format.

• Test: G89, Easy Checks: Required fields and other instructions

​

​

29

Working Definitions & Category Exercise

30

Accessibility

• Accessibility - Whether people with disabilities can use content (User Needs?)
• Conformance - Satisfying all the requirements of the guidelines. Conformance is an important part of following the guidelines even when not making a formal Conformance Claim (WCAG 3). Note: Compliance is usually mandatory
• Conformance is the fulfillment of specified requirements by a product, process, or service. (QA Framework)
• Conformance model (noun) - Rules that we write to define how to meet the guidelines and record it. Ex. We are creating conformance guidance.
• Conformance (noun) - A state of meeting the guidelines (rules). Ex. I am reporting my conformance.
• Is conformance binary, partial, a continuum, or at a level?
• To conform (verb) – the act of bringing a website that meets the rules including the testing and reporting. Ex. We want our website to conform.

31

Testing

• Test - A way to evaluate the implementation of a method.
• Test Set - Unique combination of tests that is sufficient to meet the Outcome

32

Regulatory Environment

Compliance – Meeting the requirements put in place by policies or regulations. Note: Compliance is usually mandatory

​

• Sensitivity and internationalization issues exist around the definitions of Policy & Regulation
• The following definitions are working definitions to help us better understand each other in conversations
• Policy - A “policy” is any standard, statement, or procedure of general applicability adopted by an organization or entity.
• Public policy - is a government policy made on behalf of the "public." It is oriented toward a goal or desired state, such as the solution of a problem. Policy often precedes law and the ideas from policy inform the formation of laws. A policy is unenforceable but often forms the reasoning behind the creation of a law.
• Organizational policy - policies help guide operations to communicate an organization’s culture, values and philosophy. They cover what employees and customers can expect from the organization. Policies set clear expectations to create consistency, reduce risk, and set high standards for employees. Policy allows leadership to ensure the company communicates all laws and regulations to employees. Plus, it ensures employees both understand the policies and know how these policies apply to their day-to-day jobs..
• Regulation
• A regulation is a written rule that specifies how the ideas of a law are to be implemented and can specify standards, compliance, as well as what and to whom the regulation is scoped to apply.

33

Vision 1a: WCAG maximizes guidance to address accessibility coverage.

34

Vision 2a: WCAG limits guidance to current policy expectations

35

Desired end state (?): Regulators implement an expanded set of WCAG requirements

36

Less WCAG Centered View

37

Digital Accessibility

Policy/�Regulation

Other Standards (e.g. UAAG, ATAG),

WCAG2ICT,

Maturity Model

Content Usable

etc.

WCAG

User needs

To be informed, enforcement?

What should falls in each category in WCAG 3?

• Conformance options began a similar exercise
• Substantial Conformance/Example Scenarios
• https://www.w3.org/WAI/GL/task-forces/silver/wiki/Substantial_Conformance/Example_Scenarios
• Used different divisions
• Technical standards: Accessibility and Conformance
• Guidance: Informative documentation
• Policy: Regulators
• Brainstorming document

38

Evaluating conformance proposals

39

WCAG 3 Requirements Document: Design Principles

Accessibility guidelines:

1. Wider range of people with disabilities
2. New measurement and conformance structure
3. Include emerging technologies.
4. Follow accessibility guidance�“Eat our own dogfood”
5. Be written in plain language

Guidelines process:

1. Include people with disabilities
2. Facilitate global participation and feedback
3. Data-informed and evidence-based

​

40

WCAG 3 Requirements Document: Requirements

​

​

• Multiple ways to measure
• Flexible structure
• Multiple ways to display
• Technology neutral
• Attention to readability and usability
• Suitable for regulatory environment
• Motivation to do more than minimum
• Scoped for a diverse group of stakeholders

41

Considerations for WCAG 3 Conformance

Situation 1: Bugs and other issues of oversight occur in content

Situation 2: When large volumes of content are accumulating too rapidly to make fully conforming

Situation 3: When making large volumes of content fully conform is not achievable immediately

Situation 4: Content provider does not own or directly control the content

Situation 5: When content providers have dependencies on other services

Situation 6: Current limitations in providing the same level of conformance for live content

42

Considerations for WCAG 3 Conformance - continued

Situation 7: Current limitations in making some types of content fully conform

Situation 8: When content is rarely used, if ever

Situation 9: Content is experimental for all users, including people with disabilities

Situation 10: Not all accessibility requirements are always applicable to all content

Situation 11: Small businesses face unique challenges

43

Additional Criteria to Evaluate Success

• Does the proposal support internationalization?
• Does the proposal support evolving technology?
• As coverage increases or when coverage is universal within a closed ecosystem, can the user agent and AT support can be claimed as conforming even if the author did nothing?
• Does the approach provide equity to all disability categories?
• Does the proposal significantly increase the testing time?
• Does the proposal increase the testability?

44

Decide which Conformance Option Sets Solve the Issues

45

Criteria for Evaluating Results

• Criteria from Research Report of Web Accessibility Metrics - 2012 Symposium and workshop on accessibility testing metrics.
• Metrics and Plan for Evaluating Conformance Scoring for WCAG 3
• Validity - how well the measurement reflects the actual accessibility
• Reliability - “interrater reliability”, reproducibility, and consistency of scores
• Sensitivity - how changes to the site are reflected in metrics (too much or too little)
• Adequacy - the type of data being collected is appropriate. For example, that scores are not influenced by rounding.
• Complexity - the resources required to do the conformance testing
• Conformance Architecture Testing subgroup wiki - includes links to the testing web sites.

46

Possible Conformance Approaches

• 100% passes or it fails
• Example: WCAG 2
• Current state: Tests pass/fail but we want to explore wider options for conformance
• Percentages (FWPD feedback indicates this is not a realistic option)
• Examples: WCAG 3 FPWD (partial) and Benchmark
• Current state: Issues and member feedback indicate the level of effort for this is too high
• Points
• Example: Scoring Tests Based on Functional Categories
• Current state: Possible area for exploration
• Adjectival scoring
• Example: Suggestions for a WCAG 3.0 Conformance model
• Current state: Possible area for exploration
• Evaluating severity in context -- minor errors may not fail, but critical errors do.
• Examples: WCAG 3 FPWD (partial)
• Current state: Possible area for exploration

47

Additional Factors in Conformance

• Prioritizing by functional needs
• Examples: WCAG 3 FPWD (partial)
• Current state: Possible area for exploration
• Minimum scores
• Examples: WCAG 3 FPWD (partial)
• Current state: Possible area for exploration
• Weighting
• Example: Weighting prototype
• Current state: Tested in Silver, agreed not viable in several group discussions
• Protocols
• Examples: Protocols and Assertions and Involving PwDs in digital authoring process

48

Action Items & Next Steps

1. Action Items: See Parking Lot
2. Summarize TPAC
3. Retrospective on subgroup process
4. Start new subgroups
5. Continue conformance options discussion
6. Publish a working draft with revised architecture, tests, and issue severity
7. Bring regulators to a meeting to give feedback on working draft

49