PROTEA: A Process for procuring accessible software
Dr Richard Walker
E-Learning Development Team Manager & VLE Service Group Leader
University of York
Professor Helen Petrie
Professor of Human Computer Interaction
University of York
ABOUT THE UNIVERSITY OF YORK
Ranked #7 in The Times Higher Education 100 Under 50 world ranked universities (2013)
Home to more than 15,000 students and 3,000 staff
Blackboard users since 2005
Currently on Blackboard Learn v9.1 Service Pack 12 (licensing community, content and learning management system and mobile learning building block)
ABOUT THE HCI RESEARCH GROUP AT YORK
One of the oldest research groups studying human-computer interaction in the UK, if not the world
Founded in 1984 – centred in Computer Science, but with members in Archaeology, Electronics, Health Sciences, History, Management, Psychology, Sociology and Theatre Film TV
Key values – putting humans at the centre of human-computer interaction
Particular interest in people with disabilities, older people,
e-learning
BACKGROUND – FROM THE E-LEARNING DEVELOPMENT TEAM
Have a strong commitment to accessibility and usability
Institutional staff and student VLE surveys have highlighted usability issues, for example:
BACKGROUND – FROM THE E-LEARNING DEVELOPMENT TEAM
Have worked with the HCI Group on two occasions:
BACKGROUND – FROM THE HCI RESEARCH GROUP
We have done many evaluations of accessibility and usability of websites and web-based systems, in particular:
A formal investigation of websites for the Disability Rights Commission of Great Britain
We evaluated 100 websites with 50 people with disabilities, 10 people per website
Asked them to conduct typical tasks on each websites
We found that if we evaluated with blind, partially sighted and dyslexic participants we found 80% of accessibility problems
�OUR CHALLENGE
We wanted to support the university in providing more accessible software, for both students and staff
One of the difficulties is in procuring new software – how do we know it will be accessible, most software does not come with any details of accessibility
If it does, it is usually a statement of conformance to guidelines, perhaps Web Content Accessibility Guidelines (WCAG) – is this sufficient?
OUR CHALLENGE
Two problems:
OUR SOLUTION:�PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY
So we created a protocol for user testing of software:
OUR SOLUTION:�PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY
Need to evaluate the software with 3 blind people, 3 partially sighted people and 3 people with dyslexia, as close as possible to the real users of the software being evaluated
The blind and partially sighted people should use a mix of different assistive technologies to access the software
OUR SOLUTION:�PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY
Need to pick a series of 3 - 5 “typical tasks” for the participants to do:
OUR SOLUTION:�PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY
A facilitator sits with the participant as they go through the tasks, to assist if needed
If possible, sessions should be recorded for later analysis and discussion
Participant talks through what they are doing (“think aloud” or “verbal protocol”)
OUR SOLUTION:�PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY
If the participant finds a problem, they pause and rate it on a scale 1 (= cosmetic, small annoyance) to 4 (= major, I’m really stuck)
If the participant gets really stuck (e.g. the screen reader cannot interpret the page), the facilitator helps move the participant on
BUT participant should not be “lead” through the task – not told where to click or the particular names of button, links etc
OUR SOLUTION:�PROTEA: PROTOCOL FOR TESTING E-ACCESSIBILITY
Occasionally might have some of the development team sit in
BUT only if the participant is comfortable with that and will not feel intimidated
Gather all the problems encountered, analyse for which groups of users/assistive technologies are being affected
OUR RESULTS
We applied PROTEA as part of the recent procurement process for the VLE at York
We did find some unexpected accessibility problems that we fed back to Blackboard
We also highlighted some issues where we need to educate our staff members who develop content to ensure accessibility
OUR RESULTS
Issue 1: Assistive Technology unable to give access to functionality
Problem: screen reader users could not interact fully with the Content Editor
Consequences: could not send emails, contribute to fora, answer quiz questions
OUR RESULTS
Issue 2: Navigation within software
Problem: Moving between frames was difficult and inconsistent for screen reader users
Consequences: users could not navigate to where they wanted to be, became “trapped” in frames
OUR RESULTS
Issue 3: Auditory information needs different handling to visual information
Problem: In character limited text boxes, screen readers announced the character count after every keypress
OUTCOMES AND RECOMMENDATIONS
Blackboard have been very responsive in investigating these issues
They are the kinds of problems which will not necessarily be picked up by testing to guidelines
Need really realistic evaluation with users and a range of assistive technologies
OUTCOMES AND RECOMMENDATIONS
Other useful outcomes of using this kind of evaluation:
VLE support staff become much more aware of problems that students with disabilities may face
Can create better support materials, training for students/staff who are creating content
OUTCOMES AND RECOMMENDATIONS
We have used the PROTEA method not only with Blackboard, but with several systems being procured by the university and created in-house
Both systems for students and for staff
We have also worked with the UK government (on DirectGov) and a major bank
THANK YOU!
If you would like more information, we will be setting up a website in the next weeks:
www.yorkhci.org/protea
We are happy to provide general advice for free and of course can undertake evaluations
Helen.Petrie@york.ac.uk