1 of 33

Shared research experiences

RAPID REPORTING AND SHOWCASING OF USABILITY TESTING

​

NEIL ALLISON @USABILITYED

1

2 of 33

Hi I’m Neil…

HEAD OF PROSPECTIVE STUDENT WEB CONTENT �AT THE UNIVERSITY OF EDINBURGH

NEIL ALLISON @USABILITYED

2

3 of 33

Hands up…

​

  • If you’ve ever conducted a usability test

​

  • If you’ve ever watched a usability test

​

  • If all this is totally new to you

NEIL ALLISON @USABILITYED

3

4 of 33

Usability testing

IT’S EASY, RIGHT?

NEIL ALLISON @USABILITYED

4

5 of 33

What can be challenging

​

  • Getting the go ahead to use your time on usability testing

​

  • Getting colleagues to take on board what you uncover

​

  • Getting fixes to problems implemented

NEIL ALLISON @USABILITYED

5

6 of 33

These are common challenges…

NEIL ALLISON @USABILITYED

6

7 of 33

Standing on the shoulders of…

​

Caroline Jarrett and Steve Krug surveyed� the UX community in 2012.

​

They blogged about why usability �problems go unfixed: �http://bit.ly/krug-jarrett-unfixed

​

NEIL ALLISON @USABILITYED

7

8 of 33

A story about change

MY CHALLENGE BACK IN 2013…

NEIL ALLISON @USABILITYED

8

9 of 33

Moving out of our comfort zone…

  • I was the University’s CMS Service Manager, responsible for user support and training, community engagement, user research and requirements validation, co-design facilitation…

​

  • We had a vendor-supplied CMS that everyone hated, with a service around it that people quite liked.

​

  • Features were introduced in waterfall projects. We had a small group delivering these work packages that was stable, working in a user-centred fashion. And we trusted each other.

​

  • Then in 2012 it was announced we were switching CMS, going open source and developing our own system…

NEIL ALLISON @USABILITYED

9

10 of 33

Changing times, new challenges

  • New people, new methodology, new urgency…

​

  • A development team of 3 plus a business analyst, working agile �to deliver a new CMS and transition the service in 2 years.

​

  • We didn’t have a culture of user-centred design.
  • The project team needed to be won over.
  • The CMS user community wanted reassurance.

​

  • Old ways of working were not going to fit this new situation…

NEIL ALLISON @USABILITYED

10

11 of 33

Steve Krug to the rescue

​

  • Free to use observation session materials

​

  • Everyone listing top 3 issues for �each test participant�
  • Bringing the development team closer �to the CMS user group that the �service team knew so well

NEIL ALLISON @USABILITYED

11

12 of 33

Good, but not great (sorry Steve)

  • Testing every 4 weeks.

​

  • Live observations, the CMS user group keen to get involved.

​

  • Team (broadly) happy to dedicate time.

​

  • But after the tests, discussion meandered. �People hung on to pet issues. Tech solution chat dominated.�
  • We would run out of time, no closure.

​

NEIL ALLISON @USABILITYED

12

13 of 33

Plus we wanted to go public

​

  • We wanted to bring in the user community as contributors, not just as test participants

​

  • The post-test conversations were meandering with 6-8 people.

​

  • What would happen with 40?

NEIL ALLISON @USABILITYED

13

14 of 33

Enter David Travis

“RED ROUTE” USABILITY TO THE RESCUE

NEIL ALLISON @USABILITYED

14

https://bit.ly/red-route

15 of 33

An objective prioritisation flowchart

​

  • Three questions to grade every usability issue

​

  • Post observation prioritisation complete�in 30-40 minutes

​

  • Scalable: works the same with 30+ people

​

NEIL ALLISON @USABILITYED

15

16 of 33

The process

HOW TO DO IT YOURSELF

NEIL ALLISON @USABILITYED

16

17 of 33

Step-by-step

​

  1. Get the right people in a room
  2. Watch a small number of short sessions with users doing something
  3. Prioritise the issues they see
  4. Collaboratively consolidate their priority lists
  5. Agree actions for usability issues
  6. Repeat every few weeks

​

NEIL ALLISON @USABILITYED

17

18 of 33

Who are the right people?

  • Everyone with a stake in the service

​

  • If you’re influencing design decisions or priorities �you should be there

​

​

​

NEIL ALLISON @USABILITYED

18

“Have you had your recommended dose of research?”

http://bit.ly/gds-research-dose

​

19 of 33

What do we watch?

  • Real users doing real tasks

​

  • Facilitated usability testing sessions�
  • Agree the focus of testing within the team

​

NEIL ALLISON @USABILITYED

19

“Research shows that teams make better services when everyone on a project team observes users first hand.”

Jared Spool

​

http://bit.ly/spool-exposure

20 of 33

How many do we watch?

�As many as you can fit into the time you have

Example:

    • Intro – 10 min
    • Test session 1 + 5 minutes
    • Test session 2 + 5 minutes
    • Test session 3 + 5 minutes
    • Issue prioritisation – 40 minutes

“The most striking truth of the curve is that �zero users give zero insights.”

Jakob Nielsen

​

​

NEIL ALLISON @USABILITYED

20

http://bit.ly/nielsen-5-users

MYTH?

21 of 33

How do we prioritise?

Rocket Surgery template:

    • Individual notes while observing
    • Distil to 3 issues after each participant

​

​

​

“Running a usability test has been compared �with taking a drink from a fire hydrant…”�David Travis

​

​

​

​

​

NEIL ALLISON @USABILITYED

21

http://bit.ly/usability-prioritise-travis

22 of 33

How do we consolidate?

“If you prioritise usability problems using �'gut feel' or intuition, you run the risk �of being exposed as a fraud…”�David Travis

NEIL ALLISON @USABILITYED

22

http://bit.ly/usability-prioritise-travis

23 of 33

xxx

NEIL ALLISON @USABILITYED

23

24 of 33

Taking action on the insight

​

  • This matrix is a great way to separate issues from ‘solutions’

​

  • You might put the same issue on the matrix more than once, if multiple solutions exist

​

  • Agree the fastest, cheapest way to validate your solution

NEIL ALLISON @USABILITYED

24

Minor issue for users

Major issue for users

Easy solution available

No easy solution available

25 of 33

Top tips

  • Testing focus
    • Agree priority tasks together
    • Trial test with the team
    • No surprises!
  • Participants
    • Recordings are far less stressful than live review sessions
    • Krug – “Recruit loosely, grade on a curve”
  • Do whatever it takes to get observers in the room
    • Start over lunch break
    • Supply refreshments
    • Bribery, favours, threats…

​

​

  • Be well organised so you don’t waste the observer group’s time
    • Test everything before hand
    • Provide a schedule in advance

​

  • Stick to the process
    • It’s easy to digress when you’ve all seen so much

​

  • Encourage collective reflection on the session
    • Admitting usefulness is first step to getting observers to turn up next time
    • Have the next one scheduled ASAP

​

NEIL ALLISON @USABILITYED

25

26 of 33

Everything you need

NEIL ALLISON @USABILITYED

26

27 of 33

What happened next

A TOOL FOR CULTURE CHANGE & ADOPTION

NEIL ALLISON @USABILITYED

27

28 of 33

Usability testing showcases

​

  • I approached a number of business-critical services and offered usability testing for free on the condition that we reviewed as a collaborative event.

​

  • This made the case for a pilot usability testing service which then grew into a pilot UX consultancy service.

​

  • These ‘freebies’ have developed into longer-term funded initiatives.

NEIL ALLISON @USABILITYED

28

29 of 33

Learn VLE service enhancements

​

  • Collaborating with schools on a rolling basis�to expose teaching staff and learning�technologists to student experiences.�
  • Making the case for greater consistency and �developing guidelines in the open.

​

  • Exposing the institution to the range of �approaches in existence, and the impact �on the student.

​

  • http://bit.ly/VLE-UX

NEIL ALLISON @USABILITYED

29

30 of 33

IT self help iterative improvements

​

  • Bringing frontline and second line support teams together around the highest volume support call themes.�
  • Creating a cycle of testing,�analytics and content development.

​

  • http://bit.ly/IT-self-service �

​

NEIL ALLISON @USABILITYED

30

31 of 33

Showcasing the future of degree profiles

​

  • Keeping the student recruitment and marketing community engaged and informed as we explore, design and deliver a replacement for our current online degree finders.
  • Stakeholder exposure to end users interacting with our evolving prototype fosters trust and engagement.
  • https://bit.ly/future-state-degree-programmes �

​

NEIL ALLISON @USABILITYED

31

32 of 33

In conclusion

  • Involve the team at every stage if you want usability testing to have impact
    • From setting a focus through to review and prioritisation�
  • Work to establish a virtuous cycle that others can adopt
    • Focus on inclusion and empowerment
    • Little of this is “specialist” – it’s everyone’s responsibility
    • This only works if you keep it going

​

  • Few read your reports and fewer believe what you write
    • Give them real exposure to users
    • Channel the group to write their own “report”

​

  • Go as public as you can get away with
    • The greater the exposure, the bigger the organisational ripples

NEIL ALLISON @USABILITYED

32

33 of 33

Thank you

QUESTIONS?

​

NEIL ALLISON�HEAD OF PROSPECTIVE STUDENT WEB CONTENT�THE UNIVERSITY OF EDINBURGH�WWW.ED.AC.UK/PROFILE/NEIL-ALLISON

​

NEIL ALLISON @USABILITYED

33