Usability Testing

Alpha Phase - Round 2

Test Plan

The goal is to evaluate if the filters applied show expected benefits and if users understand the child/parent relationship.

​

1 hour total

​

HTML Prototype:

https://cdsvacpoc.azurewebsites.net/A

​

Methods/Instruments:�

• First Impression Observations

​

• Task Completion

​

• Post Test Interview with card exercise

​

Introduction

1 minute

Hi, my name is Michelle/Raluca and this is ____. We are from the Canadian Digital Service. Our goal is to make government services better and easier for people to use.

​

We are currently working with the Department of Veterans Affairs (VAC) to create an online tool.��We would love to show you a prototype on the computer and take you through a couple tasks. ��

Consent Form

2 minutes

[hand consent form to participant]

​

This paper says all the things I just mentioned, but please take your time to read it.

��Do you have any questions?

​

Would you be willing to speak with us?

​

Great! Let's get started.

​

​

​

​

​

​

This session is confidential. We will not be using your name anywhere.

​

If you are in the process of applying for benefits, this will not change or affect your status or any future applications.�

You may also stop at any time for any reason.

​

​

��

Introducing the Prototype

Before we begin, we want you to know we are testing the prototype, not you.

​

There are no wrong answers so please be as honest with us as possible.

​

The more feedback we can get, the better we can make this website for everyone!

​

Please think out loud as you look through it. We will be taking notes throughout the test. [Give example of Think Aloud].

2 minutes

What you are looking at is a prototype that is just black and white. The look and feel of it is not what the end result will be as we are primarily testing its functionality right now.

​

Do you have any questions before we begin?

First Impressions

[open up HTML prototype]��Set the context: [Participant’s personal background knowledge of VAC]…Ex. You know a bit about VAC and have already applied to one benefit. Now you’re curious what else VAC can do for you. You browse through the VAC website and came across this tool.

5 minutes

​

​

1. Without clicking on anything, what do you think you are able to do within this page?

Tasks

10 minutes

2. What kind of benefits are you interested in? [capture response]

​

• Use this tool to find those benefits [answer to question #1].

[Observe how participant goes through prototype. Ask questions according to their interactions]

​

1. [Filters Applied] Is this what you expected to find? Are there any sort of benefits you expect to find here that you didn’t?
2. [Opening Benefit Card] Is this what you expected to find? Is there any sort of information you expect to find here that you didn’t?
3. [Reading descriptions] What do you think this benefit gives you?

​

4. On a scale of 1 - 10 how confident do you feel that you found the right services to help you? 1 being very low confidence to 10 being very high confidence.

Tasks

10 minutes

5. Would you use this tool to find benefits available to your family members? [Yes/No]
6. What kind of benefits would you look for? [capture response]
7. Use this tool to find those benefits [answer to question #1].
1. [Observe how participant goes through prototype. Ask questions according to their interactions]
2. [Filters Applied] Is this what you expected to find? Are there any sort of benefits you expect to find here that you didn’t?
3. [Opening Benefit Card] Is this what you expected to find? Is there any sort of information you expect to find here that you didn’t?
4. [Reading descriptions] What do you think this benefit gives you?
8. On a scale of 1 - 10 how confident do you feel that you found the right services to help you? 1 being very low confidence to 10 being very high confidence.

Content

[allow participant to navigate the prototype as they answer the questions]

9. Find 3 words on the page that are the most unclear to you.

​

• Which words would you use instead?

​

​

​

​

5 minutes

Post-Test Interview

11. If you had a magic wand to make this tool better/easier in any way possible, what would you do?

​

• Do you have any other comments or questions?�

1 minute

Conclusion

That was all the questions we had!

Thank you so much for your help!

We appreciate your time and feedback.

​

Do you have any questions for us?

If you have any questions, feel free to reach-out anytime. You have our contact information on the consent for.

​

1 minute

Screening Questionnaire

[hand questionnaire to participant]��Here are a few standard questions we ask everyone.

​

Please take your time to read and answer everything. Let me know if you have any questions.

​

​

​

[collect questionnaire]

2 minutes