1 of 24

Veterans Affairs Canada

Field Interview Guidelines

March 28, 2018

1

2 of 24

2

Roadmap overview

Evolve product

Monitor metrics & user feedback

Continue user testing

Integrate backlog

Build releases

Detailing problem

USER NEEDS. ..ummm?

Stakeholder objectives

Requirements

Constraints

Define solution

Done

Solving the problem

Ideation

Prototype

Testing Prototypes

Validating solution

^*In-Progress

Iterative Development

Feature backlog

Back-end

Front-end

QA

Building the solution

^*In-Progress

Iterative

Implement metrics

Communications

Training

Support

Discovery

Design

Develop

Deploy

Evolve

^✔

2

3 of 24

3

Problem Statement

How can we enable a simple way for veterans to discover VAC’s range of benefits and suitably determine which benefits are relevant to them?

4 of 24

4

Wide Range of User Groups

Image source: VAC Website

4

5 of 24

Why conduct user research?

5

“We spend a lot of time designing the bridge, but not enough time

thinking about the people crossing it...”

Dr. Prabjot Singh

Image source: https://tgam.ca/2HbCTws

6 of 24

People are experts In their own experiences

...we try walking in their shoes

6

Image source: http://bit.ly/1pwUhCT

This is accomplished by techniques such as user interviews, observation studies of users performing tasks, shadowing users in their natural environment, conducting design exercises with users to elicit their responses and more.

This methodology has been used for over three decades in the private sector in disruptive innovation, technology sectors and management.

But the problems that government products and services solve are often determined by the organizational needs, without seeing the problem through the eyes and the voice of the users. Therefore, almost all government products and services, require users to continuously adapt their attitudes and behaviors to [the organizational needs] in order to use them. They are not intuitive to people!

This is why, for the last decade - user centered design has also been used in public sectors around the world - including federal and provincial government teams in Canada.

Because, the result are products and services that offers a more efficient and satisfying user experience, which is likely to increase adoption and trust in government services. Building adoption and trust is vital.

7 of 24

Who are our users?

Young, modern day Veterans

‘The early majority’

7

Image source: http://bit.ly/2oW8pHd

These criteria, all evidence and iteration gathered till date point towards a group that we broadly identify as “Young, Modern Day Veterans” or in research terms the ‘Early Majority’ group of users (Rogers, 1995).

Based on the Sociologist, Rogers’s innovation adoption curve, if one-end of VAC users are extreme adopters of new product innovations (such as proactive, independent veterans) and the other are absolutely slow adopters such as very vulnerable veterans - The Early Majority - are not on either extreme but characterized as ‘pragmatists’ - those who are willing to try new things out but only after they are comfortable that it won’t put them at risk.

We have determined that the user segmentation of the Early Majority or Young, Modern Day Veterans are also the largest segment of veteran groups.

8 of 24

Research phase I: target user profile

Small sample size of 8-10 individuals.
Honourably discharged veterans and still serving military members, currently undergoing transition to be released (soon to-be veterans).
Have served or still serving ‘Regular Force’ from Canadian Armed Forces.
Younger, recently released.
Owns a smartphone or a personal computer with internet access.
Not selecting vulnerable or high risk participants.

8

But order to know whom to select for our research it is not enough to know general characteristics about this user group - we can’t solve specific problems with generalizations.

So we set out by well defining the profile of the “core or target user”, as follows:

A small sample size of 6-8 individuals.
Honourably discharged veterans and still serving military members, currently undergoing transition to be released (soon to-be veterans).
Served or still servin ‘Regular Force’ from Canadian Armed Forces.
Younger, recently released.
Owns a smartphone or a personal computer with internet access.
Not selecting vulnerable or high risk participants.

Details:

On the criteria of Discharge: Honourably discharged veterans and still serving military professionals who are currently undergoing their transition to be released. Still serving members but only those undergoing transition as they are soon-to-be veterans.

On the criteria of Release: We are looking only at non-medically released individuals (voluntary and service completed release included). This is because we believe the medically released individuals would have existing medical files with the department.

On the criteria of Type of Duty: We are looking at still serving or served veterans from Regular Force of the Canadian Armed Forces. We are focusing on CAF because it allows us to focus on VAC’s largest client department and therefore reaches the maximum number of VAC users.

On the criteria of Age Range: We are looking at individuals between the age of 25 - 55 years. Broadly as younger veterans.

On the criteria of Time Since Release: We are looking at individuals who are recently released (0-10 years or released since 2008).

On the criteria of type of Needs: We are not selecting participants based on any type of need - but we are particular about not selecting vulnerable or high risk participants - as they fall under extremes and are not the majority.

On the criteria of personal support networks: Again, we are looking at the average so individuals with at least some personal support networks - not vulnerable participants with no personal support networks whatsoever.

And finally on the criteria of technology usage: We are looking at individuals who owns a smartphone or a personal computer with internet access. This is a basic need for our average user profile.

9 of 24

How do we choose methods

and approaches?

It depends on the problem

we’re trying to solve.

9

10 of 24

10

11 of 24

Semi-structured Interviews

11

Semi-structured interviews are used during the needs assessment phase of product and program design.

Image source: http://bit.ly/2I7lhD3

Now that we know who our users are - we determined the best method to conduct our research as “semi-structured interviews”

Semi-structured interviews are used to gather focused, qualitative textual data to clarifying the research domain or specific research question. In our case, our question is: What are the needs and goals that young veterans want to fulfill in their lives and what meaning do young veterans assign to the government?

Semi-structured interviews are often used during the needs assessment phase of product design. This method can uncover rich descriptive data on the experiences of participants.

Information gathered during semi-structured interviews can move the innovation process from general topics (domains) to more specific insights (factors and variables).

It can be used to develop a preliminary hypothesis, explain relationships and create a foundation for product development and further testing of research.

So, what?

The best way to understand the hopes, desires and aspirations of young veterans that we’re designing for, than by talking with them directly.

Approaching them in a space that’s comfortable to them - is key to unlocking all kinds of insights and understanding that sitting in our offices will not give us.

This process will allow us...

To use quotes, stories and pictures as evidence of real veteran experience to summarize our findings.
To have deep understanding by describing young veterans context in detail.
To look for patterns and themes to show how the transition period from military to civilian life occurs for young veterans.

----

Semi-structured interviews sit halfway between a structured survey and an unstructured conversation.

12 of 24

Semi-structured interviews

Guidelines for

interview questions

Style: Open-ended questions rather than ‘yes’ ‘no’ questions.
Biases: Avoid leading questions.
Language: Use terms that participants can understand given their background.
Context: Be mindful of the social or cultural contexts of your questions.
Concise: Keep the questions as short and specific as possible. Avoid asking two-in-one questions, such as, “Do you travel by car and by bike?”
Frame: Avoid questions with a strong positive or negative association.

Semi-structured interviews offer

topics and questions to the interviewee but are carefully designed to elicit the interviewee’s ideas and opinions on the topic of interest, as opposed to leading the interviewee toward preconceived choices.

12

Source: http://designresearchtechniques.com/casestudies/semi-structured-interviews/

13 of 24

During the interview process

Know your role (i.e. interviewer, note-taker).
Introduce yourself, explain the research and get the participant’s consent.
Ask warm-up questions first then get into the interview guide.
Make sure to write down exactly what the person says, not what you think they might mean. This process is all about hearing exactly what people are saying.
Try not to interrupt participants; make a note and come back to the idea later.
Use probing questions to gather as much information as possible.
Avoid repeating questions already answered
Keep the conversation focused on the main domains, avoiding tangents.
Ask the participant if there is anything else they’d like to share.

13

14 of 24

Guidance on Public Opinion Research

Researchers are not to discuss with participants any of the following topics:

Political party preferences
Voting (preferences or habits).
Thoughts on political figures, such as the Prime Minister, the leader of the opposition, members of parliament, or other government officials.

If the conversation moves to discuss a specific political party or member of parliament, researchers are to change the topic as quickly as possible and shall not record any information from the interview participant.

14

15 of 24

Observation guidelines

What the person says is only one data point.

Be sure to observe your interviewees body language and the context in which you’re talking.

POEMS technique

People – The demographics, roles, behavioral traits, and quantity of people in the environment
Objects – The items the people are interacting with, including furniture, devices, machines, appliances, tools, etc.
Environments – Observations about the architecture, lighting, furniture, temperature, atmosphere, etc.
Messages – The tone of the language or commonly used phrases in taglines, social/professional interactions, and/or environmental messages
Services – All services, apps, tools, and frameworks used

15

16 of 24

Maintain a mindset of...

Learning
Engaging
Openness

16

17 of 24

Learning

Watching what people do and how they interact with their environment gives you clues about what they

think and feel. It helps you to learn about what they need.

By watching people you can capture physical

manifestations of their experiences, what they do and say. This will allow you to interpret intangible meaning

of those experiences in order to uncover insights. These insights will lead you to the innovative solutions.

The best solutions come out of the best insights into human behavior. But learning to recognize those

insights is harder than you might think.

Why? Because our minds automatically filter out a lot of information in ways we aren’t even aware of. We need to learn to see things “with a fresh set of eyes” – tools for empathy, along with a human-centered mindset, is what gives us those new eyes.

17

18 of 24

Engaging

Engaging with people directly reveals a tremendous amount about the way they think and the values

they hold. Sometimes these thoughts and values are not obvious to the people who hold them.

The stories that people tell and the things that people say they do—even if they are different from what they actually

do—are strong indicators of their deeply held beliefs about the way the world is.

Engage to

Uncover needs that people have which they may or may not be aware of
Guide innovation efforts
Identify the right users to design for
Discover the emotions that guide behaviors

18

19 of 24

Shed judgement

Don’t judge. Just observe and engage users without the influence of value judgments upon their actions,

circumstances, decisions, or “issues.”

• Question everything. Question even (and especially) the things you think you already understand. Ask questions to learn about how the user perceives the world. Think about how a 4-year-old asks “Why?” about everything. Follow up an answer to one “why” with a second “why.”

• Be truly curious. Strive to assume a posture of wonder and curiosity, especially in circumstances that seem either familiar or uncomfortable.

• Find patterns. Look for interesting threads and themes that emerge across interactions with users.

• Listen. Really. Lose your agenda and let the scene soak into your psyche. Absorb what users say to

you, and how they say it, without thinking about the next thing you’re going to say.

19

20 of 24

And some more tips (never too many)

Always say yes to an offer
Treat people like partners in research
Be comfortable with silence
Try very intentionally to fall in love with each person (even if it’s just a little bit)
Wear generic clothing

20

21 of 24

Trust the process even

if it feels uncomfortable!

21

22 of 24

Ethical Considerations

Conducted under TBS Authority

22

Informed Consent

Privacy by Design

Only Collect Necessary Information

Anonymized Responses

23 of 24

Recruitment

Completely Voluntary

No undue influence

Collected under TBS authority

Personal Information Bank: Outreach (938)

23

1

2

3

24 of 24

Interviews

Informed Consent

1-2 days in advance of interview
PSU: 942 or 938

Consistent with Tri-council & TBS policy�

Anonymized Participation

Behavioural & Factual Questions only

24

1

2

3

4