1 of 26

UX Sampler Sessions

Interview & Survey Methods, part 2

Spring 2025

User Experience Standing Group (UXSG)

Orbis Cascade Alliance Diversity & User Experience (DUX)

This series intended to offer a sampler or a menu of user experience (UX) research methods. Intended for a broad audience, showcasing a variety of methods that can be used in different areas of library work to answer different types of user experience questions.

Observational UX Research Methods. Usability testing and physical space studies – Tuesday, April 8, 1:00pm-2:00pm
Interview & Survey UX Research Methods, part 1. Photo studies and love/break-up letters – Tuesday, April 22, 1:00pm-2:00pm
Interview & Survey UX Research Methods, part 2. Surveys and focus groups – Tuesday, May 6, 1:00pm-2:00pm
Participatory UX Research Methods. Journey mapping and cognitive mapping – Tuesday, May 20, 1:00pm-2:00pm
Data Town Hall Conversation. Managing and analyzing data once you’ve gathered it – Monday, June 2, 1:00pm-2:00pm

2 of 26

Introductions

Christy Scott (she/her)

Walla Walla University

Amanda Pirog (she/her)

University of Washington Tacoma

3 of 26

Agenda

UX research cycle & methods
Methods: Surveys & Focus Groups

Overview, tips, and best practices
Case study
Activity

Discussion
Resources for further learning
Survey for future trainings

4 of 26

UX Research Cycle

Adapted from: Priestner, A. (2021). A handbook of user experience research & design in libraries. UX in Libraries.

The primary goal of UX work is to center the experience of the user as we assess services, digital and physical environments, and more.

This double diamond model comes from Andy Priestner in his book A Handbook of User Experience Research & Design in Libraries. The diamonds are composed by divergent and convergent phases.
The divergent phases or states are usually visually represented by open-ended arrows to represent staying open to new possibilities and opportunities. In divergent stages rather than relying on what you know or think you know, you are allowing new input and thinking expansively about the problem before jumping to solutions.
In convergent phases, visually represented by close-ended arrows, you are considering many ideas from a systematic and logical standpoint, and then moving towards a single solution or approach to pursue further.
The four phases, Discover, Define, Develop, and Deliver, suggest four phases from start to finish, and while most of us do not have time or resources to pursue all four stages, we share this graph to contextualize the UX methods we are sharing, and note that a UX method can produce meaningful data without being part of ‘complete’ research cycle.

5 of 26

UX Research Methods Landscape

Adapted from: Priestner, A. (2021). A handbook of user experience research & design in libraries. UX in Libraries.

There are many UX methods to choose from, and there is no “one best method” to rely on. Each method offers unique insights and is suited to particular contexts and needs. On this slide, you’ll see a landscape of various UX research methods that are plotted along two axes that represent four types of data.
On the vertical axis, we see a spectrum between behavioral and attitudinal techniques. Behavioral techniques tell us what users actually do (which is often different than what they say they do), and attitudinal techniques tell us what users feel and think and what they say they do.
On the horizontal axis, we see a spectrum between qualitative and quantitative data. Quantitative data gives us data on the measurement of use, and it’s often associated with numerical data. Qualitative data gives us a user’s description of their experience and can often reveal a fuller and more vivid picture than numerical data alone.
None of these types of data are superior to the others. We need to know how users feel about our services just as much as we need to know which services are actually used. Some situations may clearly require a specific type of data, but in general, the most insightful data will come from a combination or pairing of methods on both ends of the spectrum – attitudinal and behavioral, qualitative and quantitative. Also note that individual UX methods can be adapted to gather different data than what’s shown on this particular landscape. For example, a survey could be entirely qualitative, entirely quantitative, or somewhere in-between.
During this UX Sampler Sessions series, we are planning to showcase UX methods that span the ranges from behavioral to attitudinal and from qualitative to quantitative.

6 of 26

Method #1

Surveys

7 of 26

What is a Survey?

A Survey is a method of examining opinions, preferences, and behaviors, by asking users questions and recording their responses.

Image by Biljana Jovanovic from Pixabay

8 of 26

Use a Survey when you want

To quantify or measure (i.e. ‘how many, how much, or how often something occurs’)
To generate data suitable for communicating library value to stakeholders
To compare results in services over time
An overview of library services

9 of 26

Survey Pitfalls

Pre-existing surveys like LibQUAL are easy to administer, but may be difficult to map to local goals/time consuming to take
Local surveys may be hampered by lack of formal training in survey creation & analysis
Surveys alone cannot answer questions about why students use a service

10 of 26

Case Study: University of Illinois, Chicago

Chose local survey to customize questions and response options
Quantitative survey of undergraduate & graduate students
Distributed Spring 2016 & 2018
Purpose: Quantify students’ library visits and use of library resources

11 of 26

Case Study: University of Illinois, Chicago

Piloted survey with 6 - 8 students who read questions and shared thoughts aloud
Demographic information requested included key demographics: gender, age, class level, first generation status, and commuter status
Students asked to agree to an ‘agreement to participate’ document before taking the survey

12 of 26

Case Study: University of Illinois, Chicago

2016

2018

13 of 26

Case Study: University of Illinois, Chicago

Over-all in-person library visits increased.
However, because demographic data was included - the research group could see graduate in-person visits declined.

14 of 26

Analysis of Results

“Report writing proved to be a key step in changing services based on survey results” (Carlsson & Torngren)
Reviewing results, provides an opportunity to discover areas for further investigation.

Carlsson and Torngren note that “Report writing proved to be a key step in changing services based on survey results” and is a supporting indicator that surveys must be reflected upon and analyzed to be a useful tool for change.
At Walla Walla University we create reports for our student survey that mark changes over time, and those are given to our library faculty & staff, our campus library committee, and shared with administration and other shareholders as appropriate.
Surveys provide an excellent overview for a large general user group and when deployed consistently over time can provide quantitative data to mark changes in use
These changes, particularly decreases, provide opportunities for further ethnographic survey methods to understand WHY users are behaving in the way that they are.
Multiple studies on the use of surveys in library suggest that Survey analysis can help pinpoint areas that need further investigation with qualitative usability methods such as Focus Groups or Interviews that help to dig into WHY users are behaving in a certain way and therefore discover what changes could be made to improve services.
As an example, our most recent survey revealed that we had a particular user group on campus that was struggling with finding and accessing periodicals – which is a clear place where we want to do further research.

15 of 26

Analysing Activity

Considering data gathered from this survey, what are areas that could be targeted for further investigation?

16 of 26

Survey Method Tips

Consider if a survey is the best method to gain the information you want (see #DitchTheSurvey)!
Run questions by users to be certain you will get the answers you want
Check with your campus for ethical and safe data practices
Don’t be afraid to shift questions to get better answers

First, consider if a survey is the best method to gain the information you want! If you are wanting to know why users are behaving in a certain way – you likely need a different method. See #DitchTheSurvey in the source list – perhaps an ironic source to use for this presentation, but it provides an excellent overview of surveys, challenges to doing them well, and alternatives that may work better – this way you can know you’re using the right method for your questions.
Run questions by users so that you know that they understand what you are asking and thus you will get the answers for your question
Check with your campus for ethical and safe data practices. This is essential if you are using any identifying markers, and you will want to check with your institutions human research department for guidelines for safe and ethical data keeping.
Finally, if you are doing studies over time – don’t be afraid to shift questions to get better answers.

17 of 26

Method #2

Focus Groups

18 of 26

Overview of focus groups

An in-depth, structured discussion with users about a topic
Divergent/discover phase of the UX research cycle but coding themes from responses falls under convergent/define
Attitudinal and qualitative - what users feel about something and WHY they feel that way
Use this method for complex user questions where surveys or mapping activities don’t provide enough insight into user thinking

At the most basic level, a focus group is an in-depth structured discussion with users about a topic
Focus groups fall in the divergent/discover phase the UX research cycle, although coding user responses is more of a convergent
As a method it is very attitudinal and qualitative (similar to the love and breakup letters mehtod, presented earlier in this series) - users describe what they feel about something, whether in a physical library space or on a website, and they can tell you why the feel that way
It is a good method for complex user questions and as a follow-up to another method such as surveys or mapping. Especially when you need to know more about user thinking behind a behavior. For example, survey data says that students are aware of library course reserves and are burdened by textbook costs, yet circulation data says they are not checking out course reserves. Why is that? It’s an opportunity to have a discussion with students to learn more.

19 of 26

Focus groups - basic process

Select representative users
Craft open-ended, non-leading questions
Follow procedures for user consent and anonymity
Hold groups sessions, recording users responses
Code and summarize responses

Image source: https://en.m.wikipedia.org/wiki/File:Christ_University_focus_group_6.jpg

To get started with focus groups, you want to
Select users - targeted outreach to a specific population (participation should be voluntary), if you have more volunteers than you need to try to select a representative sample (i.e. students in different years or programs at your institution). under 10 people
Craft your questions - this seemingly simple process is often harder than it looks. If possible, test your questions on out on a colleague, friend, student woker to see how they respond and if you need to broaden or refocus questions. Very similar to what Christy talked about for surveys
Consent procedures - while your participants are voluntary, you should still formally get their consent. In the activity section of my presentation, I’ll talk more about where to look for guidance and examples on consent procedures
Hold your focus groups - the most fun part of the process! Make sure you have a comfortable space for your participants, that their access needs are met, and that you have a plan for recording and note taking (one or two colleagues to help with this process is beneficial!). Focus groups can also easily take place on Zoom. On Zoom consider enabling captions, posting questions in the chat, taking short breaks, allowing participants to stay off-camera and other accessibility needs.
Last you’ll want to code and summarize responses before moving in to the next phase of the UX research cycle

20 of 26

Tips

Schedule in advance
Compensate participants
Assign moderator and notetaker roles
Practice being comfortable with silence
How will participants learn more afterwards?

Having held focus groups and having been a focus group participant, these are some of the things I’ve learned to make the process easier.

Scheduling - have times for your focus group times set far enough in advance so participants know their availability and can block out times to their schedule (around a month)
Compensate users for their participation - especially with students this demonstrates their participation is valuable and will help drive participation
Assign roles to the staff running a focus group - moderator and notetakers. Moderators play an especially important role in keeping the conversation going.
Practice being comfortable with silence - give users plenty of time to respond. You can practice this before hand with colleagues or friends by asking each other questions and then waiting 30 seconds to a minute before you can respond. This really hard for me. Don’t fill the silence.
Even if you don’t know what deliverables will come out of the focus group, think about how/when participants can learn more (website, blog, campus listserv).

21 of 26

Case study of a focus group

Course Reserves focus groups at UW Tacoma Library

Who: UWT students
When: Spring 2024
Why: Review of course reserves program after looking at 5 years of circ data and survey on course material usage
Findings: Length of checkout too short, not enough awareness of the program
Next steps: Longer checkouts, faculty roadshow, promotion

22 of 26

Activity

Research consent forms

Your institution / UW
Office of research, human subjects division

https://www.washington.edu/research/hsd/do-i-need-irb-review/

Interested in doing focus groups? As I mentioned earlier, getting participant consent is an important part of holding a focus group. When I started the process of our course reserves focus groups, I was happy to learn that the Unversity of Washington had robust resources for staff and faculty doing research with human participants. I was able to determine that I did not need Institutional Review Board review and even find example participant consent forms and consent scripts to use during focus groups.

Take the next 5 minutes to research what support your institution has for non-Institutional Review Board or exempt research. Try looking for content under the department of research or the human subjects section of a department of research. This content could be procedures, example consent forms, people to contact. Feel free to share anything you find in the chat! If you’re having trouble finding information, take a look at what UW’s Human Subjects Division.

https://www.washington.edu/research/hsd/do-i-need-irb-review/

https://www.washington.edu/research/hsd/guidance/consent/examples/

23 of 26

Discussion

If your library completed surveys or focus groups in the past:

What lessons did you learn?
How did you use the data to improve your services?

What questions about your library users could a survey or a focus group help answer?

24 of 26

Resources for further learning

Surveys

Carlsson, H., & Torngren, T. (2020). Understanding library users via surveys and other methods—best practices for evidence-based library development. Journal of Library Administration, 60(8), 925-944. https://doi.org/10.1080/01930826.2020.1820276

Fink, A. (2017). How to conduct surveys: a step-by-step guide (Sixth edition.). SAGE.

Halpern, R., Eaker, C., Jackson, J., & Bouquin, D. (2015). #DitchTheSurvey: Expanding Methodological Diversity in LIS Research. In the Library with the Lead Pipe, 1–13.

Scoulas, J. M., & De Groote, S. L. (2020). University students’ changing library needs and use: A comparison of 2016 and 2018 student surveys. Evidence Based Library and Information Practice, 15(1), 59-89.

Stephen, P., & Hornby, S. (1995). Simple statistics for library and information professionals. Library Association Pub.

25 of 26

Resources for further learning

Focus Groups

Reading (and Recording) the Room: Focus Groups (Library Research Service)
Focus groups from home (CRL)
Focus groups 101 (Nielsen Norman)

26 of 26

Feedback

Sharing a link to a quick survey in the chat.

We will also send out this same survey via email after the UX Sampler Sessions series is completed.

Please take the survey now if you won’t be attending future sessions.