1 of 28

UX Sampler Sessions

Observational Methods

Spring 2025

User Experience Standing Group (UXSG)

Orbis Cascade Alliance Diversity & User Experience (DUX)

2 of 28

Introductions

Leah Cover (she/her)

The Evergreen State College

Thomas Dodson (he/him)

Southern Oregon University

3 of 28

Agenda

UX research cycle & methods
Methods: Observational space studies & Usability testing

Overview, tips, and best practices
Case study
Activity

Discussion
Resources for further learning
Survey for future trainings

4 of 28

UX Research Cycle

Adapted from: Priestner, A. (2021). A handbook of user experience research & design in libraries. UX in Libraries.

The primary goal of UX work is to center the experience of the user as we assess services, processes, digital and physical environments, and more.
What you see here is one version of a UX research cycle that begins, continues, and ends with user-centered data. This double diamond model was adapted and shared by Andy Priestner in his book A Handbook of User Experience Research & Design in Libraries.
The diamonds are composed of divergent and convergent phases. The divergent phases, which are visually represented by open-ended arrows, are all about “remaining open to new possibilities and opportunities rather than relying on what you think you already know” (Priestner, 2021, p. 15). Divergent states are often creative, spontaneous, and free-flowing. Divergent states are helpful because they allow us to think expansively about a problem for a while before jumping to solutions.
The convergent phases, visually represented by closed-ended arrows, move in the opposite direction – moving towards a single solution or approach to the exclusion of other options. These phases allow us to consider many ideas from a systematic and logical standpoint and then make decisions about which ideas to pursue further.
The double diamond model includes four phases from start to finish: discover (learn about user needs and behaviors), define (processing and analyzing user data), develop (ideate and devise prototypes), deliver (test and refine prototypes). Most of us don’t have the time or resources to pursue all four stages of UX work for every problem that we encounter, even if that is the ideal. We’re sharing this model to help contextualize some of the UX methods that we’ll share, and we want to emphasize that a UX method can produce meaningful data without being part of a “complete” research cycle.

5 of 28

UX Research Methods Landscape

Adapted from: Priestner, A. (2021). A handbook of user experience research & design in libraries. UX in Libraries.

There are many UX methods to choose from, and there is no “one best method” to rely on. Each method offers unique insights and is suited to particular contexts and needs. On this slide, you’ll see a landscape of various UX research methods that are plotted along two axes that represent four types of data.
On the vertical axis, we see a spectrum between behavioral and attitudinal techniques. Behavioral techniques tell us what users actually do (which is often different than what they say they do), and attitudinal techniques tell us what users feel and think and what they say they do.
On the horizontal axis, we see a spectrum between qualitative and quantitative data. Quantitative data gives us data on the measurement of use, and it’s often associated with numerical data. Qualitative data gives us a user’s description of their experience and can often reveal a fuller and more vivid picture than numerical data alone.
None of these types of data are superior to the others. We need to know how users feel about our services just as much as we need to know which services are actually used. Some situations may clearly require a specific type of data, but in general, the most insightful data will come from a combination or pairing of methods on both ends of the spectrum – attitudinal and behavioral, qualitative and quantitative. Also note that individual UX methods can be adapted to gather different data than what’s shown on this particular landscape. For example, a survey could be entirely qualitative, entirely quantitative, or somewhere in-between.
During this UX Sampler Sessions series, we are planning to showcase UX methods that span the ranges from behavioral to attitudinal and from qualitative to quantitative.

6 of 28

Method #1

Space Observation Studies

7 of 28

Space Observation overview

observe how users actually use spaces
most often used in initial divergent / discover phase
behavioral (what people actually do)
mostly quantitative (measurement of use)
For a more complete picture, pair this observational method with a attitudinal method, such as survey or focus group.
Pros: Low-cost, minimal researcher bias, flexible design
Cons: Limited data, cannot answer “why” questions

8 of 28

Space Observation basic process

Determine the space(s) you’d like to observe
Create a template to record observations consistently
Observe spaces according to a set schedule
Compile data for analysis

Gullikson, S., & Meyer, K. (2016). Collecting space use data to improve the UX of library space. Weave: Journal of Library User Experience, 1(5). DOI: 10.3998/weave.12535642.0001.502

Determine the space(s) you’d like to observe

Multiple ways to observe spaces, depending on what data you want to capture. Sit in a space for an hour at a time or walk through a space on set schedule to observe how space is being used.
What types of space usage are you curious about? How will you record that data? For example, are you interested to see simply how many people are sitting in a specific area, or are you also curious about the technology they’re using in that space? Or if they’re alone or in groups?

Create a template to record observations consistently

The image you see here is a floor plan from Carleton University’s library used by librarians to observe space use. The goal in this case was to observe exactly where patrons were sitting, so they included all seating options and marked off occupied seats as they walked through on regular intervals.
Templates can be much simpler than this, and we’ll look at another example in a moment.

Observe spaces according to a set schedule
Compile data for analysis

9 of 28

Space Observation tips

Flexible scale & scope
Approximately 30 observations
Observe at different times of day, week, year
Many options for recording observations, but think ahead about how you will compile and analyze data
Only need one observer, but more can be useful
Participant privacy and consent

Scalability & scope

Observation studies can be scaled to align with need and available resources. Doesn’t have to be time-intensive to yield meaningful results.
You also don’t have to observe an entire space; you could observe the use of one room, one type of space across the library (e.g. study carrels or computer stations), one floor, or an entire multi-level library.

How many observations do you need?

There’s no rule, but I’ve seen multiple studies collect approximately 30 observations in total. Some studies collect way more data over longer periods of time. Ultimately, it depends on staff resources, how large and varied your space is, how complex your question is, the rhythms of traffic in your space, and how much data you expect you’ll need to see meaningful patterns.

When to observe?

In most cases, you’ll want to observe at different times of day, different days of the week, and different weeks of the year. If possible, it’s useful to observe both busy and slow times of day/week/year. If you only look at the busiest times of day/week/year, you’ll see how patrons use space when there are limited options. If you balance this with observing slower times, you’ll see how people use space when more options are available.

How to record observations?

Depends on the data you want to gather and how much time you want to spend observing.
The floorplan that we saw on the previous slide was chosen as a recording template, in part, because it allowed for quick data gathering that would be relatively less intrusive for the observed patrons.
Another option could be to have a template where you answer a series of questions with numerical or qualitative data during each observation.
Think ahead about exactly what data you need and how you will compile and analyze it once you have it.

How many observers do you need?

If you’re simply counting the number of people in a study room, there’s little room for interpretation and it’s probably fine to have one observer. But if you’re observing other factors or recording any qualitative data, you’ll get a fuller understanding of the space when you have more than one observer. People notice different things. If you have multiple observers, make sure there are very clear directions for how to record observations consistently so that your data will be easier to analyze at the end.

Participant privacy and consent

Since you are simply observing people in public spaces and not interacting with them, asking for consent is not necessary.
Protect patron privacy by keeping observation notes anonymous. Don’t include any personally identifiable information in observation notes.

10 of 28

Space Observation case study

Study room use at North Carolina State University Libraries in Fall 2023
Observed 10 study rooms 3x each during peak dates/hours = 30 observations
During the observations they recorded:

use of the room’s screen
the number of people in the room
use of whiteboards, writable tables, and writable walls
collaboration of groups in the rooms
patrons engaged in video conferences

Source: https://www.lib.ncsu.edu/projects/study-rooms-observational-study-report

11 of 28

Space Observation case study

What did they find?

Wall-mounted screens were rarely used (7%)
Median occupancy was 1.5; overall occupancy rate was 35%
Someone was writing on whiteboards, tables, or walls (45%)
In cases with 2+ people, people were interacting/talking ~66% of time

How did they use the data?

Combined findings with Student Experience Survey from same semester
Consider the need for more, smaller study rooms for solo use
Used data to identify questions for further exploration

Wall-mounted screens were rarely used (2/27 times, 7% of observations)
Median occupancy was 1.5; only 1 person in room 12/30 times (40%); overall occupancy rate was 35%
In 12/27 (44%) observations, someone was writing on whiteboards, tables, or walls.
In 10/15 observations with 2+ people, people were interacting/talking.

How did they use the data?

Combined findings with Student Experience Survey from same semester, which had questions dedicated specifically to library study rooms.
Since 2020, rooms are often used by solo individuals. Consider the need for more rooms that are smaller.
Used data to identify questions for further exploration:

Why do patrons so seldom use the screens in study rooms? Is it because they do not need them, or is it due to technical difficulties?
Do we need dedicated Video Conferencing Rooms?

12 of 28

Space Observation activity

Scenario: You’re planning to use a floor map as the template for your space observation study.

What strategies would you use to compile the data for analysis?

Gullikson, S., & Meyer, K. (2016). Collecting space use data to improve the UX of library space. Weave: Journal of Library User Experience, 1(5). DOI: 10.3998/weave.12535642.0001.502

13 of 28

Method #2

Usability Testing

14 of 28

Overview of Usability Testing

The term “usability” refers to a characteristic of user interfaces: essentially, how easy is a given interface to use?

Usability testing is a research method for evaluating the usability of an interface; it involves observing the behavior of typical users as they interact with an interface, often in a controlled environment (Blakiston, 2015)

The term “usability” refers to a characteristic of user interfaces: essentially, how easy is a given interface to use? Usability is defined and assessed by such factors as “learnability: how easy is it for users to accomplish basic tasks the first time they encounter the design?” and “errors: how many errors do users make, how severe are these errors, and how easily can they recover from the errors?” (Nielsen, 2012).

Usability testing is a research method for evaluating the usability of an interface; it involves observing the behavior of typical users as they interact with an interface, often in a controlled environment (Blakiston, 2015). A facilitator provides the participant with scenarios representing “common user needs or behaviors” and encourages them to narrate their thought process as they complete them (Guay, 2019, p. 88). The facilitator and note-taker record what the participant says and does: What seemed to confuse the user? How many attempts did they make before they successfully completed a scenario? Or before they gave up in frustration?

15 of 28

Usability Testing in UX Process

About the method

Convergent. Solution-oriented.
Behavioral. What users do.
Qualitative.

Use this method when...

Have a prototype or an existing product you want to test and iterate on

I think of usability testing as something that occurs in the second diamond of the UX Research Cycle, in the Deliver phase. That’s because you’re testing with real users, whether this is on your actual site or app; lo-fi prototypes or even designs on paper rather than on actual screens and devices. In any case, you’ve actually got something to test that came out of the earlier stages of the R&D process.

The method is behavioral, focusing on what users actually do as opposed to attitudinal, what users say they do.

It’s also qualitative rather than quantitative, because although you will be crunching some numbers—how many users completed a task for example, and how long and how many tries it took them—you’re not aiming at statistical significance. You’re looking instead for the most obvious problems users are encountering on your site and the ones you’ve actually got the development resources to fix.

Finally, it's convergent because you’re trying to zero in on the most critical problems with your web property and then implement specific changes to solve problems users encounter while testing your site. This isn’t to say you wouldn’t start the whole research and design cycle over again to, say, develop a new menu if you discovered your existing one wasn’t very useable. Often, however, usability testing will reveal at least some problems that can (and probably should) be solved immediately without taking the time to go through the full cycle.

16 of 28

Methods for In-Person Testing

Recruitment

Intercept Recruitment
Formal Recruitment

Location

In-the-Field Testing
In-the-Lab Testing

Technology

Local
Participants’

Some of the first decisions you’ll make about how to conduct your usability testing will have to do with methods. Specifically: recruitment, location, and technology.

I would say the gold standard for usability testing is to schedule appointments with subjects in advance for formal testing, usually in a room set aside for that purpose. This approach allows you to have more control over the environment and hold longer session, perhaps 90 minutes. It does, however, require more valuable incentives to make sure your participants show up. It can also be seen as less authentic than in-the-field testing.

Another common approach, particularly for testing a discrete change to a site or app, is informal in-the-field testing using intercept recruitment. In this approach, you recruit participants at the time of testing and present them with simple tasks and scenarios that, in total, don’t take more than a few minutes. Setting up a table at a library outreach event with a laptop and paper prototypes and giving away mugs or T-shirts to participants in return for their time is an example of this kind of low-cost testing.

The last consideration is whether to use technology you provide or the user’s technology. Allowing participants to use their own devices is more authentic. It can be more difficult logistically, however. It makes it harder to use recording software, for example.

17 of 28

Formal Usability Testing | Process

Select & Recruit Users
Assign Roles (Moderator & Notetaker)
Craft Tasks & Scenarios
Prepare Consent Forms, Releases & Moderator Script
Select & Prepare Technology
Hold Individual Sessions (Speak Out Loud)
Prepare Report for Stakeholders

18 of 28

Tips

Only need 5 participants (Nielsen 2000)
Student workers know too much (Sexton 2022).
Have alternates if a participant doesn’t show up.
Compensation for formal usability testing is typically $75/session (can get by with $50 for students)
Consider specialized recording software (e.g., Lookback)

19 of 28

Example of Usability Testing

SOU

Hannon Library

November 2024

Library Website & Primo VE

Logo Reel

20 of 28

Activity

Tasks to Scenarios

Task. “What you want the user to do (for internal purposes).”
Scenario. “What you tell them to do (for participant)”
Blakiston 2015, p. 46.

21 of 28

Task

What you want the user to accomplish
User never reads this
Clearly define success (may be multiple paths)
Can use “library language”

22 of 28

Scenario

Read aloud by the user during testing
More like a brief narrative
Do not use “library language”
Do not use names of links or buttons

23 of 28

Examples

Task

Find & Access an eBook about artificial intelligence
Success: Using Primo VE, search for, locate, and access an eBook about artificial intelligence.

Scenario: You’re interested in artificial intelligence and want to find a good book to read about the topic. You’re comfortable at home, though, and you don’t want to go to the library or the bookstore. You’ve heard that the library has lots of online books that you can access from home. Find and access an online copy of a book about artificial intelligence.

24 of 28

Create a Scenario from One of These Sample Tasks

Task: Find the research guide for Psychology.

Task: In the catalog, find a book on reserve.

Task: Find a specific library event.

25 of 28

Discussion

Have you or your library conducted observational studies? If so, what did you do, and how did you use the data?

What types of questions could observational studies help to answer in your library?

26 of 28

Resources for further learning

Space Observation Studies

27 of 28

Resources for further learning

Usability Testing

Blakiston, R. (2015). Usability testing: A practical guide for librarians. (Practical guides for librarians, No. 11). Rowman & Littlefield.

Dodson, T., & Juliani, J. (2025). Hannon Library Web Properties: Fall 2024 Usability Testing. Hannon Library. https://thomasadodson.com/wp-content/uploads/Usability-Testing-Report_01_2025.pdf

Guay, S., Rudin, L., & Reynolds, S. (2019). Testing, testing: A usability case study at University of Toronto Scarborough Library. Library Management 40(1–2), 88–97. https://doi.org/10.1108/LM-10-2017-0107.

Lewis, C. H. (1982). Using the "thinking aloud" method in cognitive interface design (Technical Report No. RC 9265). IBM Thomas J. Watson Research Center.

Nielsen, J. (2000). Why you only need to test with five users. Nielsen Norman Group. https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Nielsen, J. (2012). Usability 101: Introduction to usability. Nielsen Norman Group. https://www.nngroup.com/articles/usability-101-introduction-to-usability/

Nodder, C. (2022). UX foundations: Usability testing. [Online course]. LinkedIn Learning. https://www.linkedin.com/learning/ux-foundations-usability-testing

Sexton, M. (2022). Convenience sampling and student workers: Ethical and methodological considerations for academic libraries. Journal of Academic Librarianship 48, Article 102539. https://doi.org/10.1016/j.acalib.2022.102539

28 of 28

Feedback

Sharing a link to a quick survey in the chat.

We will also send out this same survey via email after the UX Sampler Sessions series is completed.

Please take the survey now if you won’t be attending future sessions.