2019 Firefox Account Settings Information Architecture User Research

Closed Card Sort

Holly Collier

September & October 2019

How to cite this report: 2019 Firefox Account Settings - Information Architecture User Research - Closed Card Sort Results �Holly Collier, Firefox User Experience.

Public

Acknowledgements

Thank you to everyone who provided feedback, reviewed the report, and contributed hypotheses to the experiments.

​

Jennifer Davidson

Jeff Pfaller

Janice Cramer

Julia Starkov

Ryan Feeley

Alex Davis

​

2

Summary

3

Mozilla Public

Key Takeaways

• The most mentioned reason for visiting an account in the questionnaire was changing a password.
• Include changing a password as a task in Tree Test.

​

• Category labels and structure
• Keep as proposed:
• Profile
• Put additional “password” in Account Security, but otherwise keep as proposed.
• Account Security
• Keep an eye on “Delete account” in Tree Test.�
• Consider renaming (keep category content):
• Subscriptions & Billing
• Make it clear that these are paid product subscriptions and not email subscriptions.
• Account Security
• Make it clear that this also includes Sign In-related items

4

Mozilla Public

Overview:

Information Architecture Research for Firefox Account Settings

​

5

Mozilla Public

Information Architecture Research for FxA Settings

6

To answer the research questions and team goals, we will use the following methods:

​

Full Protocol

Full description of the Background & motivation, Purpose & goals

Mozilla Public

Lean User Research & IA Process

7

This is the second report in a three part research study:

​

Resources:

Competitive Analysis

Test 1: Open Card Sort Results

We Are Here

​

Figure 2: Lean User Research & IA Process Diagram

Mozilla Public

Closed Card Sort: Research Question

& Method

8

Mozilla Public

Research Questions - Closed Card Sort

9

• How do users think about the current and proposed pieces of content on the Firefox Account Settings website?
• Do the proposed names for categories make sense to users?
• How do they group items based on the categories names?

Mozilla Public

Method & Recruiting

10

Method: Remote, Unmoderated Closed Card Sort

30 participants will take part in a closed card sort using Optimal Sort, an asynchronous, web-based card sorting platform.

​

“Card sorting is a user-centered design method for increasing a system's findability. The process involves sorting a series of cards, each labeled with a piece of content or functionality, into groups that make sense to users or participants.” (Boxes & Arrows)

​

“A closed card sort is a card sort where participants are asked to sort topics from content within your website into pre-defined categories. A closed card sort works best when you are working with a pre-defined set of categories, and you want to learn how users sort content items into each category.” (Usability.gov - Card Sorting)

​

Recruiting: Desktop users, any browser or OS, US only

Mozilla Public

Cards: Content/Functionality

Existing:

Profile picture

Display name

Username / Primary email

Backup email*

Account recovery keys*

Configure Two-step authentication*

Active Firefox account sessions*

Password

Delete account

Sign out everywhere*

Communication preferences*

Preferred email language

Manage email newsletters*

​

​

Proposed/New:

Recovery device*

Sign out*

Security log*

Phone number

Manage paid subscriptions*

Contact support*

Payment method*

Billing address

Billing date and frequency

Purchase history

11

*Used description instead of current item label for card content. See 2019 Firefox Account Settings - Cards, Categories & IA - Planning & Analysis for details

Mozilla Public

Provided Category Set

12

Profile

Communication Preferences

Account Security

Subscriptions & Billing

Mozilla Public

Closed Card Sort:

Results

MINIMUM AMOUNT OF TEXT

13

Mozilla Public

Popular Placements Matrix

% of the time each card was sorted into a specific category

Figure 3: OptimalSort Popular Placements Matrix data visualization

Mozilla Public

Card Sorting: Intended Menu Tested

Category recommendations for top-level site navigation (based on findings & IA grouping/naming activity).

​

*Single menu items omitted from card sort.

Mozilla Public

Card Sorting Results: Overview

*Single menu items omitted from card sort.

Mozilla Public

Card Sorting Results: 60% +

60% placement or better in purple

​

Mozilla Public

Card Sorting Results: 80% +

80% placement or better in pink

​

Mozilla Public

Card Sorting Results: 90% +

90% placement or better in blue

​

Mozilla Public

We asked two questions following the card sort:

“Were there any cards where you felt uncertain about what category to place them in?”

• Participants were split: Half said “yes,” half said “no.”
• Mentioned: “password,” “profile,” “profile vs accounts security” �

“Think about the last time you adjusted account settings for some online account. Please describe that experience, and what you were changing.”

• Changing a password was mentioned by 8 participants
• Also mentioned: email address, payment method, contact info, phone number, login method
• 6 participants used adjectives like “easy”, “no issues”
• 4 participants used adjectives “frustrating and confusing,” “awful,” “not clear,” “difficult”

​

Recommendation: Include changing a password as a task in Tree Test.

20

Mozilla Public

Closed Card Sort:

Observations & Recommendations

21

Mozilla Public

Observations & Recommendations

Profile

• High agreement category. Keep as proposed.

​

Account Security

• High agreement category. Keep as proposed and add “Password.”
• The only item that caused confusion was “Delete account” (50%) which some participants placed in Profile (23%) or Subscriptions & Billing (23%). Given that we don’t want the delete action to be too prominent, I recommend keeping this placement.

​

Subscriptions & Billing

• High agreement category. Keep as proposed.
• There was some confusion with “Contact support” at (43%) — 33% placed it in Communications Preferences. Need to dig into why this is (could renaming it help?).
• Consider—HMW make it clear that these are paid product subscriptions that we’re providing support for and not email subscriptions?

22

Mozilla Public

Observations & Recommendations (cont.)

Communication Preferences

• This category performed the poorest, but not terribly overall.
• “Preferred email language” was 57% (27% placed it in Profile)
• “Choose text vs. HTML email format settings” was 57% (with 17% placing it in Profile and 20% in Subscriptions & Billing)
• “Manage Email Newsletters”, at 43% (37% placed it in Subscriptions & Billing). My hunch is that this may be because “newsletters” are something people think of “subscribing” to.
• I recommend that we consider keeping category content as proposed for now, for two reasons:
• This functionality is currently hosted on a different site (we can consider moving it later).
• In the open card sort, these cards showed up as pairings in the similarity matrix:
• “preferred email language” + “choose text vs html” was a 68.
• “Manage email newsletters” was most closely related (but still only 44) to “choose text vs email” even when worded as “subscribe or unsubscribe from email lists” See Test 1 Open Card Sort Results
• Consider:
• Could renaming Subscriptions & Billing help users understand the difference between the two categories?

23

Mozilla Public

Card Sorting: Intended Menu Tested

Category recommendations for top-level site navigation (based on open card sort findings & IA grouping/naming activity).

​

*Single menu items omitted from card sort.

Mozilla Public

Updated Intended Menu for Tree Test

Category recommendations for top-level site navigation (updated based on closed sort findings).

​

*Single menu items omitted from card sort.

Mozilla Public

Discussion

26

Mozilla Public

Next Steps

27

Mozilla Public

What is a Tree Test?

The tree test will test our hierarchical category structure by having users find the locations where specific tasks can be completed.

​

Tree testing is incredibly useful as a follow-up to card sorting because it:

• Evaluates a hierarchy according to how it performs in a real-world scenario, using tasks similar to a usability test; and
• Can be conducted well in advance of designing page layouts or navigation menus, allowing inexpensive exploration and refinement of the menu categories and labels.
• Tree test can be used later to give us a clear, quantifiable benchmark for us to improve on in future taxonomy iterations (If we test our first tree with 8 tasks, and then test revised our tree with the same 8 tasks, we’ll be able to pinpoint exactly how our changes have improved the findability of your information).

​

https://www.optimalworkshop.com/101/tree-testing#incorporateTesting

28

Mozilla Public

What questions does a Tree Test answer?

It helps us understand:

• How many people completed the task successfully?�
• Of those who were unsuccessful, what information did they select instead?�
• Did people go straight to an answer, or did they have to move back up the tree?�
• Exactly which path did people take before they selected an answer?

29

Mozilla Public

Example Task in Treejack

“How would you find out the Acme company‘s mailing address?”

Participants would see this on the tree like this:

​

And they would click through the tree until they could select ‘Our location’ to complete the task successfully:

​

30

Figures 4 & 5: Treejack - Tree Test interface screen captures

Mozilla Public

Example Treejack Visualization: Pietree

Helps us visualize overall success & directness of task completion

31

Figure 6: Treejack - Pietree data visualization

Mozilla Public

Tree Test Scope

Getting relevant and effective results.

What will we test: The proposed taxonomy for the Firefox Account Settings website, informed by two rounds of card sorts.*

​

Who: browser users in the US

​

Why are we testing: The current Firefox Account Settings website (accounts.firefox.com) will expand later this year to include subscription settings and support. Given that a refactoring of the underlying code is planned for early 2020, we want to propose a design and structure for the account website that takes into account Firefox’s expansion from a browser to a complex ecosystem of cross-platform products and services and is rooted in customer mental models.

​

*We can also test one version and iterate with future tests. If we have some strong competing ideas, we can test multiple versions. We can also conduct additional tree tests in the future, as we expand the functionality of the Account Settings website.

32

Mozilla Public

Tasks for Tree Test

List determined by team vote and product prioritization exercise*

Task 1: Account Security

Add back-up email address, account recovery key, recovery device

“You've gotten locked of accounts in the past. What could you do now to make sure that if you forget your Firefox password in the future, you'll have another way to get into your account?”

​

Task 2: Account Security Profile

Change Firefox account password

“You found out that a password you've been using for your Firefox account was caught in a security breach and you want to update it. Where would you go to do that?”

33

Task 3: Profile

Change Firefox account primary email address

“You got a new email address and you want to use it with your Firefox account. How would you change your email address?”

​

Task 4: Account Security

Sign out of Firefox account (locally)

“You got a new computer and want to sign out of Firefox on this computer before you sell it. Where would you go to sign out of your Firefox account?”

Mozilla Public

Tasks for Tree Test

List determined by team vote and product prioritization exercise*

Task 5: Account Security

Sign out of Firefox account (remotely)

“You lost your phone. Where would you go to unlink it from your Firefox account?”

​

Task 6: Subscriptions & Billing

Cancel a paid subscription

“You have been paying monthly for a data breach monitoring service and have decided you don't want it anymore. Where would you go to stop that monthly payment?”

​

34

Task 7: Subscriptions & Billing

Update credit card information

“You recently were issued a new credit card and want to use it to pay for the data breach monitoring service you pay for monthly to. How would you do that?”

​

Task 8: Communication Preferences

Change email subscription settings

“You're getting more emails from Firefox than you would like. How would you reduce the number of the emails you're receiving?”

​

​

Mozilla Public

Taxonomy for Tree Test

See the whole tree here:

Tree Test - Proposed Taxonomy v1

35

Figure 7: Treejack - Tree Test interface screen capture

Mozilla Public

Tree Test Schedule & Action Items

Week of 9/30-4

Holly PTO 9/30

10/1-3: Tree test & Analysis on OptimalWorkShop.com

10/3: Informal checkin with jdavidson (+jpfaller, jstarkov, jcramer). Edit based on feedback.

X/X: Repeat Tree Test with alternate tree structure if needed?

​

10/11: Findings Presentation Share Entire IA Study Summary report with Firefox Account Settings stakeholders

36

Mozilla Public

Appendix: Resources & Previous Findings

37

Mozilla Public

Background & Motivation

38

Study Motivation

The current Firefox Account Settings website (accounts.firefox.com) allows Firefox account holders (regardless of the browser or products they use) to manage their global Firefox account settings, including but not limited to email and password, synced devices and applications. Later this year, it will expand to include subscription settings and support. �The current design was intended to accommodate three or four items, not the more complex ecosystem of cross-platform Firefox products and services that we have today. Given that a refactoring of the underlying code is planned for early 2020, we’d like to take the opportunity to propose a design and structure for the website that takes this expansion into account and is rooted in customer mental models. (See also Manage Accounts Design Requirements Kick-off (Product Design Brief) )

Figure 1: Firefox Account Settings website

Prior Research

2019 Firefox Account Settings - Information Architecture User Research - Open Card Sort Results�

Mozilla Public

Purpose & Goals

39

The purpose of this work is to inform the information architecture of the Firefox Account Settings website (accounts.firefox.com) before planned refactoring of the underlying code.

​

Our goal in this multi-part study is to understand how users think about the current and proposed pieces of content on the Firefox Account Settings website including:�

• Understanding how participants would group and name the current and proposed Firefox Account Settings website content.�
• Evaluating the usability of the proposed information architecture with a series of tasks the participants will be asked to complete��2019 Firefox Account Settings User Research - Pt II - Information Architecture Protocol

Mozilla Public

Resources & Previous Findings

• Competitive Analysis
• Open Card Sort
• Full Protocol - Open Card Sort
• Full Analysis of Open Card Sort data (Optimal Sort)
• Open Card Sort Results

​

• Closed Card Sort
• Full Protocol - Closed Card Sort
• Full Analysis of Closed Card Sort data (Optimal Sort)

40

Mozilla Public

Questions?

Holly Collier

hcollier@mozilla.com

Thanks!