1 of 29

Caetra

Navigation card sorting exercise

Data, Analysis, and Conclusions

2 of 29

NAVIGATION OBJECTIVE

​

Study how Caetra users would organize the navigation menu, observe conserved organization preferences & create a sensible navigation architecture

3 of 29

Executive summary

AIM AND PROCESS

This exercise aimed was to propose an improved navigation menu for Cymetric. We used a card sorting exercise to get suggestions from seven team members. Participants were asked to put themselves in the mindset of an average user and sort cards into categories. Recommendations were based on analysis of raw data, a similarity matrix, and two types of dendrograms.

FINDINGS

Team members from ulta.io and Harris Beach were involved in the exercise, and while we had some common suggestions across the group, there was also significant variation, particularly among the Harris Beach team. However, there were still elements of the navigation that were conserved across all sorts, so the most common approaches were grouped together for the final recommendation.

4 of 29

Executive summary (cont.)

RECOMMENDATIONS

Looking through the raw data, similarity matrix, and dendrograms, categories were identified from clusters of card suggestions, and each category then has its own menu item with sub-headings.

NEXT STEPS

  • Implement a recommended navigation menu (see next page);
  • Discuss and refine navigation category names; and
  • Adopt a process of continual improvement, redesigning and perform tests in the future with end users to optimize navigation.

5 of 29

Recommended Navigation Menu

​

  • Dashboard
    • Reports
  • Information
    • Information types
    • Information systems
  • Controls
    • Security Controls
    • Draft controls
    • Control instances
  • Auditing
    • Risks
    • Assessments
    • Policies
  • My Organization
    • People
    • Vendors
    • User management
    • Security Frameworks
  • Settings
    • Profile

Note - will review Controls section as part of Control UX work (Sprint 11)

6 of 29

STUDY 1:

Semi-open (hybrid) card sort

7 of 29

Damon Jackman

Persona: Product manager

  • 6 categories
  • Noticed missing features & cards

​

8 of 29

Xuan Yue

Persona: Product designer

​

  • 6 categories

9 of 29

Dan Green

Persona: Product technical lead

​

  • 6 categories

10 of 29

STUDY 1:

Analysis

11 of 29

Study 1 Similarity Matrix Findings

​

  • Control-related items were grouped together
  • Assets, systems, resources were grouped together
  • Dashboards & reports were grouped together
  • Auditing, risks, assessments, and policies were grouped together
  • Info types and info systems were grouped together
  • My org and settings were grouped together
  • Contacts, user management, and vendors were grouped together

12 of 29

Actual agreement method

13 of 29

Recommended Navigation

​

  • Dashboard
  • Policies
  • Information
    • Info types
    • Info systems
  • Controls
    • Controls
    • Draft controls
    • Control instances
  • Risks & Assessments
    • Risks
    • Assessments
  • My Organization
    • People
    • My Org
    • Vendors
    • User management

Actual agreement method

14 of 29

Best merger method

15 of 29

Recommended Navigation

​

  • Dashboard
  • Policies
  • Information
    • Info types
    • Info systems
  • Controls
    • Controls
    • Draft controls
    • Control instances
  • Risks & Assessments
    • Risks
    • Assessments
  • Profile
  • My Organization
    • People
    • My Org
    • Vendors
    • User management

The Actual Agreement and Best Merger Methods yield same result, with exception of Profile

Best merger method

16 of 29

STUDY 2:

Open card sort

17 of 29

Study 2 Similarity Matrix Findings

​

  • Auditing, risks, assessments, and policies were grouped together
  • Reports and policies were associated
  • Control-related items were grouped together
  • Assets, systems, resources were grouped together
  • Settings & contacts were grouped together
  • My org and security frameworks were grouped together
  • Dashboards & reports were grouped together
  • Info types and info systems were grouped together
  • Contacts, user management, and vendors were grouped together
  • People, controls, profiles had very loose association

18 of 29

Dawn Russell

Persona: Technical User

  • Core functions vs. auxiliary functions
  • 5 categories

19 of 29

Alexander D. Rosati

Persona: C-Level User

  • Core functions vs. auxiliary functions
  • 3 categories

20 of 29

Alan Winchester

Persona: Member, Cybersecurity Protection and Response Practice Group

  • More granular breakdown
  • 6 categories

21 of 29

Mark Keith

Persona: System engineer

  • More granular breakdown
  • 6 categories

22 of 29

STUDY 2:

Analysis

23 of 29

24 of 29

Recommended Navigation

​

  • Profile
  • Core
    • Draft controls
    • Security Controls
    • Control instances
    • Report
  • Assets
    • Info types
    • Info systems
  • Auditing, Risks & Assessments
    • Policies
    • Auditing
    • Risks
    • Assessments
  • My Organization
    • Security frameworks
    • Contacts
    • Vendors
    • User management

Actual agreement method

25 of 29

26 of 29

Best merger method

Recommended Navigation

​

  • Security Frameworks
  • Assets / Org overview
    • Info types
    • Info systems
  • Controls
    • Draft controls
    • Control instances
  • Auditing / R&A
    • Risks
    • Assessments
    • Policies
    • Auditing
  • My Organization
    • Contacts
    • Vendors
    • User management

Results are chaotic, but “My Org”, “Assets” & “Risk and Assessments” are conserved across participants

27 of 29

Conclusions

28 of 29

Comparison of results from 2 studies

Study 1, A.A. Method

​

  • Dashboard
  • Policies
  • Information
    • Info types
    • Info systems
  • Controls
    • Controls
    • Draft controls
    • Control instances
  • Risks & Assessments
    • Risks
    • Assessments
  • My Organization
    • People
    • My Org
    • Vendors
    • User management

Study 1, B.M. Method

​

  • Dashboard
  • Policies
  • Information
    • Info types
    • Info systems
  • Controls
    • Controls
    • Draft controls
    • Control instances
  • Risks & Assessments
    • Risks
    • Assessments
  • Profile
  • My Organization
    • People
    • My Org
    • Vendors
    • User management

Study 2, A.A. Method

​

  • Profile
  • Core
    • Draft controls
    • Security Controls
    • Control instances
    • Report
  • Assets
    • Info types
    • Info systems
  • Auditing, Risks & Assessments
    • Policies
    • Auditing
    • Risks
    • Assessments
  • My Organization
    • Security frameworks
    • Contacts
    • Vendors
    • User management

Study 2, B.M. Method

​

  • Security Frameworks
  • Assets / Org overview
    • Info types
    • Info systems
  • Controls
    • Draft controls
    • Control instances
  • Auditing / R&A
    • Risks
    • Assessments
    • Policies
    • Auditing
  • My Organization
    • Contacts
    • Vendors
    • User management

29 of 29

Learnings

  • Need to clearly define terminology for next round of studies prior to sorting
  • Chaotic data from study #2--participants arrived at very different conclusions for how to sort
    • Need to simplify future studies
    • Consider closed card sort with pre-defined categories
    • Study should include more similar persona-types so that answers would cluster
    • Not enough data collected to be certain of results per group
  • Navigation structure is more conserved across ulta internal members, but Caetra team helped to guide the prioritization of navigation items