Heuristic

Evaluation

All Minds Math

Contents

• Heuristic Evaluation Overview
• Summary
• General Considerations
• Account Creation
• Math Topics
• Customization
• Transition Screens
• App Settings
• Feedback for UX Writing

Product: ADHD Math App

Task: Child Onboarding and Settings

Date: 10/12/2023

Evaluators: Research Team

• Andrew Salvatore
• Angela Yun
• Irina Lavrova
• Nardos Mechal
• Stefanie Kruse

Heuristic Evaluation

Nielsen’s 10 Usability Heuristics

Source: https://www.nngroup.com/articles/ten-usability-heuristics/

#1 Visibility of System Status

The design should always keep users informed about what is going on, through appropriate feedback within a reasonable amount of time.

#2 Match Between System and the Real World

The design should speak the users' language. Use words, phrases, and concepts familiar to the user, rather than internal jargon. Follow real-world conventions, making information appear in a natural and logical order.

#3 User Control and Freedom

Users often perform actions by mistake. They need a clearly marked "emergency exit" to leave the unwanted action without having to go through an extended process.

#4 Consistency and Standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform and industry conventions.

#5 Error Prevention

Good error messages are important, but the best designs carefully prevent problems from occurring in the first place. Either eliminate error-prone conditions, or check for them and present users with a confirmation option before they commit to the action.

#6 Recognition Rather Than Recall

Minimize the user's memory load by making elements, actions, and options visible. The user should not have to remember information from one part of the interface to another. Information required to use the design should be visible or easily retrievable when needed.

#7 Flexibility and Efficiency of Use

Shortcuts — hidden from novice users — may speed up the interaction for the expert user such that the design can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

#8 Aesthetic and Minimalist Design

Interfaces should not contain information that is irrelevant or rarely needed. Every extra unit of information in an interface competes with the relevant units of information and diminishes their relative visibility.

#9 Error Recovery

Error messages should be expressed in plain language (no error codes), precisely indicate the problem, and constructively suggest a solution.

#10 Help and Documentation

It’s best if the system doesn’t need any additional explanation. However, it may be necessary to provide documentation to help users understand how to complete their tasks.

Summary

Open Questions

Please provide answers to research team before usability test

Usability testing scope

Is the settings screen part of the usability test?

Presence of an adult

We assume, the majority of the onboarding should be done by the child. But some options might be too complex for an 8 year old (e.g. signup).

• Should an adult be present during the whole or parts of the onboarding process?
• If so, what action do we expect adults to take?

Clickable math buddy

What should happen if the math buddy is clicked? Currently it’s clickable. Is the math buddy supposed to read out questions?

Name vs. user name

What does the app need name and user name for? What is the difference?

Age

What is the age used for? Based on that, what format is required?

Transition screens

What is the purpose of screens 06 and 11? Simply provide encouragement or also to provide a break, so that kids don’t need to focus for so long?

Visual preferences

Is someone selects a modification (e.g. dark mode), will the whole app change or only the lesson screen?

Breaks

What exactly does the breaks setting do?

Math topics

Can users later adjust the topic choices?

This is a summary of all the open questions listed in this report for easy reference.

You will find the questions as well in blue boxes in context of the screens and issues they relate to.

High Priority Issues

Ideally fixed before the usability test

Back, skip, exit

Provide a back, skip, and exit/skip all action on all screens where it is possible.

Consistency in onboarding vs. settings

Ensure consistency between onboarding and settings OR decide not to test settings page.

Missing input labels

Add field labels that are always visible at the top left of every input field.

Forgot password

Remove “Forgot password” option from Sign up screen.

Automatic screen change

Depending on the purpose of the screens 06 and 11:

• extend and announce the delay OR
• provide a button for user to actively continue

Issues are listed in order of appearance.

Usability Test Objectives

What we should pay special attention to during the test

Length of onboarding process

Does the process feel too long?

Necessary vs. optional customizations

What do users require or can even assess before trying out the app?

Visual preferences

Do users understand how this screen works?

Do their expectations of changes match our idea of what these settings do?

Breaks

Do users understand how this screen works?

Is the input pattern problematic?

Start with lesson

Are users happy to start with the first lesson after onboarding or do they want to explore the app on their own?

Name, user name and age

Do these inputs provide problems for users? Do users understand what and why they are required?

These objectives only summarize the insights from this heuristic evaluation.

Understand them as additions to the existing test objectives from the design team

The items are listed in order of (subjective) importance.

General Considerations

It is good practice to keep onboarding processes as short as possible to:

• reduce cognitive demand
• allow users to experience value from the app before investing too much time in settings.

Currently we force users through a lot of customization options.

Especially if our UVP is customization, we shouldn't force users to make certain choices at a specific point in time, if they don't need them.

In the usability test, we should focus on exploring what customizations user want or need before trying out the app.

This is in line with design’s goals for the usability study.

General

Length of Onboarding Process

Currently all steps in the process are mandatory and users cannot go back to change previous choices.

• Some users might want to try out the app and explore if it brings value.
• Some users might do a lesson before knowing which customizations they need.

We don’t have any user data on what customizations are strictly necessary, popular, or optional yet.

Therefore, we recommend to add simple Back, Skip, and Exit options to each applicable screen before the usability test.

That way, we can get behavioral data on what, when, and why users want to change or skip: It is easier for testers to use “skip” than to tell us “I’m annoyed by the length of the flow”.

General

Flexibility in Onboarding Process

#3 User Control

Issue

• Users cannot go back.
• Users cannot skip customizations.
• Users cannot exit the onboarding process to get straight to the app.

Suggestions

Provide a back, skip, and exit/skip all action on all screens where it is possible.

Confidence

5/5

evaluators

Priority

High: Fix before usability test

General

Consistency between Onboarding & Settings

#4 Consistency

#9 Error Recovery

Issue

The customization options are currently inconsistent between onboarding screens and settings, e.g.

• not all visual preferences from onboarding are listed in visual settings
• big font vs. font size
• visual examples only in onboarding

Suggestions

Ensure consistency between onboarding and settings.

Confidence

5/5

evaluators

Priority

High: Fix before usability test OR decide not to test settings page

Question

Should this screen also be tested in the usability test?

Account Creation

#2

#3

#7

Screen 01

Sign-up and Log-in: Missing Labels

#5 Error Prevention

#6 Recognition

Issue

Input fields are missing labels. Placeholders disappear once user begins typing.

Suggestion

Add field labels that are always visible at the top left of every input field.

Confidence

4/5

evaluators

Priority

High: Fix before usability test

Screen 01

Sign-up and Log-in: Forgot Password

#4 Consistency

#8 Minimalism

Issue

Forgot password only makes sense for the Sign-in variant, not for Sign-up

Suggestion

Remove “Forgot password” option from Sign up screen.

Confidence

4/5

evaluators

Priority

High: Fix before usability test

#2

#3

#7

Screen 01

Sign-up and Log-in: Layout and Wording

#1 Visibility

#4 Consistency

Issue

Layout and wording might create confusion.

Sign up and Sign in happen on one screen, only separated by tabs, the wording is too similar, and the link at the bottom duplicates the tabs.

Suggestions

• use less similar wording, e.g. “Create account” & “Sign in” or “Sign up” and “Log in”
• keep the wording consistent throughout the page
• reevaluate tab pattern in favor of 2 screens accessible via link

Confidence

4/5

evaluators

Priority

Medium: Fix in high-fidelity version according to industry standards

#2

#3

#7

Screen 01

Sign-up and Log-in: Minimal Design

#8 Minimalism

Issue

There are a lot of elements on this screen which might overwhelm users.

Suggestions

Consider removing elements and/or splitting the information into separate screens, e.g.:

• welcome
• value of the app
• sign up
• log in

Confidence

2/5

evaluators

Priority

Medium: Fix in high-fidelity version according to industry standards

#2

#3

#7

Screen 02

Learner’s Profile: Missing Labels

#5 Error Prevention

#6 Recognition

Issue

Input fields are missing labels. Placeholders disappear once user begins typing.

Suggestion

Add field labels that are always visible at the top left of every input field.

Confidence

4/5

evaluators

Priority

High: Fix before usability test

#2

Screen 02

Name and User Name

#4 Consistency

#8 Minimalism

#10 Help

Issue

Do we need both name and user name? Can one be removed or the difference explained?

Suggestion

depends on open question

Confidence

3/5

evaluators

Priority

tbd: Evaluate in usability test

Question

What for does the app need name and user name?

#2

Screen 02

Age

#4 Consistency

#5 Error Prevention

Issue

Text input might be error prone

Suggestion

Use a selection pattern depending on the required age format.

Confidence

4/5

evaluators

Priority

tbd: Evaluate in usability test

Question

What is the age used for? Based on that, what format is required?

Are lessons/levels adjusted to the age? Then we should think about birth date or mm/yy to differentiate between a kid that just turned 9 vs a kid that’s almost 10.

Otherwise, if it’s not strictly necessary, can we remove it here?

#2

We could explore an option where this screen can be removed completely.

Math Topics

Screens 04 and 05

Math Topics: Selection Pattern

#1 Visibility

#2 Mapping

#4 Consistency

Issue

Selection pattern for the options can be improved:

• little difference between selected & unselected
• little affordance for clicking
• unclear if single or multi-select (compare screens 4/5 vs. 9/10

Suggestion

Fix above issues, e.g. with a common card pattern with different border & background colors, that includes radio buttons vs check boxes depending.

Confidence

4/5

evaluators

Priority

Medium: Fix in high-fidelity prototype

Note: this also applies to screens 09 and 10 for sound and buddy appearance.

Screens 04 and 05

Math Topics: Help / Explanation

#6 Recognition

#10 Help

Issue

The topic explanation is hard to access:

• users might not recognize the (i) icon
• info only visible in the modal (where the selection is not accessible)
• a lot of text in the modal

Suggestion

Include simple examples on the cards, as an image and/or subtext, e.g. a plus sign and an equation 2 + 3 = 5 for Addition. Explanation is always visible and close to the term it explains.

Confidence

4/5

evaluators

Priority

Medium: Fix in high-fidelity prototype

Note: this also applies to screen 05.

Screens 04 and 05

Math Topics

Question

Can users later adjust the topic choices?

E.g. an 8-year-old child might not need division now but in a few years. Also user might make an error here.

Question

What should happen if the math buddy is clicked? Currently it’s clickable.

Note: this also applies to screen 05.

Customization

Screen 07

Visual Preferences

#1 Visibility

#2 Mapping

#10 Help

Issue

We are not sure if our users understand how this screen works, e.g.

• how toggles work
• that the layout features an example screen
• whether they recognize the change from “bionic font” assuming they are not familiar with the term

Suggestion

Depending on usability test results

Confidence

5/5

evaluators

Priority

tbd: Evaluate in usability test

Question

Is someone selects a modification (e.g. dark mode), will the whole app change or only the lesson screen?

#3

#4

Screen 08

Breaks

#2 Mapping

Issue

Do users understand what this setting does?

• break length vs. break rhythm
• whether break occur within or between lessons

Does the intended setting match the users’ expectations?

Suggestion

Depending on intended setting and insights from usability test.

Confidence

3/5

evaluators

Priority

tbd: Evaluate in usability test

Question

What exactly does this setting do?

#1

#5

#8

Screen 08

Breaks

#4 Consistency

Issue

The horizontal picker is not common and may not be recognizable.

Suggestion

Use an industry-standard selection pattern, e.g. iOS-like wheel picker or + / - button group.

Confidence

3/5

evaluators

Priority

tbd: Evaluate in usability test

#1

#5

#8

Transition Screens

Screens 06 and 11

Automatic screen change

#1 Visibility

#4 Consistency

Issue

It is not expected that the next screen shows up after a short delay. The delay was to short (even for us) to read the text. This is especially problematic, since the user cannot go back and might worry they’ve missed sth. important.

Suggestion

Depending on the purpose of the screen:

• extend and announce the delay OR
• provide a button for user to actively continue

Confidence

5/5

evaluators

Priority

High: Fix before usability test

Question

What is the purpose of screens 06 and 11? Simply provide encouragement or also to provide a break, so that kids don’t need to focus for so long?

#2

#8

Screen 11

Duplicate Encouragement?

#2

#8 Minimalism

Issue

We are not sure if this screen is necessary, since it almost duplicates the next screen.

Suggestion

Remove / combine into one. This one (without the delay issue) might require less focus than the next screen with the modal.

Confidence

5/5

evaluators

Priority

tbd: Evaluate in usability test

Screen 12

Homepage Modal

#7 Flexibility

Issue

Users can only start with a lesson, but not explore the app on their own.

Suggestion

Provide flexibility but guidance. Remove the modal or at least let users dismiss it.

Confidence

4/5

evaluators

Priority

tbd: Evaluate in usability test

App Settings

Screen 14

Settings

#2 Mapping

Issue

• Breaks might not belong under Visuals.
• Profile area might not belong under Visuals.
• Math buddy might not belong under Profile.

Suggestion

Rework information architecture based on common mental models & industry standards.

Perform card sorting to gain confidence.

Confidence

4/5

evaluators

Priority

Fix independently of usability test

#3

#7

#8

Screen 14

Settings

#2 Mapping

#5 Error Prevention

Issue

Caregiver area makes little sense under Visuals (probably there are other caregiver-only settings in the other categories.

Having it in-between the child-accessible option encourages errors (child wants to access but can’t).

Suggestion

Consider separate areas for child settings and adult settings in the menu. The child only sees a subset of available settings controlled by the adult's choices.

Confidence

5/5

evaluators

Priority

Fix independently of usability test

#3

#7

#8

Feedback for

UX Writing

UX Writing

Simplify language for our target group

UX Writing

Simplify language for our target group

UX Writing

Simplify language for our target group

UX Writing

Simplify language for our target group

UX Writing

Simplify language for our target group

#2 Mapping

#5 Error Prevention

Issue

We had to think about if “learn together” meant

• the topics of multiplication & division covered together
• the child and the math buddy learning as a team.

Probably the second one, but it took some discussion.

Suggestion

Remove ambiguity, e.g.

• “What would you like us to learn together?”
• “What topics would you like to learn?”

Confidence

1/5

evaluators

Priority

Low