Shop & Support Covid Response
User Research
Liam Lai
May 21, 2020
Usability Testing vs. Live Site Testing
- Usability Testing
- Qualitative and directional
- Focuses on WHAT users do (vs. what they say) AND WHY they do it
- Depth of user insights
- Live Site Testing (e.g. A/B testing)
- Quantitative with scale
- “Footprint” of user behavior (tells you what users do, but not why)
- Lack of user insights (why)
- Usability Testing and Live Site Testing are Complementary
Usability Testing Methodology
3 to 5 users is optimal in usability tests (Nielsen Norman Group)
Methodology - Overview
- Usability test with 3 users ~ 1.5 hours each
- 3 users in 3 individual sessions:
User 1: Ann (Liam’s mother, frequent online shopper) in-person
User 2: John (Liam’s grandfather, frequent online shopper) via Skype
User 3: Solvay (Liam’s grandmother, infrequent online shopper) via Skype
- Users visited Shop & Support Coronavirus site: https://shopandsupport.org/?utm_source=coronavirus
- Users were asked to use the website, and while going through the flow, they were observed and asked to verbalize their impressions.
Methodology - Skype/remote
- Usability test with User 2: John
- 2 of 3 usability tests were conducted through real time skype screen sharing.
- 1 of 3 usability tests was conducted in-person
Methodology - User Flow
Home Page
Enroll Page
(Search Charity)
Parent Charity Page
Search Charity
Results Page
Charity Page
Link Credit Card Page
High Level User Themes (1 of 3)
- User 1 would have made it the furthest in the flow (stopping at credit card page due to not confirming her store and having credit card security concerns)
- User 2 would have reluctantly made it past the home page to the enroll page (to see info on stores and charities) and then stop at parent charity page due to privacy concerns
- User 3 stopped at the home page (due to security concerns and lack of stores and charity info)
- 2 out of 3 users felt there are too many steps to donate
High Level User Themes (2 of 3)
- Want to know more about Shop & Support (2/3)
- Thinks Shop & Support is a charity (1/2)
- Doesn't know how the shopping process works (3/3)
- Security concerns (does not want to link credit card) (3/3)
- Asking for user's credit card before they select store and charity is too early (3/3)
- Want store selection and charity selection before enrolling (3/3)
- Difficulty selecting charity (3/3)
- Difficulty selecting store (3/3)
High Level User Themes (3 of 3)
- Want to know that the store(s) they shop at are participating (3/3)
- Want information about charities, including coronavirus causes (3/3)
- Want to know where donation money is going (what coronavirus causes: e.g. feeding homeless, frontline workers, or vaccine research) (3/3)
- Does not want to enter email (2/3)
- Preference for local charity (2/3)
- Enroll page is not what users expected (expected to enroll) (3/3)
- Feeling of invasion of privacy (e.g. automatically showing same location) (1/3)
- Doesn't like pressure of buying online, prefers buying a gift card to use later (1/3)
Home Page
- Doesn't know how the shopping process works (3/3)
- Want to know that the store(s) they shop at are participating (3/3)
- Needs list of stores (perhaps with categories) (3/3)
- Feels charity options are limited (3/3)
- Needs list of charities (2/3)
- Want information about charities, including coronavirus
causes (3/3)
Home Page (continued…)
- Want to know where donation money is going (what coronavirus causes) (3/3)
- Want to click on stores expecting to see additional information (3/3)
- Want to click on charities expecting to see additional information (3/3)
- Needs to know more about Shop & Support (2/3)
- Want to click on "and many more"...button expecting to see stores (3/3)
Enroll Page (Search Charity)
- Enroll page is not what users expected
(expected to enroll) (3/3)
- Feels charity options are limited (3/3)
- Want to know these are coronavirus
charities (3/3)
- Wanted to see stores before charities (2/3)
- Missed X button (2/3)
- X isn't clear (2/3)
- Wanted to go back to home page but didn't know how to (2/3)
- Search box should have categories of coronavirus causes (1/3)
Parent Charity Page
- Experienced feeling of invasion of privacy (e.g. automatically showing same location) (1/3)
- Want information about Charity, including coronavirus causes (3/3)
Search Charity Results Page
- Want to know if charities shown are
covid charities (3/3)
- Thinks donations are going to
general funds (2/3)
- Clicks X instead of back button (2/3)
- Does not know what left arrow does (1/3)
- Needs to find charity supporting Covid (3/3)
- Needs to know more about charities (3/3)
- Want to know what the address means (3/3)
Charity Page
- Want information about charity,
including coronavirus causes (3/3)
- Do not feel comfortable
entering email (2/3)
- Had trouble creating password (2/3)
- Confused why left arrow has button but X does not (1/3)
- Want to find local charities (2/3)
- Want to know what the address means (3/3)
Link Credit Card Page
- Security concerns (doesn’t want to link
credit card when haven’t purchased) (3/3)
- Confused why website needs
credit card (3/3)
- Thinks credit card will be linked to charity
(after enrolling from charity page) (1/3)
- Didn't see their credit card in list and therefore didn't think could use it (1/3)
Summary
- Doesn't know how the shopping process works (3/3)
- Security concerns (does not want to link credit card) (3/3)
- Asking for user's credit card before they select store and charity is too early (3/3)
- Want store selection and charity selection before enrolling (3/3)
- Difficulty selecting charity (3/3)
- Difficulty selecting store (3/3)
- Want to know that the store(s) they shop at are participating (3/3)
- Want information about charities, including coronavirus causes (3/3)
- Want to know where donation money is going (what coronavirus causes) (3/3)
- Didn’t make it past the link credit card page (3/3)
Next Steps
- Review and digest user research findings
- Regroup as a team to align on priorities
- Begin design explorations to address priorities
Appendix
User Research Findings Table
User Research Notes: User 1
User Research Notes: User 2
User Research Notes: User 3