Usability Test Results -
Homepage redesign
Yuki Shimano
February 15, 2023
Table of Contents
The Team……………………………………………………………………….3
Project goals & scope………………………………………………………….5
Methodology & research plan………………………………………………...6
Participants / user profile…………………………………………………….7
Homepage redesign usability test results………………………………..…8
Overall feedback – Positive…………………………………………………..9
Overall feedback – Areas to continuously improve...................................10
Major findings………………………………………………………………...11
JOA exploration usability test results………………………………………32
Overall feedback – Positive………………………………………………….33
Overall feedback – Areas to improve……………………………………….34
Major findings………………………………………………………………...35
Next steps……………………………………………………………………..54
Appendix………………………………………………………………………56
The Team
Facilitator
Yuki Shimano - UX/UI Designer
Note takers
Susan Fischer - Business Analyst
Tiffany Rost - Content Strategist
Rolf Siverson - Business Analyst
Jodie Young - Business Analyst
Project goals & scope
Homepage redesign
The goal was to evaluate the redesign and determine the final improvements by answering the following questions:
Methodology & research plan
Remote moderated usability testing (1/9 – 27)
Overall steps & plan
Findings & recommendations
(Tasks are located in the Appendix.)
Limitations:
device, participants
Participants / User profile
11 Federal Employees
6 non-federal individuals who have or have had interest in federal employment
88%
were familiar with job searching on USAJOBS.
looked for a job through USAJOBS in the last year.
82%
The public participant screener is located in the Appendix
Homepage redesign
Usability test results
Overall feedback - Positive
The colors look good and not blinding.
1
The page looks more interactive due to the new Events and Career Spotlight section.
The Urgent hiring needs section looks helpful in narrowing down a job.
The page separates the sections clearly, and makes it easy to find information overall.
2
3
4
See Appendix for task difficulty ratings.
Every participant understood the purpose of the website.
5
Overall feedback - Areas to �continuously improve
There are a lot of job resources, so visitors are overwhelmed or unsure of where to start. Besides upgrading resource names, continue to measure the traffic to each job resource to remove anything unnecessary.
1
The events section does not feel relevant to the user overall either because it’s not necessary to find a job, or events previewed on the homepage are not tailored to the visitor’s interests/location.
2
Major findings
Severe or frequent usability problems
Unclear where some job resource sections will take you
"I'd assume if I click on one of these, it will show me…an overview page. I think I expected it to be like a list of jobs." -Participant 8
9/17 participants
"Thought that the ‘Browse all hiring paths’ button allows for seeing all jobs combined." -Participant 17
Finding 1
"There's a little ambiguity between the Career Spotlight and Urgent hiring needs. Is this also urgent hiring? Are Infrastructure and Intelligence Careers hot markets right now? I would like context around that." -Participant 8
"Couldn't this part [for the Find professional development opportunities] be a part of events, or is this only open to federal employees?" -Participant 4
Unclear where some job resource sections will take you
Finding 1
5/17 participants
Solution 1
New header name
New button text
Changing the header names as well as links/buttons to be more clearer will help users know what to expect when they click on a link.
The ‘Browse all hiring paths’ text was updated to ‘Learn about all hiring paths’.
a.
Solution 1
Besides changing the headings to be more specific (which will help differentiate the section from ‘Urgent hiring needs’), improve the Open Opportunities card content to clarify who the resource is for.
b.
New header name
New content
Users want to see job resources before events
Finding 2
Users want to see job resources first over events because their primary purpose is to look for jobs.
"I was originally wondering why there are events because I'm looking to apply for a job...If I'm job searching I'm looking for a job." -Participant 2
Events section
5/17 participants
53% said they will unlikely look for events on the homepage.
Solution 2
Move the Events section after the Career spotlight, so that all job resources are located together at the top.
There is a link to the events page added in the navigation that we anticipate more users will use once they are familiar with it.
Think of incorporating events in the job search process.
swap
Urgent hiring needs can be hard to spot
Finding 3
Users may skip Urgent hiring needs because they don't realize it or expect it to be important when it's an accordion.
"I didn't expect to see the urgent hiring needs section to be in an accordion, which seems to be used for things that are optional like FAQs" -Participant 1
"I didn't realize the little plus icon [on the accordion]" -Participant 14
Urgent hiring needs accordion
4/17 participants
Solution 3
Conduct a follow-up test to see if an accordion is intuitive, and maybe compare with an alternative design, like:
A design that partially displays the section with a read more link at the bottom, that will expand everything when clicked
To improve the header name, recheck if these are urgent or just random jobs. Reestablish how we define Urgent hiring needs.
a.
Possible alternative design to show in the follow-up test
b.
Visitors want to see real images
Finding 4
Images should be or look real rather than stock.
"I like the ones you've chosen, but I do prefer real images to stock. The intelligence career image looks like stock." -Participant 13
2/17 participants
Solution 4
Ask for photographs from the Office of communications, Megan, Public affairs office, or Rachel Bushman for new images. If the image needs to include people, make sure there is a diversity of the people represented as much as possible.
To be replaced
New image
The Create a profile card looks like an advertisement
Finding 5
Areas that look like an advertisement do not look important and users think it's okay to skip them.
"[The 'Start shaping America's future' infographic] looks like an Ad. On other websites, sometimes there's an Ad that pops up, and it looks like that. People might ignore it." -Participant 16
2/17 participants
Solution 5
For the 'Start shaping America's future' infographic, the card in the background will be removed to make it blend with the image and not look like an Ad.
New links conflict with the footer
Finding 6
"I like the carrot that takes you all the way to the bottom, would be nice to have one that takes you back up to the top." -Participant 3
"Why is there a 'return to the top' link? It's in between a lot of information and isn't very visible. What is the down arrow at the bottom right?" -Participant 16
2/17 participants
"What's the difference between this 'Help' [link in footer] down here and this 'Help Center' [link in card]?" -Participant 12
New links conflict with the footer
1/17 participants
Finding 6
Solution 6
Keep the ‘Return to top’ link which is part of the design system’s footer. Remove the scroll up/down icon link
a.
Solution 6
Keep the Help Center card and remove the links in the footer.
b.
The navigation icons look small and unnatural (Mobile)
Finding 7
The navigation on mobile looks small and doesn't follow the common mobile navigation design (hamburger menu).
"I think the icon at the top is small…The spacing in between doesn't make it look like a navigation. Most mobile designs will have a hamburger menu, but you have every icon laid out and it looks weird…You don't see a navigation like this with icons and text at the top." -Participant 16
1/17 participants
Solution 7
Replace the navigation with the mobile navigation menu component in the design systems. The design is not a hamburger menu though. Future research can be done to see which menu design is better.
Mentioned by only 1 participant and not a usability issue
Will place sections in alphabetical order when revamping the Urgent hiring needs section
Will think about this again when discussing search enhancements
John will see if Cyber work groups (which are empty) are still necessary to keep
Minor findings
Summary
Tasks for the designer and content strategist to update the design prototype before development.
Make header, link, and button content clearer, and state who the job resources are for.
1
Move events below job resources.
2
Reinspect Urgent hiring needs and follow-up test the accordion design.
3
Acquire non-stock (looking) images.
4
Minfographicerge introductory and ‘Create profile’ button with main banner.
5
Remove similar links.
6
Optimize navigation menu for mobile.
7
Next steps
Homepage redesign
Thank you for your time.
Any questions?
February 15, 2023
Appendix
Are you familiar with the job searching on USAJOBS?
Pre-test questionnaire 01
No
Yes
Appendix
Have you looked for a job through USAJOBS in the last year?
Pre-test questionnaire 02
No
Yes
Appendix
Appendix
Homepage redesign
Appendix
Homepage redesign
Homepage task difficulty averages
Easy
Difficult
4.3
4.7
4.4
Appendix
Participants who noticed the ‘Career spotlight’ section
4.3
4.7
4.4
Did not notice
Did notice
Appendix
Participants who noticed the ‘Hiring paths’ section
4.3
4.7
4.4
Did not notice
Did notice
Appendix
Likelihood to look for events on the homepage
4.3
4.7
4.4
Unlikely
Likely
Appendix
Appendix
Homepage redesign
Appendix