User Experience Research Education Initiative PM Study

​

Jia Cheong, UX Specialist

​

Table of Contents

• Study Overview
• Executive Summary
• Key Findings and Recommendations
• Next Steps
• Appendix and Resources

Slides

Play Report Readout Recording

2

03

07

11

26

28

Study Overview

Background

1. Understand how Product Managers are accessing the UXR material on SharePoint.

​

2. Understand how Product Managers are digesting the information (video, documentation, pain points etc.).

​

3. Analyze the mental model of a Product Manager when it comes to conducting their own research.

​

​

SharePoint site created by the UXR Education Initiative team included:

​

• The Landing Page
• Full and Guided Page
• Full UXR Page
• Guided UXR Page

​

Research Goals & Objectives

The UX Research team created a SharePoint site to provide information, guidance, and awareness of the UXR team. The team wanted to figure out what sorts of information Product Managers used to help improve their User Research endeavors and whether the information provided on the SharePoint site can be improved to best serve them.

4

Stimulus

Methodology and Recruitment

All interviews were conducted remotely by Jia Cheong via Microsoft Teams and lasted approximately 60 minutes. We spoke with 8 Product Managers across various SBU’s and across different seniority levels.*

5

Participant Details

• We spoke with at least one PM from each SBU.
• Most of the job titles were mid-level PMs, but also included some junior and some senior PMs.

*Please keep in mind that qualitative research is based on non-projectable data and is directional information due to limited sample sizes.

Executive Summary

Executive Summary

7

Landing Page:

• There was limited awareness of the SharePoint site among PMs.
• Long videos were not well received and could be easily skipped.
• In general, PMs would value easier access to samples of previous work on the landing page.
• The visual similarities between the UXD and UXR pages created confusion.

Full vs. Guided Page:

• The first step in the UXR process (contact the UXBP) was easy to miss.
• Most acronyms on the site were unclear.
• The terms "Guided" and “Full” UXR were confusing even after reading the description.

​

Communication Channels:

• Email/Outlook was the most preferred communication channel.
• Microsoft Teams was the second most preferred channel.

Executive Summary continued

8

Full UXR Page:

• The Full UXR step-by-step process visual was well received due to its succinctness and layout.

​

PM approach to UXR preparation:

• PM preparation for client-facing UX Research usually involves both internal and external resources.
• Senior PMs believed they had already acquired enough information through experience and training on how to conduct client-facing research.

​

​

Guided UXR Page:

• The introduction paragraph did not always suggest assistance from the UXR team.
• Access to reports was highly valuable, but how to access them was unclear.
• Guided UXR reference PDFs were well received, but their placement and sequence was not obvious.
• Education on various methodologies and how to use them appropriately was valuable.
• The value or purpose of uploading Guided UXR reports to The Vault was not clear.

Key Findings and Recommendations

Objective 1 : ��Understand how Product Managers are accessing the UXR material on SharePoint��

There is significant lack of awareness of the UXR SharePoint site among PMs��

• More than half of the PMs interviewed, were familiar with the SharePoint site, however, a significant number of them were completely unfamiliar with the site.
• Of the 5 PMs who had found the SharePoint site prior to the study, 2 were referred to the SharePoint site by an Insighter.

11

“Yes, because David shared it with me.”

 - P8, Research

“I didn't know about this site. I knew the site existed for the Design portion of UX, not the research portion. I think there's a lot of really good information here..”

 - P3, Platform

Recommendation

• Promote the UXR SharePoint site in different ways that we are currently doing.

“Yes, I’ve been to this site a few times, but I don’t think I’ve been to this page [Landing Page]." – P4, Research

Product Managers want easier access to research resources (reference guides) and past study results��

• On the Landing Page, more than half of the users stated that there could be more content that could help with navigating the SharePoint site better by promoting work or marketing the Site.
• Of the 6 PMs who agreed that there could be more on the landing page, 4 of them said they wanted to see Reports, a case study debrief for reference, or ‘who was involved in the projects.’ 
• 2 of the PMs suggested more promotion of the SharePoint site.

12

“I think it's a good highlight. the view reports, this is the first place that we've gotten the view reports, and I'd love for that to be on the main landing page or the full/guided page. I feel like people want to see this.". - P3, Platform

“ I would like to see a case study; an example of a report debrief for reference.’

- P6, Analytics

Recommendations

• Provide and promote documents such as PDFs in the Guided UXR, View Reports, or any useful documentation on the landing page.
• OR have users land on a page with an appendix of resources.

Product Managers were unaware of the differences between the UXR and UXD SharePoint site

• Visually, the UXR and UXD SharePoint pages were very similar in design and structure, and therefore, created confusion.

13

UXD SharePoint site

UXR SharePoint site

In some cases, PMs believed that the top navigation tabs were the logical pathway to learn more about UXR

• Some PMs did not click on the CTA titled ‘Learn more about Full and Guided UXR,’ and instead clicked on the ‘For PMs’ tab on the navigation bar.
• After clicking on the ‘For PMs’ tab and clicking on to the next page, these PMs were unaware that they had landed on the UXD SharePoint page.

14

‘I would go to 'For PMs'. – P7, Research

​

UXD SharePoint site

UXR SharePoint site

Recommendations

• Consider increasing the visibility and highlighting the CTAs further.

Sample of selectable blocks approach

15

Recommendations

• If there are other areas of focus, the team can create a page with selectable blocks of content such as: Resources, Learn More about Guided and Full, View Reports, etc.

This can say ‘Click here to view Reports of work that the UXR team did!’

An example of what the landing page could look like

Objective 2 : ��Understand how Product Managers are digesting the information (video, documentation, pain points etc.).

Product Managers liked videos as a resource, but they need to be concise���

• Videos were generally a valuable resource for PMs, but they should be only a few minutes in length.
• None of the PMs would invest time to watch the one-hour long introduction video on the Landing Page of the SharePoint site, however half of the PMs stated they would watch short videos if they were about research methodologies and research tools.
• The video in the Guided Research section, was something more than half of the PMs agreed they would watch. However, the video link was broken during testing.

17

“There's different types of UXR, there is nothing written here that is helpful for determining what is best based on the product life cycle." - P4, Research

"I'd watch a 10-minute video, but an hour video, not unless I needed specific information within that video, and I had no way to extract it quicker." – P2, Platform

Recommendation

• Create short videos of less than 10 minutes in duration that provide PMs with information on research methodologies and research tools.

Most PMs agreed that the Guided UXR documents were useful in their research���

• The Guided Research reference PDFs provided in Guided Research were considered a valuable resource, especially for junior PMs.
• Most PMs (6/8) mentioned future usage of some or all the Guided Research reference PDFs.
• Few PMs (2/8) did not like having to open the Guided Research reference PDFs in a separate window.
• There was a tendency among some PMs to not follow a sequential order when viewing the PDFs: ‘When preparing for Conversations’, ‘When Moderating Conversations’, and ‘When Reporting the Findings.’

18

‘I would highly value the Moderating PDFs.’

- P6, Analytics

‘I wish there was a way to find these documents instead of scrolling all the way down.’

 - P2, Platform

‘Under When Moderating conversations: it looks like it was a little confusing at first cause it’s under the same column. (Realized that it’s under the same color).’ – P3, Platform

Recommendation

• Consider showing the full PDF documents on their own page and above the fold and avoid stacking the reference docs on fielding research.
• The Guided Research reference PDF documents could be further promoted or highlighted on the Guided SharePoint site or on other pages.

​

‘Guided’ and ‘Full’ are terms that were confusing or misleading to PMs���

• The term ’Guided’ and ‘Full’ UXR were unclear to PMs until they read both pages thoroughly.
• Most PMs (6/8)  thought Guided meant they did the work without assistance (not consultative) and was therefore misleading.
• In ‘The Process’ section, several PMs (3) did not know the difference between a background call and an intake call.

19

‘Confused, is guided process something that your team is involved in.’

- P8, Research

‘Feels like what the actual words guided meant? What does that mean? It meant that guided is an advisory role.’

- P2, Platform

Recommendation

• Clearly explain the difference between Guided and Full UXR.
• Consider including a chart or similar device that clearly compares both processes to reduce confusion and set expectations.

​

• A Venn Diagram could be a simple and effective visual to establish the difference between both processes.

PMs expressed interest in learning and understanding more about research methodologies���

• Exposure to various research methodologies and how to use them was important to PMs.
• Half of the users wanted information on other research methodologies, such as survey creation using User ZoomGo (unmoderated study).

20

“More videos on how to do the research yourself.”

 - P8, Research

“A few sub bullets there. Something just saying training for PD and UX on how to use. I think it's like user zoom, go or training on how to write an effective survey. Something to find good people to talk ….. So you can help me with this news former research, so I don't have to do a full study every time.” - P1, Wealth

Recommendation

• Provide more UXR methodology definition beyond client-facing user interviews such as: Surveys, unmoderated studies, A/B tests, card sorts, etc. This could be in the form of short videos (possibly a Loom), credible sources and links, or short PDFs like the ones that exist on the ‘Guided Research’ page. 

The purpose of The Vault was not clear to PMs���

• The purpose of The Vault as a repository that everyone can access was unclear, and only half of the PMs mentioned it.
• Since PMs are accustomed to entering client/user engagement notes on Salesforce, some did not feel incentivized to upload their reports to The Vault.
• A few PMs stated that they liked the simplicity of the reporting template provided.

21

“The Vault should be called the void because no one can find it.”

- P8, Research

“Research should be widely accessible, and I don’t understand why things need to go the vault.”

- P1, Wealth

Recommendation

• Provide more context as to why it is important to upload information to The Vault.
• Promote The Vault as a useful research insights repository open to anyone across the organization and look at ways to enable keyword search and data automation within the repository.

The starting point of the UXR process was not clear ���

• More than half the PMs did not see ‘Contact your UXBP’, as the first step of the process.
• Most PMs did not know what the acronym UXBP meant.
• Some PMs did not understand why they needed to contact a UXBP first, since it was not explained on the page.
• The CTA two buttons at the bottom distracted some PMs, and they completely missed ‘Contact your UXBP.’

​

​

22

“The text on top doesn’t seem critical [contact UXBP] compared to the CTA buttons.”

- P1, Wealth

“What if Justin or Bill are unavailable, who do I contact?.”

- P8, Research

Mockup-Concept: Placing the blue CTA button on the top of the page in lieu of the current text but aligning it in the center can help to center the focus on what the first step is.

​

23

Recommendation

• Make the ‘You must first contact your UX Business Partner’ message more conspicuous on the page by creating a similar looking CTA style.’ This is backed up by Gestalt’s Principle of Similarity: When we see objects that look similar in color, shape, or form, we tend to group them together. 
• Have an explanation as to why they must speak to a UXBP even if PMs already believe their project is a priority.
• Spell out the acronyms such as UXBP

Office hours should be showcased in multiple pages across the site����

• Most PMs easily missed Office Hours information, but overall, considered this time slot useful.
• All the PMs eventually found out that Office Hours were not a place to initiate a research project. This was not immediately clear.
• Some PMs noticed the office hours form that was embedded in the site, and overall, it was seen as useful.

​

24

‘The UXBP link dragged me all the way down and I missed the Office Hours.”

- P6, Analytics

“This is a form for Office Hours. that's good. I like it...it's more of a Q&A if you have something to discuss without any formal meeting..” -P7, Research

Recommendations

• Include Office Hours on all pages, including the landing page and make the form more noticeable.

Objective 3 : ��Analyze the mental model of a Product Manager when it comes to conducting their own research.

Most PMs do preliminary research before conducting client-facing research���

• Many participants defined pre-study research (before conducting conversations) as researching the participant, e.g., LinkedIn, usage data, Salesforce meeting notes, and meetings with sales team members.
• Internal Resources: Pendo guides, speaking with the sales team/consultants. 
• External Sources: Google search, LinkedIn and LinkedIn Learning, videos, trainings, and blog posts, Product Institute.
• All the PMs who did not prepare for research in advance or did minimal preparation were at the Senior PM level or above.

​

​

26

“I would use Google and find some credible sources.”

 - P8, Research

“I normally get feedback from sales as research prep. At my previous company we had documents [on Guided page] to help us prepare for client research.”

- P7, Research

Recommendation

• Include a larger variety of preparation material for PMs that could include trainings/workshops, blog posts, videos (DIY preparation), etc.

The best communication channels for PMs were Email and Teams���

• More than half of PMs preferred email/Outlook as their main communication channel for notifications
• Teams was the second-best communication channel.
• Some other communication channels included: Yammer, Kaleo, Jira, and RPD. 

​

​

27

“Email is the first one because it combines everything, and I have a lot of alerts.”

 - P7, Research

"I generally use Outlook and Teams for receiving information."

 - P3, Platform

Recommendation

• When sending news and notifications about UXR related updates to PMs, the team should consider sending emails through Microsoft Outlook first.

Recommendations Summary

28

Recommendations Summary

29

Next Steps

31

31

Next Steps

1. Gather data and try to implement findings and recommendations throughout the SharePoint site.
2. Create more content for UXR education for PMs.
3. Consider the creation of additional pages to minimize deep scrolling and to surface the most important pieces of information above the fold
4. Run tests and get feedback from PMs on future content surveys.

​

​

Appendix and Resources

33

33

Appendix and Resources

​

1. The Law of Similarity

​

​

Methodology: User Interview

In each interview:

We conducted a total of 8 Interviews with internal PMs.

First, participants:

• Discussed how they best prepared, for their client-facing research and what kind of information format/medium they used to best digest information.

Next, they:

• Talked about the various communication channels they used internally within the company.
• Spoke with users about what communication channels were most effective for them.

Finally, they:

• Explored and navigated through various points within the UXR SharePoint site while providing feedback.

34