Margherita Rau

User Experience Researcher

​

Agenda

Section 1

Section 2

Section 3

Section 4

​

​

My Story

How I got into research

My Research Approach

Deep Dive: Research Projects

​

Section 1- My story

Section 2- How I got into research

Thesis: A Comparison of the Effectiveness of Cognitive Behavioral and Multisystemic Therapy in Modifying Antisocial Behaviors in Children and Adolescents

Teachers College, Columbia University

2019-2021

Confidential + Proprietary

The Challenge

• Situation of what’s going on- Due to conduct disorders being the hardest to treat, a lot of children & adolescents end up in the criminal justice system.
• How many people are affected: 6-16% of boys & 2-9% of girls.
• Why this study matters- To avoid future adults facing jail time due to untreated mental illness.

Confidential + Proprietary

What I did: Prepared with a Literature Review

​

• Research Questions
• Is Cognitive Behavior or Multisystemic Therapy more effective in treating antisocial traits in youth?
• Preparation at Columbia
• Approved proposal by Dept Chair
• Identified sponsor
• Developed a research plan
• Detailed bibliography

​

Confidential + Proprietary

What I did: Foundational Exploratory Analysis

​

• 4 month study
• Methodology
• Literature Reviews (prep)
• Competitive Analysis
• Why methods
• Synthesize prior research (not a lot of info on it yet)
• Larger sample = reliability
• Covid limitations
• Bureaucratic limitations =access to Kids (legal) = speed

Confidential + Proprietary

Research Findings

​

• Synthesis (Qualitative Analysis)
• Thematic coding by therapy type
• Reported quantitative findings (p-value, regression, ANOVA)
• Impact
• Recommendation: There is a lack of research on the cost-benefits of CBT and MST when used to treat ODD/CD in children and adolescents.
• Adaptations may be required to meet the specific needs of youth from diverse racial and ethnic backgrounds.

Confidential + Proprietary

My values

Social Equity

Empathy

&

Understanding

Technology Innovation

Confidential + Proprietary

I've learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel.

​

-Maya Angelou

​

“

”

Confidential + Proprietary

Section 3- My research process

What is my research process

1. Understand Research Need

Meet 1:1 with stakeholders to understand project background & priority level

​

Refine research question based on stakeholder goals

​

Clarify resource constraints

2. Study Prior Research

Discovery any prior research as to not reinvent the wheel

​

Conduct quantitative analysis understand magnitude

​

Create study plan

3. Recruit participants

Inclusive design

(regions, tenure, gender, ethnicity)

​

Collaborate with UXR Coordinator (adjust study plan if needed)

4. Collect data

Pilot test for surveys, usability test, focus group

​

Assess effectiveness of study plan with senior UXR and partners

​

Record sessions

5. Clean, analyze, synthesize data

6. Create and share report

Establish actionable recommendations given stakeholder goals & constraints

​

Present draft to close collaborators for review

​

Deliver report to broader network

�

​

Organized notes from study into detailed and high-level spreadsheets (i.e. inter-rater reliability if other UXR)

​

Conducted thematic coding, created user journey map

​

Drew statistics from themes

How do I select the appropriate research method?

​

• Priorities
• Risks/Assumptions
• Questions
• Deadlines

​

Stakeholder Interviews

• Budget
• Time
• Ethics
• Legal

​

Clarify limitations

How do I select the appropriate research method?

​

Goals:

• What tasks do the users need to accomplish?
• What is the design trying to achieve?
• How will user-experience improvements help?

​

Signals:

• What actions would indicate the goal had been met?
• How might success or failure in the goals actually manifest itself in user behavior or attitudes?

​

Metrics:

• How do we interpret these signals in a meaningful way?

https://clevertap.com/blog/google-heart-framework/

How do I select the appropriate research method?

​

Graph by Nielsen Norman Group, 2019

Additional Frameworks

​

Google Analytics

Typeform

Google Data Studio

Section 4- Deep Dive: Research Projects

​

Where did I make the strongest Impact

Project 1

Skytale Finance

(2022)

Project 2

New York Criminal Justice program (CJA) (2021-22)

​

​

UXR Project #1

Skytale Finance

https://docs.google.com/forms/d/e/1FAIpQLSfQqEn6wYaVK6sVOHbIZ_y3SRz1z183U9Z0USOZ6MlKDJxliQ/viewform

How did I grow as a researcher?

1. Learned the complexity of working in a startup environment.
1. Wearing many different hats
2. Not always organized

​

• Learned how to adapt research needs and scope quickly as the startup changed their projects and directions.

​

• Worked cross functionally with engineers, business development, marketing, and the CEO.
• Understood everyone’s perspectives and developed an holistic view

​

​

https://skytale.finance/

New Website Design

UXR Project #2

Supervised Release Program (CJA)

Public example of a criminal record

The Challenge: Increase client’s quality of life

​

​

​

​

Key UXR Questions

• What are the patterns of behavior reflected in their criminal record?
• What factors contributed to their life choices?
• Which partners would the client benefit from?
• What is the condition of their mental health? Are they currently impaired by substances? What crimes have the committed?
• Who are the key stakeholders in the client’s case? How do we best partner?

​

Process

• Timeline varies (1 day - several months)
• Collaborators: attorneys, mental health professionals, colleagues, client’s families, program directors & associates.

​

​

​

In New York, Rikers jail is overcrowded and living conditions are considered inhumane. Due to the 2019 NY Bail Reform, CJA received an increase in the referred criminal cases . As a result, the agency wanted to prioritize how we might be able to design a transformational space for clients to improve their life while awaiting trial, rather than waiting for sentencing in inhumane living conditions.

Confidential + Proprietary

What is my research process

1. Understand Research Need

Meet 1:1 with stakeholders to understand client background

​

Refine research question based on client goals

​

Clarify legal constraints

2. Study Prior Research

Discovery any prior research (criminal records etc.)

​

​

Create research plan

3. Recruit participants

Participant is already recruited based on the court assignments

4. Collect data

Interview- Biopsychosocial screening (1-4 hrs)

​

​

Record session notes and Score mental health screenings

5. Clean, analyze, synthesize data

6. Create and share report

Establish actionable recommendations via clinical note given client goals

​

Present note draft to close collaborators (stakeholder) for review and approval for next steps

​

�

​

Organized notes from study into detailed and high-level database

​

Conducted thematic coding.

​

​

​

Step 1: Understand Research Need

​

• Objective 1: Meet 1:1 with stakeholders to understand client background
• Talk to attorney, mental health professionals, and others involved in case to discuss client’s case.
• Objective 2: Refine research question based on client goals
• Mental health support vs. life coaching vs. mental health + substance misuse help
• Based on clients’ needs and goals

​

• Objective 3: Clarify legal constraints
• Order of protection
• Sex offernders
• Breaking confidentiality under certain circumstances

​

​

Public example of a restraining order

Confidential + Proprietary

Step 2: Study Prior Data

​

• Bird’s-Eye View: What are the overall clients’ behavioral patterns?
• Criminal Report
• Court interview notes
• Restraining orders
• Complaint from the legal case

​

• Ants-Eye View: How do I best target these behavioral patterns?
• Targeting certain behavioral issues based on crimes committed
• Do background literature review to study what type of therapy/treatment might help the most depending on the case
• Research possible rehabilitation/detox places in case client needs it

​

Confidential + Proprietary

Step 2: Study Prior Data (Qualitative)

​

• Reviewed relevant qualitative, quantitative, or mixed research on mental health topics that affected client

​

• Researched different types of therapies that might help client

Confidential + Proprietary

Step 3-4: Recruit Participants

​

​

• Participants are already recruited based on the court assignments
• Methodology: Structured interview to screen clients’ needs
• 1-4 hours depending on participant’s urgency
• Record session notes and score mental health screening

ASSIST screening tool to screen for drug use- Page 1

Confidential + Proprietary

Step 5: Synthesized data

​

​

Organized notes from study into detailed and high-level database

Thematic coding

Confidential + Proprietary

Step 6: Report & Socialize

​

​

• Establish actionable recommendations via clinical note given client’s goals

​

• Present note draft to close collaborators (stakeholder) for review and approval for next steps

Fictional example of a clinical note

Confidential + Proprietary

How did I grow as a researcher?

1. Collaborate cross-functionally
• Understand the intricacy of approaching an issue from different perspectives

​

• Learned the complexity of developing a program strategy
• Deep-dive into organizational constraints
• Customized plan to client’s specific needs

​

• Recognized the limitations of non-profit research
• Slow environment
• Often non-efficient
• Disorganized

​

Confidential + Proprietary

Thank You

UXR Project #3

Culture, Stigma, and Psychosis Lab (CSP)

The Challenge: Appropriately scoring interviews used to evaluate psychosis conditions in youth

Psychosis is a mental illness combined of two main factors that interact with each other: genetics and environmental phenomenons. Some people go on to fully develop psychosis and schizophrenia. We want to teach youth who are risk of psychosis ways to prevent the mental illness from worsening.

​

​

​

​

Key questions

• Do our participants understand the connection between genetics and the diagnosis of psychosis?
• What broader genetical concepts do participants understand?
• How can we code qualitative concepts into quantitative scoring techniques to aid the design of the website?

Project scope

• To design an educational website for youth experiencing psychosis
• To prevent psychosis as adults
• it is important to understand how those who are considered to be at clinical high-risk for psychosis (CHR) perceive possible benefits of genetic testing to inform future stakeholder education efforts.

​

What was my research process

1. Understand Research Need

Aligned w/ stakeholders on expectations and constraints

​

Collaborated with stakeholders to determine key user personas and main research questions

​

​

2. Study Prior Research

Synthesized prior quantitative & qualitative research

​

Deeply understood the mechanisms between genetics and psychosis

3. Recruit participants

Participants were clinical youth at risk of psychosis at the COPE Columbia University clinic

​

Recruited by clinical psychologists

​

4. Collect data

30-min structured interview with participants

​

Recorded sessions + detailed notes

​

​

​

​

​

​

5. Clean, analyze, synthesize data

Organized concepts from interview into several detailed and high-level spreadsheets

​

Conducted qualitative & quantitative scoring techniques

​

6. Create and share report

Co-designed an educational website

​

Collaborated with program engineers, designers, and others to develop digital content

�

​

Step 1-2: Understand research need & Study prior research

​

​

• Objective 1: Align w/ stakeholders on expectations and constraints
• Talk to the main principal investigator (lab professor) and PhD student about a time-frame, and confidentiality agreement
• Objective 2: Collaborate with stakeholders to determine key user personas and main research questions
• Prevention, education, and understanding of psychosis
• Objective 3: Study prior research
• Understand the gene-environment interaction

​

Step 3: Recruit Participants

​

• Participants (N=54) were recruited from the “Center for Prevention and Evaluation” (COPE), the prodromal clinic at Columbia University,

​

• Recruited by clinical psychologists at the hospital

​

​

Step 4-5: Collected, cleaned and synthesized data

​

​

• Methodology: Interviews (N=54, 30-mins)
• Clinicians interviewed participants to study if they understood the gene-environment interaction
• Broader idea of genetics and physical illness (blood pressure, diabetes)
• Asked to write a letter to a hypothetical friend to explain how to prevent an illness or psychosis
• Interviews were transcribed, consensus coded by graduate-level trained coders, and analyzed using deductive and inductive coding frameworks to assess the participants’ understanding of psychosis and gene interaction.

Step 4-5: Collected, cleaned and synthesized data

​

​

• Created a 1-4 scoring techniques to conceptualize interviews:
• 1: participant does not understand gene-environment interaction
• 2: participant understands gene-environment interaction for physician illness
• 3: participants understands gene-environment interaction for psychosis
• 4: participant understands and can explain how to prevent psychosis

​

​

Step 6: Create and share report

​

​

• Keywords, phrases, and concepts were individualized from interviews
• Placed into 1-4 thematic coding categories on a spreadsheet
• Quantitative concepts needed to create the website (easier for programmers)
• Co-designed and collaborated with program engineers, mental health professionals and graphic designers

​

​

​

​

​

​

*AutoTutor is an intelligent tutoring system developed by researchers to help learn critical thinking topics through tutorial dialogue in natural language.

First-author APA poster presentation based on coded concepts 3

How did I grow as a researcher?

• Collaborate cross-functionally
• Create the APA poster with my team

​

• Learned the complexity of strategic thematic coding
• Nuances in qualitative data harder to transcribe into quantitative data

​

• Recognized the limitations of academic research
• The environment moves slowly
• Difficult to make progress

​

Confidential + Proprietary