1 of 37

�Beyond the Buzzword: Demystifying "The Algorithm" and its Impact on Youth Mental Health�

July 26, 2023

Sophie Szew

Stanford Univeristy

2 of 37

My Introduction

Junior/coterm studying AMSTUD + CSRE at Stanford

Background in youth mental health policy + advocacy—MHYAF and SB 287/680

Intersection with storytelling, history of health inequity

3 of 37

What comes up for you when you think of “the algorithm” in relation to social media?

Discussion Question:

4 of 37

Overview

Bias in healthcare algorithms

Bias in social media algorithms

Result of AND contributes to systemic discrimination

Grass roots advocacy, especially among students/youth

Lead to real, quantifiable negative health outcomes

Can be traced back to eugenics or slavery medicine

Doctors may not have “racist intentions” but are discouraged from questioning a racist system

Formulas publicly available

Bias towards distributing harmful content to youth

Driven by understanding that time online = profit

Formulas not publicly available

5 of 37

History: Dr. Dorothea Roberts

Maternal Health: Dr. Darshali Vayas

Nephrology: Dr. Ameka Eneanya

Pulmonology: Dr. Nirav Bhakta

Roundtable Experts

6 of 37

Racialization of disease was used to justify slavery

Ex: higher lung capacity -> demanding physical labor

Systemic discrimination DOES lead to poor health outcomes, which justifies bias

Bias reflected and solidified through race-adjusted algorithms (lung capacity measurement, eGFR, radiation)

NASEM Roundtable Recap

Examining The History Consequences and Effects of Race-Based Clinical Algorithms on Health Equity A Workshop | National Academies

7 of 37

Bias

Poor health

outcomes

Discrimination

Medical Bias Confirmation Cycle

8 of 37

Applications in Key Calculations Discussed by NASEM

9 of 37

Applications in Key Calculations Discussed by NASEM

“Anthropoid pelvis” research -> VBAC Success Predictor ->Contributions to worse maternal health outcomes for BIPOC

10 of 37

Applications in Key Calculations Discussed by NASEM

“Anthropoid pelvis” research -> VBAC Success Predictor ->Contributions to worse maternal health outcomes for BIPOC

False beliefs about differences in muscle mass ->Race coefficient in eGFR ->Overestimation of kidney function and higher rates of negative kidney disease outcomes

11 of 37

Applications in Key Calculations Discussed by NASEM

“Anthropoid pelvis” research -> VBAC Success Predictor -> Contributions to worse maternal health outcomes for BIPOC

False beliefs about differences in muscle mass -> Race coefficient in eGFR -> Overestimation of kidney function and higher rates of negative kidney disease outcomes

Slavery medicine justifying physical labor with “lower lung capacity” myth -> race adjustments in pulmonary function tests -> lack of lung cancer screenings and transplant eligibility leading and poor lung health outcomes

12 of 37

Change

Advocacy/science

Empowerment

Combatting Bias Cycle

13 of 37

Advocacy and Progress

14 of 37

Advocacy and Progress

Doctors at Brigham and Women’s hospital published a paper challenging the use of race in VBAC success predictor -> Race correction dropped -> proved calculations can still work without race corrections, brought attention to systemic issue

15 of 37

Advocacy and Progress

Doctors at Brigham and Women’s hospital published a paper challenging the use of race in VBAC success predictor -> Race correction dropped -> proved calculations can still work without race corrections, brought attention to systemic issue

Students at Harvard med organized to remove race from eGFR calculations -> Race removed by NKF -> 28.1% of Black patients re-classified as having more severe disease

16 of 37

Advocacy and Progress

Doctors at Brigham and Women’s hospital published a paper challenging the use of race in VBAC success predictor -> Race correction dropped -> proved calculations can still work without race corrections, brought attention to systemic issue

Students at Harvard med organized to remove race from eGFR calculations -> Race removed by NKF -> 28.1% of Black patients re-classified as having more severe disease

Breathing Race into Medicine by Lundy Braun+ momentum -> American Thoracic Society removes race calculation-> greater emphasis on factors that really affect lung function: environment, stress + smoking, etc.

17 of 37

Takeaways

Algorithms are correctable and we need to grapple with their histories and implications

We cannot change systems we don’t fully understand

Race correction -> race consciousness

18 of 37

Definition According to Institute for Internet and Just Society

“Algorithms in social media platforms can be defined as technical means of sorting posts based on relevancy instead of publish time, in order to prioritize which content a user sees first according to the likelihood that they will actually engage with such content.”

19 of 37

Comparison

20 of 37

Comparison

21 of 37

What We Do Know

Internal research by Meta found Instagram’s algorithm was pushing harmful content onto teen’s explore pages

Teens exposed to body image and mental health content on tiktok every 39 seconds (Center for Countering Digital Hate)

New accounts exposed to suicide content within 2.6 min and ED content within 8 min

“Vulnerable” (loseweight) accounts 12x more likely to receive self-harm and suicide content

Surgeon general advisory

22 of 37

23 of 37

24 of 37

Why?

25 of 37

Ziggi Tyler Demonstration

Connection to biased algorithms in medicine: medical devices/measurements designed using “sample populations” of only white men (BMI and pulseox)
Lack of consideration for/capitalization on racial trauma

26 of 37

The Effects of Racial Trauma + Social Media on Mental Health

“psychologically damaging experiences of ongoing systemic racism are further exacerbated through the reoccurring circulation of videos and images”

-NAMI

“Machine learning algorithms ensure that the most grotesque, racially violent and “click worthy” content is competitively priced and easily found via keyword auctions and content monetization programs”

-Dr. Tanksley (CU Boulder)

27 of 37

Engagement

Harmful content

Profit

Social Media Algorithm Harm Cycle

28 of 37

Example

Content promoting ED/ weight loss -> Comparison loop/more time on app -> more profit for tiktok or meta

29 of 37

Change

Advocacy/Science

Empowerment

Combatting Bias Cycle

30 of 37

Example

CA age appropriate design code, SB 287/680 -> Onus off young people and onto platforms -> discourse, alleviation of moral panic

31 of 37

Policy

Age-appropriate design code
CA SB 680: “ A social media platform shall not use a design, algorithm, or feature that the platform knows, or by the exercise of reasonable care should have known, causes a child user to inflict harm on themselves or others or develop addiction”
KOSA

32 of 37

Surgeon General’s Advisory

Policymakers:

Strengthen protections

Develop health and safety standards

Support increased funding for future research

Ensure technology companies share data relevant to the health impact of their platforms

Require higher standard of data privacy

33 of 37

Surgeon General’s Advisory

Researchers:

Establish the impact of social media on youth mental health as a research priority and develop a shared research agenda

Rigorous evaluation of impact, including longitudinal studies

Role of demographics (ie: more research on intersection between marginalization and triggering content)

Develop standardized definitions and measurements

34 of 37

Attention on harms of social media must be accompanied by attention to benefits for most marginalized communities

BTWF study–nearly half (44%) of LGBTQ+ young people reported feeling "very safe" online, compared to just 9% for in-person space

35 of 37

The problem is that the world is unsafe for many

we must not lose site of this!
the very same historical inequities that cause health disparities ARE the reason we see online spaces looking the way they do

precedent of capitalizing on vulnerabilities
turning to care online when it is denied IRL

36 of 37

Final note on nuance

A lot of the coalition building to form youth lobbying/advocacy groups for SB 680 + KOSA happened on social media

Coalition building to fight race correction happened in med schools attended by the same people who practiced slavery medicine

37 of 37

Thank You��Questions and Discussion