The Prosocial Ranking Challenge

Software for Multi-Platform Social Media Experiments

Jonathan Stray, UC Berkeley

​

Joint work with

Kylan Rutherford, Ceren Budak, Julia Kamin, Mateusz Stalinski, George Beknazar-Yuzbashev, Ian Baker

​

2024-9-18

​

• International peer-reviewed competition produced 8 social media algorithms.
• Testing conflict, informedness, well-being and engagement outcomes.
• Custom browser extension changes what people see on Facebook, X, and Reddit August-December 2024
• Recruited ~7000 US social media users

Custom Browser Extension

• X, Facebook, and Reddit
• Re-order, delete, and/or add content
• Desktop only, but mobile support could be added (observational only)

What a Ranker Can Do

A

B

C

D

Re-order

D

A

C

B

A

B

C

D

Remove

A

Q

B

C

Add

​

Can add any public post on platform

D

In-feed Surveys

(on all platforms)

Normal Platform Operation

Platform

Web�App

User

HTML

DOM

Internet

Typical Research Extension: Editing HTML

Platform

Web�App

User

HTML

DOM

New

Content

Internet

Extension

A Better Approach: Intercepting API Requests

Platform

Web�App

User

New

Content

Ranking

Server

Extension

Platform

Web�App

User

DOM

Server

Platform Module

Experimental Intervention 1

Experimental Intervention N

Created by external experimenters

Core infrastructure maintained by project

and contributors

Browser Extension

Intercepts requests for content

Changes UI on screen

…

Science Data

Browser

Extension

Study

Running today

Platform

Mobile�App

User

Server

App Module

Experimental Intervention 1

Experimental Intervention N

Created by external experimenters

Core infrastructure maintained by project

and contributors

Study App

(installed by user)

Intercepts requests for content

…

Science Data

Mobile

Intervention

Study

​

​

Now developing prototype.

Android only

Platform

Mobile�App

User

Server

OCR and data reconstruction

​

Experimental Intervention

App Module

Executed by external experimenters

Core infrastructure maintained by project

and contributors

Study App

(installed by user)

Science Data

Mobile

Observation

Study

Screen

Screen Capture

Demonstrated.

Possible on iOS, Android

Ranking Server: Privacy, Security, and Performance

LLM

Content analysis

here

Total latency

< 500ms

Can add any public post

Write-only prevents data exfiltration

Hashing and Encryption to Protect Privacy

• Data breach will not reveal PII
• Engineering and admin team can’t see anything sensitive
• Separate ID→email table

Key Features of PRC Software

​

​

• Built specifically for social media – basic abstraction is a list of posts.
• Modular with support for X, Facebook, and Reddit
• Records posts seen, engagements, posts created, etc.
• Intervenes in platform API calls, so it can robustly scrape and change content (doesn’t depend on fragile HTML)
• Support for in-feed surveys, plus Qualtrics survey notifications
• Powerful interventions, delivered quickly: high performance, low-latency parallel LLM-based content analysis
• Designed for privacy and security first

​

Future Development

• Additional Platforms via modular design
• AI for ranking visual content, not just text
• Mobile support (some interventional, some observational)

Questions? Want to use it for your research?

• Contact jonathanstray@berkeley.edu
• Not publicly available due to abuse potential, but freely available to researchers