1 of 29

Towards Crowdsourced Audits of Algorithmic Management Systems

Samantha Dalal

Phd Candidate

CU Boulder InfoSci

2 of 29

2

Was he free? Was he happy? The question is absurd:

Had anything been wrong, we should certainly have heard.

The Unknown Citizen

W. H Auden

I wanted to start today’s conversation with a poetry excerpt that has resonated with me since the first year of my PhD program.

[Was he free? Was he happy? The question is absurd:

Had anything been wrong, we should certainly have heard.]

I came across this poem in a first year Anthropology course when we were reading about the evolution of privacy in the U.S–the moral of the story being that in order to receive benefits from the state you have to become visible to the state.

This idea of visibility as integral to legitimacy is the cornerstone of of the work that I’ll be presenting today.

For workers to get the support they need, they must be visible to the platform, to the organization, and to the state. However, the process of becoming visible is a complicated one, especially for workers that labor within algorithmically managed systems. How do workers become visible? And under what terms?

3 of 29

3

ride-hail drivers

couriers

sex workers

influencers

freelancers

journalists

4 of 29

4

Algorithmic management creates visual asymmetries

5 of 29

Key Takeaway

Algorithmic management systems need constant monitoring to be held accountable**

5

6 of 29

Monitoring tends to be top down

Platform Workers Directive

Third-party professional audits (PWC, Deloitte)

6

7 of 29

7

8 of 29

8

“Data Rhetoric”

9 of 29

9

What does countervisuality of an algorithm look like?

10 of 29

10

User-driven auditing

(DeVos et al, 2022)

Division of labor in participatory audits

(Li et al, 2022)

Participation vs. scale in AI

(Eshan et al, 2022)

11 of 29

11

“I was basically consumed by just… gathering all the data”

�Willy Solis on NPR’s Radiolab

One example of what countervisuality of an algorithm starts with a gig worker named Willy.

Willy worked for Shipt, a grocery delivery company, when they started switched some workers over from a transparent to black box payment system
For some workers, they were able to see exactly what factors contributed to their gross pay but for others they were put into a black box system that calculated “effort” and paid workers according to these calculations.
Willy started investigating what was going on, collecting screenshots, crunching numbers, talking with other workers…but this took time!! And he didn’t have the toolkit needed to do the analyses

Calacci, D. and Pentland, A. (Sandy) (2022) ‘‪Bargaining With The Black-Box: Designing and Deploying Worker-Centric Tools to Audit Algorithmic Management‬’, in Proc. ACM Hum.-Comput. Interact. CSCW2, p. 24. Available at: https://doi.org/10.1145/3555529.

12 of 29

12

Documenting algorithmic behavior requires significant amounts of labor

13 of 29

13

Building infrastructure to support worker-led inquiry

14 of 29

14

Vision:

Build FairFare, an infrastructure for collaborative algorithmic inquiry with workers to inform the establishment and monitoring of labor standards

Steps:

Develop data collection tool with workers and organizers
Collaborate on specific questions to answer using data shared by workers in a pilot; focusing on a specific hot inquiry
Build on this process to enable more general inquiry in the future with less expert involvement

15 of 29

15

Drivers know their own income and have many sophisticated ways of tracking their data

While drivers share anecdotal stories about earnings & take rates…

Changing laws to improve working conditions or platform design requires lots of structured data and systematic analysis to provide quantitative evidence of drivers’ lived experiences

We built FairFare, a tool for drivers to systematically collect data about their working conditions at scale and provide in-depth analysis for labor organizing

Context

Goal

Approach

16 of 29

16

17 of 29

17

Take Rate: The percentage of the total fare paid by the rider (before tips) that Uber takes

Airport Ride: trips that start or end at an address with the word “airport” in it

Surge Pricing Ride: trips that happened during a surge condition

$38

$62

Example:

Customer pays: $100

Uber takes: $38

Driver gets: $62

Take rate: 38%

18 of 29

18

13 drivers,

Uber only

10,141 rides total

1,421 rides during surge pricing

1,010 rides to/from the airport

70 months period. Rides from Dec ‘17 to Oct ‘23

19 of 29

19

Distribution of average take rates for airport vs non-airport rides

20 of 29

20

On average, Uber takes 44% on airport trip vs 36% on non-airport trips, a 22% increase in take.

On average, Uber takes 53% on surge pricing trips vs 45% on non-surge trips, a 17% increase in take.

On average, Uber takes more than one third of the total trip fare.

38% avg. take on all trips

22% higher take on airport trips

17% higher take on surge trips

21 of 29

21

LOST

When we reported our findings back to our community collaborators they told us they were surprised - the take rate was lower than they thought it would be. Interestingly, the CEO of Uber recently was interviewed by Kara Swisher on a podcast and he claimed that the average take rate was 15%...much lower than what we found in our study and seemingly much much lower than what Sandra and Brian thought was the actual take rate…

Sandra, Brian, and the rest of CIDU are getting ready for the next legislative sessions here in Colorado which will begin in the winter. They hope to bring data that their drivers are generating with FairFare to legislators to support the anecdotal evidence that they’ve gathered from drivers about their experiences.

When they tried to get a gig work transparency bill, SB23-098, passed last sessions, they were outflanked by platform lobbyists at the last minute. They hope that this time around they might be better prepared and armed with the comparable data capabilities as companies. In other words, they hope that they will be able to push back against one-sided platform narratives by furnishing rigorous, quantitative evidence of their experiences.

22 of 29

22

Won

Major victories in 2024 Colorado State legislative session

23 of 29

23

Congressional Testimony

Bill drafts

PR campaigns

Bill negotiations

Lobbyists

Think-tanks

Lawyers

Special interest groups

The audit data we collected & gave to CIDU played a relatively minor role in negotiations for the bills. During this time I did four months worth of field work as a part time organizer with CIDU, attending their union meetings, strategizing bill language, and attending testimony sessions.

Throughout these observations, data about take rate played a relatively minor role in discussions…Access to quantitative data served as a legitimizing mechanism for organizers’ arguments but did not take center stage in negotiations. Instead, personal testimony from drivers in front of state congress members served as the primary source of evidence about harms resulting from algorithmic management systems.

Additionally, throughout this process I saw the behind-the-scenes of the legislative process; who actually writes the bill, the extent to which corporate interests are involved in shaping bill language, and where in the bill writing process access to “counter-data” proves useful

24 of 29

24

Collaborate on research questions

Crowdsourced, worker-led data collection

Creating a “counter-visual” to Uber’s systems & stats

Contributions & limitations

25 of 29

25

Researcher-oriented process: limits of ‘expert’ intermediaries

Not yet engaging with ideas of worker voice

How do we know if we’re asking the most important questions for workers?

For policy or organizing?

How do we move towards worker-led and owned infrastructure?

Contributions & limitations

26 of 29

Future Directions

Investigating algorithmic wage discrimination

Do different workers get paid different amounts for the same work?

Getting to know the community through trace data

How can we leverage online communities to learn about worker concerns?

26

Some future directions of this work includes

Investigating algorithmic wage discrimination. Do different workers get paid different amounts for the same work? What criteria do we use to distinguish a “different enough” worker? What about a similar enough task? How much data do we need? What would constitute the evidence needed to make a strong legal case?
Leveraging online gathering sites, like subreddits, to learn about worker concerns (in tandem with actual outreach to local organizations). Auditing from below, crowdsourced audits require knowledge from workers themselves - how do we find them and learn about their concerns? In a traditional factory it may be easier for auditors to find workers to talk to because they need to be physically present in a given location to do their work but what about workers in “algorithmic factories” - how do we find them? Online forums combined with language processing techniques may help us synthesize and formulate concerns that the community has…ofc this can be fraught and we’re very aware of the limitations of digital ethnography and LLMs.

27 of 29

Supporting Continuous Monitoring of Algorithmic Systems

27

Data provided by employers

Professional analytical resources

Paths for communication

At the beginning of this talk I stressed the importance of continually monitoring algorithmic systems to hold them accountable. Throughout case study with FairFare we’ve identified some key resources that are needed to make this type of monitoring possible.

We need:

Access to data; workers need this data but it should be furnished by employers themselves

Resources to analyze data; to have insights into what’s happening you need some analytical capabilities → who should do this analysis?

Need a way to communicate findings from the workplace to institutions with influence → who should hear about these findings? What should they be mandated to do with these findings?

Where can law and policy help us figure out how we can get these things, what these things could look like, and what agencies should be responsible for them?

28 of 29

28

How can work be

fulfilling,

dignified, and secure?

Regulatory Structures

Workplace Systems & Tools

03

01

02

Organizational Structures

29 of 29

Thank you!

29

Dana Calacci

Penn State University

Samantha Dalal

CU Boulder

Andrés Monroy-Hernández

Princeton University

Danny Spitzberg

Turing Basin Labs�Georgia Tech

Check us out: wao.cs.princeton.edu