Buyer Uncertainty about Seller Capacity

John Horton

NYU Stern

​

​

“The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design.”

​

― Friedrich von Hayek

But market design isn’t a choice...

Golden Age for market design

• Huge growth in marketplace businesses
• Most have sophisticated tools for running experiments and collecting data
• Platforms have lots of “levers” to pull
• e.g., policies, features, price structure and levels, marketplace composition over time,
• However, most lack match setting power
• There is a (too small, IMHO) MD literature in this decentralized vein, e.g., work by Coles

​

​

Background

• Platform is an online labor market for tasks that can be done remotely
• Programming, graphics design, data entry, etc.
• See Aggrawal, Horton, Lacetera & Lyons (2014) for details about the marketplace
• Matching is decentralized and “works” more or less like a traditional labor market

​

Two channels for applications

Employer posts an opening

Applicants apply

Firm screens applicant pool;

potentially forms a match

Employer recruits

Workers that “accept” the invitation apply and join the pool of “organic” applicants.

Many vacancies go unfilled; (as in traditional markets)

​

“Ball and Urn” Matching Frictions

I = 1

I = 3

I = 2

I = 0

I = 2

Under

Subscribed

Over

Subscribed

See Petrongolo & Pissarides (2001) for overview; Albrecht & Gautier (2003)

​

Invitations received per week, by hours worked that week

“If you want something done, ask a busy person.”

-Benjamin Franklin

Is this actually good advice?

(since we are at Penn)

Response rate (accept or decline) to invitations, by hours worked

How should we think about this pattern?

• It does not imply a market failure
• Busy workers might be so good that their superiority compensates for their lower response rate
• But it does not imply efficiency either
• It could also be a market in which some sellers are over-pursued compared to what would be efficient
• The oDesk business consensus is that it is a problem
• See Paul Krugman’s advice to listen to the gentiles

Related work

• “Traditional” congestion
• E.g., Roth and Xing, Kagel and Roth
• Quit similar to Fradkin (2014), who was looking at inquiries in the Airbnb context
• Very related work:
• Managing Congestion in Dynamic Matching Markets by Arnosti, Johari and Kanoria (2014)
• Johari & co-authors were motivated by the same phenomena I explore in this paper

Status quo is likely not efficient

• Conditioning on availability is difficult
• Sellers have little incentive to inform parties about their availability
• Outside offers are valuable (for negotiation)
• Nearly free disposal on offers
• Availability “fiction” permits screening (Edelman & Luca, 2014)
• Humans search in a predictable way that would tend to exacerbate the problem

Employer search interface

Many possible causal stories for the observed pattern

Worker

Productivity

Lots of invitations

“Choosy” about invitations

Works lots of hours

Rejects lots of invitations

Supply-constrained

Empirical strategy:

Create a worker hours/invitations panel and include worker-specific fixed effects

Worker

Productivity

Lots of invitations

“Choosy” about invitations

Works lots of hours

Rejects lots of invitations

Supply-constrained

If we still see strong relationships between hours worked, invitation counts and rejections, the argument for a causal relationship is bolstered.

Pooled OLS Estimate

More hours worked associated with lower acceptance rate, but not a large effect.

More invitations associated with lower acceptance rates.

Worker-specific FE estimate

Acceptance rate is decreasing in the number of hours worked and the number of other invitations received, even with the inclusion of worker-specific fixed effects.

Interpretation

• Obviously not definitively causal, but…
• The number of invitations received in a period of time is strongly predictive of acceptance rate even with worker-fixed effects; makes purely selection stories implausible.
• This suggests that “moving around” invitations would affect the rate matches are formed

“First, do no harm…”

• We could move the application “graph” to something more equitable
• Recall that, as the platform, we control visibility in search, recommendation frequency etc.
• Research Question: Is doing so likely to “improve” the marketplace?

I = 1

I = 3

I = 2

I = 0

I = 2

I = 2

I = 1

A step back: is inequality in invitations actually a problem?

• Assume you tentatively accept my “more invitations → more stock out” causal story
• Is this actually a problem?
• How would we know?

Will an invitation rejection actually matter?

Quantifying the effects of rejections

• Take a sample of US PST time zone employers that (a) sent a single recruiting invitation (b) within the first hour of posting a job (c) for a publicly available job (d) to a worker they have never interacted with before
• How does that worker’s response (accept or reject) affect the probability that a match is formed?

An invited worker accepting an invitation has a large, positive effect on the probability that a hire is made.

Critiques and concerns

• Was there actually another worker available?
• Most of the work on the platform is commodity programming, data entry---seems very unlikely that no substitutes available
• How do we know where the invitation should have been sent?
• We don’t, but we probably don’t need to know, we just need to have invitations not sent to over-subscribed sellers

Why might this not be causal?

• Sellers screen unserious buyers
• “I reject you because you are unlikely to hire anyone; it is not that you are unlikely to hire because I reject you”
• Sellers with hard-to-fill projects are in tight markets
• “I reject you because I’m busy with other projects of the same type (and no one else who does what I does is available)

The experiment we would like to run

Employer sends an invite

Control

(Status Quo)

Treatment:

We force a rejection

Look at opening outcomes

Look at opening outcomes

A natural experiment?

Employer sends an invite

Treatment:

We force a rejection

Look at opening outcomes

Look at opening outcomes

Control

(Status Quo)

Some idiosyncratic

institutional factor that affects whether a worker

accepts an invite but that has no independent effect on the opening outcomes.

Taking a step back, why do workers apply when invited?

More or less fixed per opening

This is not fixed, but rather is time-varying. Why?

Three stylized facts about oDesk

• Openings get all the applications they will ever receive very quickly
• Employers having “rolling” admissions
• The probability of being hired is monotonically decreasing in opening “age”
• Workers more or less “keep” their home-country hours
• Large majority of employers are from the US

Applications arrive very quickly

Reason: Employer decision-time

is unknown and uncertain, so applying later is monotonically worse (b/c of the arrival of other applicants). There is a bit of a “bank run” dynamic.

​

Diurnal cycles on oDesk

High

Medium

Low

Medium

Pr(Online)

Consider two recruiting invitations:

Invitation to worker in Lahore, Pakistan

7:38pm Local Time

Invitation to worker in Manila, Philippines

10:38pm

Employer in Miami, FL

10:38am

Worker in Lahore relatively more likely to be awake (and hence, ultimately accept)

Fraction of employers from the invited worker’s country that are active at the local hour. An “awake” index of sorts.

Threats to identification?

• Better employers more likely to see awake workers?
• No - search and recommendations not time sensitive nor employer-conditioned.
• Employers that invite at weird hours are different or that invite workers from certain countries different?
• No direct effects (FE’s handle this), but could be some interaction.
• Firms seek out workers they know will be awake?
• Not likely - awakeness doesn’t predict country selection

The country-specific probability that a worker is active when an invitation

is received is highly correlated with acceptance.

The 2SLS estimate is

substantially higher (though it is imprecise).

Comments on vacancy “fill” results

• A worker rejecting an invitation to apply lowers the probability that that vacancy is filled.
• This is consistent with either
• high search costs and liquidity
• high or low search costs and no liquidity
• Not being able to cleanly distinguish between these two is a limitation of this work

Other outcomes

An acceptance lowers the probability that the firm interviews any of the “organic” applicants (unsurprising).

An acceptance lowers “late” (after first hour of posting) recruiting, consistent with rejection causing compensatory recruiting.

Comments on “follow-on” results

• Effects on interviewing and further recruiting seem to validate a “costly search” view of the process
• Fill effects show that this compensatory screening and interviewing is not fully compensating

Empirical conclusions

• Given the commodity-nature of the work on oDesk low-liquidity seems unlikely (and even fixable)
• The low ex ante information about workers during screening also suggests “there are not substitutes” is the wrong framework
• Given the above, results suggest search costs still matter, even in an online market
• Better “routing” of invitations seems promising