Advertising as Coordination

Experimental Evidence on Adding Advertising to a Market

Apostolos Filippas

Diego�Urraca

John�Horton

Economists' (very) mixed views on advertising

• "Good"
• Communicate the existence of relevant potential trading partners & prices (Stigler)
• Send signals about latent quality (Nelson; Milgrom & Roberts)
• "Bad"
• Manipulate preferences
• Increases costs

And even with the

informational view…

"The vast amount of product information available to consumers through online search renders most advertising obsolete as a tool for conveying product information"

A field experiment to test "informative" digital advertising

• Basic idea:
• Let all sellers advertise
• Randomize buyers into seeing advertising; change nothing else
• Measure how it affects the outcomes of buyers, by treatment status
• Research questions
• Are sellers virtuously selected?
• Do buyers seek out advertisers?
• Do buyers form "better" matches?

Empirical context

• A large online labor market for work that can be done remotely
• Computer programming, graphic design, data entry, etc.
• Project sizes vary dramatically, from just a few hours to years-long engagements
• An important way matching happens:
• Employer posts job
• Employer searches for suitable candidates and invites them to apply
• Workers apply (both recruited and organic)
• Employer decides who hire, if any
• One key problem:
• As sellers are individual workers, they can easily "stock out" and turn down invitations from recruiting employers

Some matching markets have sellers who are

inherently supply constrained

Labor markets

(typically only ~40 hours/week)

Supply constraints can make matching challenging

• If you recommend an unavailable seller, a match cannot happen ❌
• A house that has already been sold
• A candidate who already has a job and is not looking
• An Airbnb room already booked
• Recommender system heuristics can make the problem worse / rich-get-richer dynamic 💸
• "People like Harry Potter" →"Recommend Harry Potter to every kid"
• "Dave builds good websites" →"Recommend Dave to build every website"

Cause: Capacity is seller private information

Consequences: Buyers pursue unavailable sellers

and matches are not formed

Why not just

ask sellers about

availability?

• "As needed (open to offers)"
• "30 hours or less"
• "30+ hours (full-time)"

What is your availability?

Question asked to sellers

• "As needed (open to offers)"
• "30 hours or less"
• "30+ hours (full-time)"

What is your availability?

"30+ hours (full-time)"

• "As needed (open to offers)"
• "30 hours or less"
• "30+ hours (full-time)"

What is your availability?

"30+ hours (full-time)"

Would-be buyers condition on this

information

LinkedIn has this kind of feature

(presumably they read my paper)

But 7 years later…

Temporary

Back in 2019, about 5% of sellers

were changing their availability status each month

Today, less than 1% of seller use

the "availability" feature

Almost 90% of sellers now state

they are available "full time"

Despite this, only about 30% of buyer inquiries are responded to

by sellers

The problem is that information is missing for economic reasons, not technical reasons

The core issues: Job offers/inquiries are valuable

• Job offers are:
• Potentially better than the current option and can be taken
• Can be used to negotiate with current buyer/employer
• Can be passed on to others as a favor to later be reciprocated
• If self-interested sellers can get more offers at zero cost, they will not tell the truth about their capacity 🤥

​

Why study this particular problem?

• Labor markets are per se important and solving their problems is important
• It's a great example of an adversarial machine learning problem ⚔️ 🤖
• The platform would like to know x to predict y
• The workers has preferences over y and so manipulates x
• It might speak to larger question of what role advertising plays or can play in marketplaces

A new solution:

What if we made

sellers pay to say

they are available?

Adding advertising to a large online labor market

• The Hope 🥰:
• More available sellers are willing to advertise
• Buyers seek them out because they know those advertising sellers will be more responsive
• In equilibrium, there is separation (a separating equilibrium)
• The Fear 😱:
• 'Desperate" or worse sellers are willing to advertise; they are highly available for a (bad) reason
• Buyers ignore or actively avoid advertising sellers
• In equilibrium, there is pooling (no one advertises)

Implementation: Pay $p/week to the platform;

get this badge added to your seller profile

This is what it looks like to buyers

Why might

this "work"?

Suppose there is some distribution of seller values to having the badge

"Value to seller from having the badge in equilibrium"

Platform picks a price, $p/week

"Value to seller from having the badge in equilibrium"

Everyone with a value

greater than this advertises

Buyers focus on advertisers because of what is signals, making advertising valuable

"Value to seller from having the badge in equilibrium"

But what if p is too low given value?

👋 Hand wavy mention of tatonment and equilibrium---see paper model for details

Ok, but why would

buyers seek out advertisers?

Nelson (1974):

But why would buyers be interested in advertisers?

Costco Founder & previous CEO

Are advertising positively selected ("peaches" 🍑 ) or adversely selected ("lemons" 🍋)?

• The "lemons" argument
• "If you're so good, why do you need to advertise?"
• "If you want something done, ask a busy person to do it"
• The "peaches" argument
• Buyers don't like pursuing unavailable sellers and there are lots of reasons a seller might have low capacity
• There's lots of hidden information in markets and advertising sellers might have lower costs, greater upside to doing more work, and so on
• An interesting tension:
• If advertisers are "lemons" buyers will learn to ignore advertisers, in which case the platform will need to give advertisers more prominence or trick people into thinking advertisers are actually organic sellers

In conventional labor markets, employers are biased against those who been out of work for longer periods 🍋

Platform proceeded cautiously: A four step procedure

1. Let select sellers buy paid advertising if they choose
2. Randomize buyers to being able to see that advertising
3. See what happens
4. If promising, fiddle with advertising prices to optimize information

Take-up of advertising was rapid: About 40% of eligible sellers opted in within two weeks

Buyer posts a job

Control:

Does not see ads

Treatment:

Does see ads

Randomization

Buyer posts a job

Control:

Does not see ads

Treatment:

Does see ads

Randomization

A Key Point:

No change in assortment or ranking!

About 40K would-be buyers in treatment and 40K in control

Allocation happened over time, as buyers posted jobs (the point of allocation)

Main research questions 🤔

First step:

Let's look at the level of the

individual seller/buyer impression

About 3.5M impressions (From our 80K allocated buyers searching)

In about 50% of those impressions, the seller was advertising (and in only half of those would the buyer know this)

Pooled over all impressions, buyers send an inquiry to about 8% of impressions they see

Let's compare inquiries by seller advertising status, position and buyer treatment

Seller position in search results

1 = first position

10 = last position on the page

Probability that seller

receives an inquiry from the buyer

Buyer

clicks here

Sample are "AdOff" sellers

i.e., sellers who did not buy

advertising

No ad

▲Treated buyers: Can see Ads

● Control buyers: Can not see Ads

No difference in treatment and control �(all sellers are non-advertisers)

Holds true generally;

no difference.

Now let us look at when advertising sellers are getting the impression

When exposed to advertising sellers, treated buyers seek them out

Main research questions 🤔

What are the

effects on buyers from

seeing ads?

Let's move to the cumulative

buyer experience view

(as buyers are the unit of randomization)

Estimate a Poisson regression and QMLE and robust SEs

Buyer outcome of interest, e.g., Count of buyer inquiries sent

Treatment effect w/ Poisson: Interpretable as % changes, more or less

About 3% more inquiries sent to sellers in total

Now we look at count of inquiries specifically to advertisers

About a 6% increase in inquiries to advertising sellers among treated buyers

Are non-advertising sellers

harmed or "crowded-out"?

No measurable decline in inquires to non-advertisers!

Main research questions 🤔

About 3% more actual proposals received

from inquiries (i.e., more options)

About 3% more contracts formed

More proposals from these sellers seems to have a nearly linear effect on matches formed

Main research questions 🤔

Main research questions 🤔

​

Why did buyers seek out

advertising sellers? Let's compare

advertisers and non-advertisers.

We can compare the pre-experiment attributes of advertising and non-advertising sellers

Advertisers and non-advertisers had nearly identical stated availability 🤥

But have more complete resumes (+6%), higher feedback scores from past contracts (+20%) and lower wage asks (-2%)

Advertisers are also, on average, getting more inquiries and responding more positively to buyer inquiries.

​

It seems they truly are "Available Now"

Statistically

Modeling selection

into advertising

Predictors are standardized pre-treatment seller attributes

Fit a logit with seller data where outcome is an "AdsOn" indicator

= 0

= 1

1 standard deviation (SD) increase in number of inquiries received is associated with a 5% reduced probability of advertising

1 SD increase in in proposals conditional on an inquiry associated with 9% increase in probability of advertising

Advertisers were more active by any measure on factors they could control and had good reputations

Lower average hourly rates and higher past feedback

A simple interpretation:

Advertisers want more business

and are not getting it "organically"

Main research questions 🤔

Was advertising

effective for sellers?

Other empirical results (not covered)

• Does advertising help sellers get more contracts?
• Yes (with a diff-in-diff with a seller panel)
• Is it ROI positive to advertise?
• We don't know because we don't know seller their margins
• Why would sellers vary in capacity?
• In a nutshell, matching frictions. See model.
• Sure, this worked in practice, but does it work in theory?
• Yes. See model.

Interesting follow-on questions

• What is the optimal price for this kind of advertising?
• If everyone advertises, not informative
• If no one advertises, not informative
• Maybe ½?
• Is the optimal message space finer than binary?
• E.g.,
• "I'm super-duper available" vs. "I'm somewhat available"
• Can the platform extract more revenue with other pricing schemes?
• Should the platform incorporate status into ranking algorithms?
• How do you "sell" movement along a gradient?
• How can the platform ensure that advertising sellers are virtuously selected?
• Can it make the "good rich" as opposed to just hoping the "rich are good"

Stepping back

Marketplace advertising more generally

Two online marketplace problems

• Nearly all marketplaces face a "visibility allocation" problem
• What sellers should buyers see? What buyers should sellers see?
• This has mostly been addressed algorithmically
• Information retrieval approach to ranking, personalization, recommendations, reviews
• Given the stakes to sellers, lots of gaming and manipulation ⚔️ 🤖
• All marketplaces have to raise revenue
• Fixed fees screen out low-value transactions the platform would like to intermediate
• Ad valorem fees can screen out low-margin transactions the platform would still like to intermedia
• Fee levels are set by the platform and could be too high or too low relative

"Feed two birds with one scone"

(nicer version of "Kill two birds with one stone")

Platform's need

for revenue

Platform's visibility allocation problem

On-platform

advertising

Thank you!

Market design implications

• Advertising as an economic mechanism allowed revelation of private information and improved efficiency
• It actually raised additional revenue for the platform
• Advertising expenditure directly
• Indirect monetization from more matches formed
• Gives sellers more control

When exposed to advertising sellers, treated buyers seek them out