Five years of the
Right to be Forgotten
Kurt Thomas, with Theo Bertram, Elie Bursztein, Stephanie Caro,�Hubert Chao, Rutledge Chin Feman, Peter Fleischer, Albin Gustafsson,�Jess Hemerly, Chris Hibbert, Luca Invernizzi, Lanah Kammourieh Donnelly, Jason Ketover, Jay Laefer, Paul Nicholas, Yuan Niu, Harjinder Obhi,�David Price, Andrew Strait, and Al Verney
Security and Privacy Group
Right to be Forgotten (RTBF)
Delist “inaccurate, inadequate, irrelevant,
or excessive” information surfaced by search queries containing the name of the requester.
Security and Privacy Group
Balancing between individual privacy, public interest
Security and Privacy Group
Balancing between individual privacy, public interest
Decision made by search provider via manual review
Security and Privacy Group
Example of balancing test
Request
A former politician requested to delist 3 URLs from Google Search reporting on his recent departure from politics in connection with a drug scandal.
Outcome
Google delisted the 3 URLs as they disclosed the politician’s private home addresses, not just information about the scandal.
Security and Privacy Group
Example of balancing test
Request
A former politician requested to delist 3 URLs from Google Search reporting on his recent departure from politics in connection with a drug scandal.
Outcome
Google delisted the 3 URLs as they disclosed the politician’s private home addresses, not just information about the scandal.
Security and Privacy Group
Over the last five years then...
3M
Requested URLs
502,000
Requesters
45%
URLs delisted
Security and Privacy Group
Our measurement study
Types of sites requested
Information
present on site
Entities creating requests
Security and Privacy Group
Provider greater transparency around how the RTBF is applied in practice
Security and Privacy Group
Review process & dataset
Security and Privacy Group
Data present in a request
Email address
URLs to delist
Country
Timestamp
Security and Privacy Group
Manual annotations added during review
Requesting entity
Minor, government official, corporate entity...
Social, directory, news, government records
Type of site
Personal information, professional information, crime, political, self-authored ...
Information on page
Security and Privacy Group
Five years of data, since implementation
47,000
Average URLs per month
Security and Privacy Group
Average time to arrive at a decision
2014
2019
85 days
6 days
Security and Privacy Group
Average time to arrive at a decision
2014
2019
85 days
6 days
Security and Privacy Group
Which sites are requested for delisting?
Security and Privacy Group
Two dominant intents for delistings
Legal history
19%
News
2%
Government
16%
Directory
13%
Social media
Personal information
Security and Privacy Group
Influenced by regional privacy attitudes and local norms
France
Italy
Spain
42%
Directory, Social media
(vs. 29% across Europe)
33%
News
(vs. 19% across Europe)
10%
Government
(vs. 2% across Europe)
Security and Privacy Group
Delisting rates reflect public interest balancing
53%
Directory,�Social media
35%
News
19%
Government
Security and Privacy Group
Increasing share of requests to news
dailymail.co.uk�ouest-france.fr�telegraph.co.uk
repubblica.it
Examples include:
Security and Privacy Group
Declining share of requests to social media
facebook.com�twitter.com�youtube.com�plus.google.com�instagram.com
Examples include:
Security and Privacy Group
Influence of GDPR on directory requests
After GDPR, only 55% of the top 500 requested directory sites remain online.
Examples include:
118712.fr�societe.com�192.com
Security and Privacy Group
What information is requested for delisting?
Security and Privacy Group
Professional & personal information most common
24%
8%
Professional information
Personal information
Sensitive personal information
34% of all requested URLs
Professional activities, contact info, addresses, medical status, and more.
Predominantly on directory sites.
2%
Professional activities, contact information, phone numbers, and mailing addresses.
Security and Privacy Group
Criminal records and negative reviews also common
17% of all requested URLs
24%
8%
Crime
Professional wrongdoing
9%
Convictions, acquitals,� or negative reviews.
Security and Privacy Group
Remaining types of common information
24%
20%
9%
4%
Name not found
Self-authored
33% of requested URLs
Political
24%
Security and Privacy Group
Affinity of types of information to different sites
| News |
Professional information | 18% |
Personal information | 3% |
Crime | 22% |
Professional wrongdoing | 22% |
Self authored | 5% |
Name not found | 10% |
... | |
Security and Privacy Group
Affinity of types of information to different sites
| News | Social Media |
Professional information | 18% | 8% |
Personal information | 3% | 5% |
Crime | 22% | 3% |
Professional wrongdoing | 22% | 2% |
Self authored | 5% | 33% |
Name not found | 10% | 29% |
... | | |
Security and Privacy Group
Delisting rates reflect public interest
97%
Personal information
48%
Crime
3%
Political
Criticism of platform �or activities
Personal addresses, contact info, photos
Security and Privacy Group
Who makes delisting requests?
Security and Privacy Group
Majority of requested URLs come from private individuals
84%
Private individual
6%
Minor
2%
Corporate entity
4%
Politician
4%
Public figure
Security and Privacy Group
Small number of requesters make heavy use of RTBF
Requested URLs from just 10K requesters
34%
Security and Privacy Group
Long tail of hundreds of thousands of requesters
Requested URLs from just 10K requesters
34%
Requested URLs from 400K requesters
29%
Security and Privacy Group
Requester activity varies by country
URLs requested per
1000 Internet users
12
7
3
France
Italy
Greece
Security and Privacy Group
Decreasing number of new requesters
6,800
Average new requesters per month
Security and Privacy Group
Relationship between requester’s origin and audience
News Site | Belgium | Germany | Spain | France | UK |
89% | 0% | 0% | 1% | 2% | |
93% | 0% | 0% | 0% | 0% |
Over 89% of requests to top Belgian news sites � come from local requesters
Origin of requester, by volume of requested URLs
Security and Privacy Group
Relationship between requester’s origin and audience
News Site | Belgium | Germany | Spain | France | UK |
89% | 0% | 0% | 1% | 2% | |
93% | 0% | 0% | 0% | 0% | |
0% | 96% | 0% | 1% | 2% | |
0% | 1% | 96% | 1% | 1% | |
0% | 0% | 97% | 1% | 0% |
Origin of requester, by volume of requested URLs
Security and Privacy Group
Relationship between requester’s origin and audience
News Site | Belgium | Germany | Spain | France | UK |
89% | 0% | 0% | 1% | 2% | |
93% | 0% | 0% | 0% | 0% | |
0% | 96% | 0% | 1% | 2% | |
0% | 1% | 96% | 1% | 1% | |
0% | 0% | 97% | 1% | 0% | |
0% | 0% | 0% | 100% | 0% | |
0% | 0% | 0% | 99% | 0% | |
bbc.co.uk | 0% | 0% | 0% | 0% | 97% |
dailymail.co.uk | 0% | 1% | 0% | 2% | 92% |
Origin of requester, by volume of requested URLs
Security and Privacy Group
Conclusion
Challenge in providing transparency without de-anonymizing specific requesters.
Nuanced, dynamic usage of the RTBF over last five years.
Influenced in part by local privacy concerns and media norms.
Security and Privacy Group
Research now reflected in Transparency Report
Security and Privacy Group
Thanks!
kurtthomas@google.com
Security and Privacy Group