1 of 21

Checking Websites’ GDPR Consent Compliance for Marketing Emails

Karel Kubicek, Jakob Merane, Carlos Cotrini,�Alexander Stremitzer, Stefan Bechtold, and David Basin

The 22nd Privacy Enhancing Technologies Symposium (PETS 2022)

July 13, 2022

2 of 21

Marketing emails

2

3 of 21

Legal requirements – GDPR and ePrivacy Directive

ePrivacy Directive – opt-in

GDPR – defines consent

Goal: analyze the compliance and create a dataset to enable future studies.

3

4 of 21

Annotation by registering

  • "Geometrically" sampled Alexa 1M list
  • Unique email address per registration
    • self-hosted

​

  • 22 legal properties
  • 7 annotators (degree in law)
  • Every page annotated twice (Cohen's κ 0.74)
  • Disagreements resolved by third annotator

​

  • 666 websites with successful registration

​

4

✔

✔

5 of 21

Emails

  • 568 websites sent us an email
  • We annotated >5000 emails as marketing/servicing
  • First email:
    • 59% double opt-in
    • 36% confirmation (against best practices)
    • 5.5% marketing
  • Legal notice and unsubscribe in marketing emails:
    • 15.3% missing legal notice
    • 3.5% missing unsubscribe
  • Password leakage:
    • 2.3% send user-provided password in plaintext
    • additional 9.2% hijackable authenticators

5

6 of 21

Emails – third-party sharing

Inspecting senders of all emails

Mathur et al., US presidential campaign: 7%

​

6

7 of 21

Decision procedure for potential privacy violations

7

8 of 21

Potential violations in registration process

8

9 of 21

Future work and conclusions

  • Automated classification (proof of concept already in this work)
  • Automating registration
  • Reporting violations to data protection authorities
  • Legal taxonomy + decision procedure using the labels
  • 1000 websites analyzed and annotated
  • Reported 7 types of potential violations, found on 21.9% of websites

Authors:

Karel Kubicek karel.kubicek@inf.ethz.ch, Jakob Merane jakob.merane@gess.ethz.ch,

Carlos Cotrini, Alexander Stremitzer, Stefan Bechtold, and David Basin

ETH Zurich

​

​

9

Project page: https://karelkubicek.github.io/post/reg-pets

10 of 21

Pilot study

  • 100 annotated websites
  • Testing the instructions, legal properties
  • Based on this we:
    • designed the annotating tool,
    • improved instructions by reducing ambiguity,
    • created set of annotation examples, and
    • tuned labels.

10

11 of 21

Automation

  • Prefiltering crawl: detecting registration forms
    • From 6k randomly sampled websites, it found registration form at 1.1k of them → 1k annotations
    • We manually checked 100 discarded websites:
      • 13 were having registration form
      • We registered to them and found no statistical deviance from sample in our study
  • Autofill:
    • Annotator fills the form with keywords, e.g., 'pwd' for password field
    • The annotating interface interacts with Firefox to fill the form with predefined credentials
  • Double opt-in: automated registration confirmation
    • By opening confirmation link - reliable
    • By filling OTP code - unreliable
    • Manually checked and confirmed when needed for the whole study

​

These tools are essential building blocks of automation of the whole procedure.

11

12 of 21

Legal properties

  • Marketing consent (ma_consent): The website asks for consent from the user for marketing emails on the registration page.
  • Marketing purpose (ma_purpose): Registering with the website is only, or mainly, for receiving marketing emails.
  • Marketing checkbox (ma_checkbox): There is a checkbox that the user must tick to give consent for
  • marketing emails.
  • Privacy policy checkbox (pp_checkbox): There is a checkbox for consent for the website’s privacy policy.
  • Terms and conditions checkbox (tc_checkbox): There is a checkbox for consent for the website’s terms and conditions.
  • Pre-checked checkbox (X_pre_checked): The corresponding checkbox X is already ticked by default.
  • Forced checkbox (X_forced): It is required to tick the corresponding checkbox to successfully register. This is often indicated with asterisks on the registration forms.
  • #tying_Y: There is only one checkbox asking for (tying) two or three consents together. Therefore, Y ∈ {ma_pp, ma_tc, pp_tc, ma_pp_tc}.
  • #forced_Z: The website does not ask for consent to the privacy policy and/or terms and conditions, but assumes it through the registration process. Hence Z ∈ {pp, tc, pp_tc}.
  • #settings: Refusing consent requires more clicks, therefore the consent is assumed by default.
  • #age: The user’s age or the date of birth are required for registration.
  • #colortrick: The colors on the website nudge the user to consent. For example, giving consent is highlighted with green, while refusing it is red.
  • #hidden: The declaration of consent can be easily missed by users.

12

13 of 21

Annotation instructions

  • 8 pages legal instructions
  • 22 examples of annotation
  • 2 pages technical instructions

13

14 of 21

Violations with ranking

Note that more popular websites are not more compliant than lower ranked websites.

The number of websites of rank 1k-10k not sending legal notices is far larger than the websites of other ranks (including high-rank websites). The p-value of the two proportions Z-Test of the rank < 1k against data of all other ranks is 0.054 after adjustment for multiple measurements by Holm–Bonferroni method.

Moreover, for the potential violation “Email despite no opt-in,” the websites with high rank show more potential violations than those with low rank. This observation has a p-value of 0.156.

14

15 of 21

Logistic regression

15

16 of 21

Inter-annotator agreement

16

17 of 21

Registration state

17

18 of 21

Interdependence of legal properties

18

19 of 21

Emails in time

For our study, we annotated emails during the period starting in September 2020 and ending in February 2021, so we were able to observe several marketing trends influencing the email content. We observed that 5.8%, 11.7%, and 4.2% of marketing emails were related to Black Friday, Christmas, and New Year, respectively. These topics become relevant during autumn and winter, but we did not observe an overall increase in the number of marketing emails. Also, 17.2% of all processed emails were related to the Covid pandemic. As the frequency of marketing emails did not change during these periods (see Figure 13), the observations suggest that trending topics are used to improve marketing campaigns, but they do not generate new newsletter traffic. This hypothesis is based on the fact that during the limited period of the study, we did not observe any spikes in the number of newsletters during these periods. However, to confirm this hypothesis, we would need a more longitudinal study.

19

20 of 21

Legal sources

20

21 of 21

Author:

Karel Kubicek, PhD candidate

karel.kubicek@inf.ethz.ch

​

Team:

Karel Kubicek, Jakob Merane, Carlos Cotrini, Alexander Stremitzer, Stefan Bechtold, and David Basin

​

Project site: https://karelkubicek.github.io/post/reg-pets

​

ETH Zurich

D-INFK

Institute of Information Security

​

This presentation is licensed under CC by SA 4.0.