1 of 53

Your Secrets Are Safe:�How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Yuxi Wu, Panya Gupta, Miranda Wei,

Yasemin Acar, Sascha Fahl, Blase Ur

2 of 53

2

https://www.w3.org/Consortium/

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

3 of 53

Users have misconceptions about private mode

3

X X

(

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

4 of 53

Study Motivation

Consider the role of browsers

in communicating what private mode actually does

i.e., protection from local threats

(and not most remote ones)

4

[0] These misconceptions were a major motivator for our study. In line with past work, we wanted to further investigate the difference between what users think private mode can do, and what it can actually do.

[1] But to take it a step further, we also wanted to consider at the role that the browsers themselves play in generating or confirming these misconceptions.

This is because while all major web browsers include a private mode,

[2] they differ significantly in how they communicate what private mode does.

[3] Generally, private browsing modes are only a means of protection for shared devices, or against an attacker who has physical access to the machine. Generally, these modes protect against local privacy threats by not storing browsing history, cookies, or temporary files across browsing sessions on the machine.

[4] Note then, that private mode does not protect against most remote threats, such as the visibility of IP address, or the ability of visited sites to store information about the visitor.

5 of 53

Disclosure

the explanation that a browser presents when users open a new window in private mode

5

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

6 of 53

Disclosures vary

6

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

7 of 53

Disclosures vary

7

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

8 of 53

Disclosures vary

8

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

9 of 53

Mobile disclosures vary even more

9

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

10 of 53

Survey Methodology

460 participants on Amazon’s Mechanical Turk

30 minutes, $5 compensation

13 conditions

12 conditions based on disclosures from the wild
1 control condition

10

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

11 of 53

11

Desktop

Mobile

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Each disclosure was tested as a survey condition.

First, there were 7 desktop versions.

Brave
Chrome v59
Chrome v60, which came out right before the study was launched
Microsoft Edge
Firefox
Opera
Safari

All disclosures were taken from the Mac OS version, except Edge, which was taken from Windows 10.

So there were 5 mobile versions, from:

Brave, Chrome, Firefox, Opera and Safari

We tested the mobile disclosures that differed most from the desktop version, so the Chrome and Safari disclosures were taken from iOS, and the Brave, Firefox, and Opera were taken from Android.

A note about these disclosures: we wanted to avoid any potential confounds from participants‘ biases with a particular company, or from the visual layout of the disclosure. This is why these 12 disclosures use the same color and font, and we told participants they were from a new browser called Onyx.

12 of 53

12

Control Condition

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

13 of 53

Survey Content

standard

private

standard

private

20 different scenarios

Two sets of questions:

13

Distinguishing

Comparative

https://

?

[>, <, ==]

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

[0] Participants were presented with 20 different scenarios and asked how the browser would behave. These 20 scenarios came from:

Previously documented misconceptions
Things mentioned in browser disclosures

[1] After participants saw the scenario, they were asked questions about how the browsers would behave. These questions were most naturally expressed in in one of two ways.

[2] We call the first distinguishing. First, we ask what would happen in standard mode? And then, what would happen in private mode?

[3] The second type of questions we called comparative. So looking at standard and private mode at the same time, does standard mode do something more, less, or the same than private mode?

And now, I’ll pass it to Yuxi to talk about how participants answered these questions.

14 of 53

Results

14

15 of 53

Our participants (n = 460)

15

Gender

54.6% male

44.6% female

0.8% nonbinary/NA

Tech background

86.5% no background

11.5% worked, educated in tech

2.0% NA

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

16 of 53

Non-misconceptions

16

3:14 pm picturesofcorgis.com

3:15 pm picturesofcorgis.com

3:16 pm picturesofcorgis.com

3:17 pm picturesofcorgis.com

3:18 pm picturesofcorgis.com

3:19 pm picturesofcorgis.com

3:20 pm picturesofcorgis.com

3:21 pm picturesofcorgis.com

3:22 pm picturesofcorgis.com

3:23 pm picturesofcorgis.com

3:24 pm picturesofcorgis.com

3:25 pm picturesofcorgis.com

3:26 pm picturesofcorgis.com

downloads

89.8% correct

94.8% correct

99.3% correct

Photos not cached

History not saved in menu

Downloaded files remain

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

17 of 53

Overestimated Protections

17

What is a corgi?

127.0.0.1

56.3% incorrectly believed

25.2% incorrectly believed

40.2% incorrectly believed

...that searches could not be associated

(while logged in)

...that IP address could not be collected

...that geolocation could not be estimated

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Now let’s get into misconceptions, or scenarios where more than 20% of participants answered incorrectly. Many of these misconceptions arose when participants overestimated what private mode could do.

For example, 56.3% of participants (259) believed that even if they were logged into their account, if they were using private mode, their search queries would not be associated with with account.

144 participants conflated browser’s local history with Google’s history
77 participants believed private mode would remove logged info retroactively, as in, once they closed the private mode’s tab

25.2% of participants (116) believed that in private browsing mode, their IP addresses could not be collected.

89 participants said that it was because that was simply a feature of private browsing mode

Estimating geolocation was also a common misconception. 40.2% of participants (185) answered this scenario incorrectly.

125 participants believed location hiding was simply a feature
21 participants believed that it would be impossible because IP address would be hidden

18 of 53

Misconceptions - comparative differences

18

https://

Corgi dog beds for just $49.99!

X X

(

27.1% incorrectly believed

30.9% incorrectly believed

...that there was a difference in the level of virus and malware protection between modes

...that the proportion of targeted ads was the same between modes or higher in private mode

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

19 of 53

19

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

20 of 53

Individual disclosure results

20

Desktop

Mobile

Control

vs.

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

21 of 53

No different than control

21

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

22 of 53

Better than control

22

Chrome v59 (desktop)

Chrome v60 (desktop)

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

23 of 53

Key Takeaways

23

24 of 53

Understanding comes from interaction

24

3:14 pm picturesofcorgis.com

3:15 pm picturesofcorgis.com

3:16 pm picturesofcorgis.com

3:17 pm picturesofcorgis.com

3:18 pm picturesofcorgis.com

3:19 pm picturesofcorgis.com

3:20 pm picturesofcorgis.com

3:21 pm picturesofcorgis.com

3:22 pm picturesofcorgis.com

3:23 pm picturesofcorgis.com

3:24 pm picturesofcorgis.com

3:25 pm picturesofcorgis.com

3:26 pm picturesofcorgis.com

downloads

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

25 of 53

Language matters

25

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

26 of 53

Language matters

26

Opera (mobile)

Firefox Focus (mobile)

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

27 of 53

27

https://

Browsers’ disclosures

make a difference

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

One limitation of our study is the argument that users have become habituated to browser disclosures, and don’t really read them anymore.

However, browser disclosures are the most reliable part of a user’s private browsing mode experience. They will see these every time they open a new tab.

In our study, we found that browser’s disclosures do make a difference. Some disclosures led to significantly more correct answers, while the others were the same as a useless disclosure.

Obviously, it is really important for users’ security and privacy that they understand what private mode does and does not do. For future work, we suggest looking at ways to improve disclosures even beyond static text. We could imagine graphics or even interactive visualizations to explain how private browsing mode works.

28 of 53

Future work...

28

Directly related

Contextual notifications about private browsing features
Interactive visualizations about private browsing features
Exposure to a user’s own browsing activity in private mode

Tangential questions

Exposure to “more private” browsing
Usability tradeoffs in “more private” vs regular private browsing mode
Proportion of private browsing mode features actually used
Differences in usages of various private browsing modes

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

29 of 53

Your Secrets Are Safe:�How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Yuxi Wu, Panya Gupta, Miranda Wei,

Yasemin Acar, Sascha Fahl, Blase Ur

{yuxiwu,weim}@uchicago.edu

RQ: Do disclosures impact misconceptions of private browsing mode? If so, how?

Methodology: 460-participant survey testing 13 disclosures in 20 scenarios

Results: Disclosures have an impact and contribute to many misconceptions

29

30 of 53

30

https://FancyWines.com

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

31 of 53

31

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Interestingly, there were also some scenarios where participants underestimated private mode’s protections. In these scenarios, private browsing mode had an effect where participants did not expect them to.

On this graph, we have the same axes.

First, participants did not know that search queries they made in private mode while not logged into their Google account would not be associated with previous searches that they made, whether logged in or not. However, the fact that many participants also answered incorrectly about standard mode indicates that this may just be a scenario most people don’t think about.

In contrast, while many participants understood that standard mode would keep the downloaded file name in the list of downloaded files, they also believed this to be true in private mode. This may be because downloaded files themselves persist after closing a private mode tab, but the the actual list of file names does not.

32 of 53

Roadmap

Why use private browsing mode?
Past work
Study motivation
Survey methodology
Results
Key takeaways

32

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

33 of 53

Aggregate Confidence

Males were 3.46 times more confident ...

Younger participants were …

Less time in private mode were more confident ...

33

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

34 of 53

What do we use private browsing for?

34

35 of 53

Disclosures vary

35

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

36 of 53

Questions we anticipate getting:

But no one reads disclosures?

What were all of the correct answers, anyways?

Have disclosures been updated since publication?

What should disclosures say?

36

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

37 of 53

Notes from Blase

make title slide look like private mode disclosure

include universities

start with motivation

- people have misconceptions about private browsing mode

- all major browsers have private mode, meant to be local

- some limited prior work on misconceptions

- our main research question: browsers differ by how they explain what private mode does

- show real disclosures, show 3 different ones in full page; maybe mobile versions

- to what extent do the disclosure impact whether people have misconceptions

- no citations

“previously documented misconceptions”

survey methodology

- keep main lines, remove dots

move the walk through to right before 460 participants

- maybe flip; abstract or example

results

- participant demographics: as briefly as possible, more graphics; separate reasons and usage

- drop reasons for using

- start with the scenarios, confidence/which disclosure did best at the end

- 4 types of scenarios —> remove org slide

- make axes bigger, explain how to read the slide (incorrect answers)

- no graph for non-misconceptions

- show blank graph, then insert data

- separate distinguishing and comparative

- “overestimated protections”

- have quotes in notes

implications/discussion

- less words

- screenshot of disclosure language and show the logo of each browser

how do the disclosures impact misconceptions

- at beginning of results: show results for disclosures in aggregate

- then specific disclosures at the end

- meaninglessly vague control, read it outloud; see how the other ones compare

- scale of how many they got right

- misconceptions that disclosures particularly emphasized

end on key takeaway slides, and contact info, pictures of coauthors

37

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

38 of 53

Imagine there is a new web browser called Onyx.

38

Onyx has a mode called “private browsing mode.”

39 of 53

39

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

40 of 53

40

PicturesOfCorgis.com

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

41 of 53

41

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

42 of 53

42

MyGamingSite.com

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

43 of 53

43

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

44 of 53

44

google.com

the meaning of life

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

45 of 53

45

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

46 of 53

Our Participants (N=460)

46

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

47 of 53

Our Participants (N=460)

47

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

48 of 53

Our Participants (N=460)

48

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

49 of 53

Overestimated Protections

49

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

Search queries associated (56.3% participants answered incorrectly)

144 participants conflated browser’s local history with Google’s history
77 participants believed private mode would remove logged info retroactively

Geolocation can be estimated (40.2%)

125 participants believed location hiding was simply a feature
21 participants believed that it would be impossible because IP address would be hidden

Employer, government, or ISP can track browsing (37.0%, 22.6%, 22.0%)

__ participants because no search history saved, so no one could track

Better protected from viruses or malware (27.1%)

“it’s a no brainer really, when in private mode nothing is stored” (P146)
“when you are in private mode there is no direct link to you or your computer so malware etc. would not be saved onto your computer or in your cookies” (P67)

** note that all graphs are out of all participants, where 60 is the max (should it be 100?)

** show correct answers?

50 of 53

50

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

51 of 53

51

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

52 of 53

Underestimated Protections

52

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode

53 of 53

Our Participants (N=460)

53

Yuxi Wu, Miranda Wei | Your Secrets Are Safe: How Browsers’ Explanations Impact Misconceptions About Private Browsing Mode