1 of 27

CS111, Lecture 17�Trust and Operating Systems

Benjamin Xie, Ph.D.

Embedded EthiCS Fellow�benjixie@stanford.edu | benjixie.com

Do Now:

  1. Say hello to your neighbor!
  2. Discuss 2-3 people, services, or things you both trust and why.

This document is copyright (C) Stanford Computer Science and Nick Troccoli, licensed under Creative Commons Attribution 2.5 License. All rights reserved.�Based on slides and notes created by John Ousterhout, Jerry Cain, Chris Gregg, and others. NOTICE RE UPLOADING TO WEBSITES: This content is protected and may not be shared, uploaded, or distributed. (without expressed written permission)

1

2 of 27

Who/What do you trust? How do you warrant that trust?

2

3 of 27

CS111 Ethics Topic: Trust

Synchronization

Thread Dispatcher

Trust

Locks, Condition Vars, & Trust

Assign 2

Assign 3

This (mini)

Lecture

Assign 4

3

4 of 27

Learning Goals

Understand how trust emerges and manifests with operating systems

4

5 of 27

Plan For Today

  • Motivation: Importance of trust in OS
  • What is trust?
  • How does trust emerge?
  • Example: Trusting Linux

​

​

​

5

6 of 27

Plan For Today

  • Motivation: Importance of trust in OS
  • What is trust?
  • How does trust emerge?
  • Example: Trusting Linux

​

​

​

6

7 of 27

Trust in OS for Standardization

  • OS provides efficiency through standardization
  • Users rely on technology built on OS
  • App developers build off of OS
  • OS creators make decisions that ripple far and long

7

8 of 27

Plan For Today

  • Motivation: Importance of trust in OS
    • OS is public infrastructure of software
  • What is trust?
  • How does trust emerge?
  • Example: Trusting Linux

​

​

​

8

9 of 27

Trust as an unquestioning attitude

  • Trust is to stop questioning the dependability of a thing
  • Efficiency/safety tradeoff:
    • Trust lowers the barrier of monitoring, challenging, checking, and questioning
    • Results in more efficiency
  • Involves
    • Intentions
    • Dependence
    • Vulnerability/Risk
  • Example: what/who did you trust to get to class today?

9

10 of 27

Trusting software is extending agency

  • agency: our capacity to take actions that align with our goals
  • “when we trust, we try to make something a part of our agency... To unquestioningly trust something is to let it in—to attempt to bring it inside one’s practical functioning.”
  • Example: glucose monitoring

​

​

CT Nguyen: Trust as an unquestioning attitude

10

11 of 27

Risk: Agential Gullibility

  • Trusting more than warranted
  • Difficult to judge how trust is warranted given how quickly software changes, hard to inspect
  • Example: glucose monitoring issues w/ Android update

11

12 of 27

Takeaway: Trust is powerful, necessary, risky

If I trust people or things (e.g. SW), I …

  • Integrate it with my own functioning
  • Work more efficiently with them (stop questioning)
  • Feel betrayed when they fail us

​

=> Trust (by extending agency) with great care!

12

13 of 27

Self-assessment on trust

Think back to the person/thing/service you trusted…

How does trusting them extend your agency/functioning?

​

How might/did you exhibit agential gullibility? (trust more than is warranted)

​

What would be/was the result of your trust being violated?

13

14 of 27

Self-assessment on trust

Think back to the person/thing/service you trusted… TurboTax Tax Preparation Software

How does trusting them extend your agency/functioning?

> Able to complete taxes more efficiently and had more confidence I did it correctly.

How might/did you exhibit agential gullibility? (trust more than is warranted)

> Tricked into paying for service even though it was legally supposed to be free.

What would be/was the result of your trust being violated?

> Feeling of betrayal. Stopped using software. �

Learn more: https://www.propublica.org/article/inside-turbotax-20-year-fight-to-stop-americans-from-filing-their-taxes-for-free

14

15 of 27

Plan For Today

  • Motivation: Importance of trust in OS
    • OS is public infrastructure of software
  • What is trust?
    • Extending agency to software through unquestioning attitude
  • How does trust emerge?
  • Example: Trusting Linux

​

​

​

15

16 of 27

Three paths to trust

  1. Assumption: trust absent any cluses to warrant it
    1. E.g. using unknown third party library b/c deadline nearing
    2. Trust first result in online search
  2. Inference: reputation is based on past performance, characteristics, institutions
    • Some weaker (e.g. trust in brands or affiliation)
    • Some stronger (e.g. past performance)
    • Trust in prior versions of software
  3. Substitution: structural arrangements that partly substitute need for trust
    • Often involves separation of code, responsibilities
    • E.g. user permissions of file system, keeping personal info off work accounts, devices

Paul B. de Laat: How can contributors to open-source communities be trusted? On the assumption, inference, and substitution of trust

16

17 of 27

Self-assessment on how trust manifests

Identify one person/thing/service that you trust by…

​

Assumption (trust absent clues to warrant it)

​

Inference (trust from evidence of past performance, characteristics, institutions)

​

Substitution (structural arrangement to partly decrease the need for trust)

17

18 of 27

Self-assessment on how trust manifests

Identify one person/thing/service that you trust by…

​

Assumption (trust absent clues to warrant it)

> Anyone warning me about imminent danger (e.g. “look out for the car!”)

​

Inference (trust from evidence of past performance, characteristics, institutions)

> Password management service (inferred trust based on online reviews, review of privacy policy)

​

Substitution (structural arrangement to partly decrease the need for trust)

> Keep some important passwords stored locally and not on app

18

19 of 27

Plan For Today

  • Motivation: Importance of trust in OS
    • Trust amongst tech users, app developers, and OS developers is intertwined
  • What is trust?
    • Extending agency to software
  • How does trust emerge?
    • Assumption, inference, substitution
  • Example: Trusting Linux

​

​

​

19

20 of 27

Linux is hard to trust

1.1 million commits

13.9k contributors

8+ million lines of code

20

21 of 27

Users Trusting Linux

  • Why: People use Linux-based tools to extend their agency
    • Smartphones: Android based on Linux kernel
    • Servers: 13.6% of servers run on Linux
    • Supercomputing: 498/500 supercomputers run on Linux
  • How trust emerges?
    • Assumption
      • “never thought about it”
      • ”no other option”
    • Inference
      • Trusts open source software
      • Have previously used it without issue
    • Substitution
      • Redundant security protocols (e.g. strong password, third-party anti-virus software, isolate/encrypt sensitive files)

21

22 of 27

App Developers Trusting Linux

  • Why: Standardization and tools of OS enable efficiency
    • High cost to build and maintain new OS
    • LINUX is familiar (UNIX-family of OS), lowers learning time developers
  • How trust emerges?
    • Assumption: somewhat rare given affordances to infer trust
    • Inference
      • Used by other app developers
      • “lots of stars on GitHub”
      • trust Linus Torvalds
    • Substitution
      • code is open source (read it, fork it)
      • Add “redundant” checks in code (ex: spurious wakeup)

​

​

22

23 of 27

OS Developers Trusting Linux

  • Why: No single person can build & maintain an OS. Need to extend agency to others to support.
  • How trust emerges?
    • Assumption: rarely happens (risks of bugs, exploits)
    • Inference
      • Known in community
      • Previously code submissions were high quality
    • Substitution
      • Formalization: tools and procedures to streamline cooperation
      • Division of roles
      • Decision making: Linus has final authority

“I don’t like the idea of having developers do their own updates in my kernel source tree. (...) �“there really aren’t that many people that I trust enough to give write permissions to the kernel tree.” �– Linus Torvalds

23

24 of 27

Trusting systems involves trusting people

Thompson, Ken. "Reflections on trusting trust." Communications of the ACM 27.8 (1984): 761-763.

24

25 of 27

Recap

  • Trust amongst tech users, app developers, and OS developers is intertwined
  • Trust is about extending agency, enabling “unquestioning attitude”
  • Trust emerges through assumption, inference, substitution
  • Linux kernel to used broadly and large, so users, app developers, OS developers must trust through inference and substitution

​

​

​

Ethics takeaway: Trust is often required, powerful, and dangerous. Key design challenge is how we design structures that enable us to substitute trust.

Benjamin Xie, Ph.D.

Embedded EthiCS Fellow�benjixie@stanford.edu | benjixie.com

25

26 of 27

Interaction between apps and OSs

  • Developers build off of OS
  • OS creators make decisions that ripple far and long
  • => changes affect each other

Operating Systems

Technology built over OS

26

27 of 27

Examples of OS and app interactions

Changes to OS can affect applications

Example: Windows 95 disabled competitors’ apps

Changes to applications can affect OS usage

Example: virtual desktop introduces vulnerabilities

Eric Roberts, CS181

27