Ethics of Machine Learning

Wanheng Hu, Ph.D.

Embedded Ethics Fellow, Stanford University

CS229, Winter 2025

What are we talking about when we talk about the “Ethics of Machine Learning”?

In-class interaction: What comes to your mind when you think of “ML ethics”?

2

Unpacking “Ethics of ML”

Ethics

• Moral and immoral
• Right and wrong
• Good and bad
• Just and unjust
• Harms and benefits

​

Machine learning

• As a technique: ethical choices by ML practitioners
• E.g. Is it right/moral to use this data?

​

3

Unpacking “Ethics of ML”

Ethics

• Moral and immoral
• Right and wrong
• Good and bad
• Just and unjust
• Harms and benefits

​

Machine learning

• As a technique: ethical choices by ML practitioners
• As a technology: ethical risks induced by the model
• E.g. Is the model’s output just?

4

Unpacking “Ethics of ML”

Ethics

• Moral and immoral
• Right and wrong
• Good and bad
• Just and unjust
• Harms and benefits

​

Machine learning

• As a technique: ethical choices by ML practitioners
• As a technology: ethical risks induced by the model
• As a tool: ethical consequences of its use
• E.g. Will the system harm certain people?

​

5

Learning Goals

• Perspective: Understanding ML as a socio-technical system rather than merely a technical artifact.
• Awareness: Recognizing that design choices may cause or prevent ethical risks and consequences.
• Acknowledgment: Ethics is rarely straightforward or universal—ethical considerations are often complex and context-dependent.
• Ability: Identifying potential ethical issues and making informed, nuanced ethical decisions.

6

A Quick Roadmap

1. Why care?
2. ML as socio-technical system
3. A spectrum of ML-related ethical challenges
4. Case discussions
• Medical imaging, fairness and biases
• Generative-AI & stereotypes

7

Why care? Making ethics visible

• What you’ve learned in CS229 so far is mostly about specific techniques through lectures and assignments
• Linear regression, house price prediction
• Logistic regression, edge detection
• Neural networks, recognizing hand-written numbers
• Language models & transformers
• Decision tree, spam filtering
• …

​

8

9

Why care? Making ethics visible

• These examples and cases are mostly simplified, pre-packaged for your learning purposes, highly “purified” from practical concerns and social consequences
• Well-curated datasets readily available
• Clearly defined tasks and technical trajectories
• Illustrative demos that are not really deployed

10

Welcome to the “real-world”…

• All kinds of real-world issues and cases

11

12

13

14

Sociotechnical systems

• A technology can be viewed as a sociotechnical system that extends the engineering realm.
• A “seamless web” of technical, economic or financial, political, environmental, and social considerations.
• Consider the electric power (Hughes, 1983)

​

​

15

16

https://anatomyof.ai/

What’s “sociotechnical” about ML in practice?

In-class activity: in groups of 2-3, choose an ML system and discuss its sociotechnical dimensions – what are the humans and resources that made its creation and functioning possible?

17

Rethinking ML models

18

Rethinking ML models

19

A spectrum of ethical issues for ML-ers…

​

​

20

(Saltz et al, 2019)

Oversight related challenges

Q1: Which laws and regulations might be applicable to this project?

• Laws and regulations are related to ethics and norms
• E.g. Privacy: HIPAA (Health Insurance Portability and Accountability Act) in the U.S. and GDPR (General Data Protection Regulation) in Europe
• Regulations often lag behind
• E.g. informed consent, ambiguous data ownership, etc.

Q2: How is ethical accountability achieved?

• Who will be accountable for the harms?
• Identify potential stakeholders and evaluate possible harms
• Ethical review board or algorithmic accountability reporting

​

​

21

Data related challenges (privacy + anonymity)

Q3: How might the legal rights of organizations and individuals be impinged by our use of the data?

• Right to control data sharing (GDPR, consent): individuals should control how their data is shared and used.
• Ownership and transfer rights: legal ambiguity exists around which rights/obligations transfer as data ownership changes.
• Third-party sharing: data shared with third parties that may follow different privacy standards.
• Transparency and accountability: making data supply chain visible, avoid secrecy about data sources and partnerships.

​

​

​

22

Data related challenges (privacy + anonymity)

Q4: How might individuals’ privacy and anonymity be impinged via aggregation and linking of the data?

• Re-identification risks: linking anonymized data points (e.g., zip code, birthdate, gender) may accurately re-identify ppl.
• Aggregation and correlation: harms may come from combining and correlating individual data points.
• E.g. the Netflix case: Users’ viewing preferences were exposed when Netflix data (published for a competition) was combined with IMDB data.

​

​

23

Data related challenges (availability + validity)

Q5: How do you know that the data is ethically available for its intended use?

• Legal access ≠ Ethical use: just because data is accessible (e.g., census data) doesn’t mean it’s ethical to use it.
• Public data ≠ Open use: even if content is public (e.g., tweets), how it’s used matters.
• Informed consent is tricky: true informed consent for data use is often ambiguous and complex.
• Alignment with intent: data should be used in ways that align with the provider’s expectations.

​

​

24

Data related challenges (availability + validity)

Q6: How do you know that the data is valid for its intended use?

• Accuracy matters: data must be accurate to ensure ethical and reliable outcomes (e.g., imputing missing values or excluding cases with missing values could skew results.)
• Fitness for purpose: data must suit the context and the goal of the model (e.g., patient-generated data for clinical use, student scores for teacher performance evaluation).
• Putting data back context: data should be used in line with its original context and limitations.

​

​

​

25

Model related challenges (bias)

Q7: How have we identified and minimized any bias in the data or in the model?

• Model bias from training data: models can inherit societal biases (e.g., ageism, ethnicism, sexism) from the data they’re trained on.
• Modeler’s use of protected attributes: using protected categories (e.g., gender, race) to make decisions can lead to legal and ethical issues.
• Algorithmic grouping: ML can create new, unprotected categories that lead to unfair treatment (e.g., “dog owners aged 38-40” → lower insurance rate).

​

​

26

Model related challenges (bias)

Q8: How was any potential modeler bias identified and then, if appropriate, mitigated?

• Subjectivity in model design: decisions on metrics, algorithms, and data sources reflect subjective choices (e.g., whether to rank applicants based on economic background, race, academic achievements etc. in admissions).
• Power relationships: design choices can be impacted by the power dynamics in the institutional structures of the modeler
• Unintended new biases: attempts to fix bias can introduce new forms of bias.

​

27

Model related challenges (trans + accu)

Q9: How transparent does the model need to be and how is that transparency achieved?

• Black box problem: complex models (e.g., neural networks) are hard to interpret, even for experts.
• High-stakes decisions: transparency is critical when outcomes impact regulated areas or vulnerable groups (e.g., lending money).
• Alternatives to complexity: simpler models (e.g., logistic regression) can improve explainability.
• Process transparency: transparency can also come from clear documentation and institutional accountability.
• Proprietary constraints: some models remain closed due to competitive or security concerns.

​

28

Model related challenges (trans + accu)

Q10: What are likely misinterpretations of the results and what can be done to prevent those misinterpretations?

• No perfect accuracy (statistical limits): models predict probabilities, not certainties.
• Scope and limitations: model validity depends on data quality and context.
• Responsibility for clarity: the duty to communicate limitations and context to reduce misuse.

​

29

Imagine you’re developing a medical ML device

​

​

30

(Vayena et al, 2018)

The spectrum can be continued…

• Job losses
• Environmental impacts
• Autonomous weapons
• Surveillance
• …

​

31

Let’s look at some popular ML applications

• Computer vision
• Tech: Supervised learning
• Case 1: Medical imaging, biases, and fairness
• Generative AI
• Tech: Self-supervised learning, Reinforcement learning
• Case 2: Text-image generation and demographic stereotypes

​

​

​

32

ML in medical imaging

33

(Esteva et al., 2019)

ML models can be biased

• ML systems can be biased against certain sub-populations: they present disparate performance for different subgroups defined by protected attributes such as age, race/ethnicity, sex or gender, socioeconomic status, etc.
• In healthcare, this can be against bioethics principles: justice, autonomy, beneficence and non-maleficence
• E.g. in diabetic retinopathy, a strong gap in the diagnostic accuracy (73% vs. 60.5%) for light-skinned vs. dark-skinned subjects, certain patients being sent home without receiving the care they need

​

​

​

​

34

(Ricci Lara et al., 2022)

But what does it mean for a model to be fair?

• Multiple measurements
• Tensions and trade-offs
• Group-fairness metrics

35

Where do biases come from?

• Data
• Model
• People

36

How do we mitigate biases?

• Before training
• E.g., collecting more representative data
• During training
• E.g., data augmentation, adversarial training
• After training
• E.g., prediction calibration

37

38

A few quick take-aways

• A socio-technical perspective: ML systems do not get developed in a social vacuum; they are deeply intertwined with socioethical choices and contexts
• There is no such thing as “raw data”
• There can be multiple reasonable design choices
• These choices may lead to different consequences
• A sophisticated ethics view: what counts “ethical” is complicated, and requires reflective and thoughtful decisions
• A sense of agency: there are important things you can do!

39

Generative AI

40

Text-image generation AI apps

• Demographic stereotypes

41

(Bianchi et al., 2023)

42

?

43

?

44

45

Discussion

• What – biased, fair?
• Where
• How

​

46

References (recommended readings in bold)

• Eubanks, Virginia. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press, 2018.
• Gray, Mary L., and Siddharth Suri. Ghost work: How to stop Silicon Valley from building a new global underclass. Harper Business, 2019.
• O'neil, Cathy. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown, 2017.
• Hughes, Thomas Parke. Networks of power: electrification in Western society, 1880-1930. JHU press, 1993.
• Saltz, Jeffrey, et al. "Integrating ethics within machine learning courses." ACM Transactions on Computing Education (TOCE) 19.4 (2019): 1-26.
• Vayena, Effy, Alessandro Blasimme, and I. Glenn Cohen. "Machine learning in medicine: addressing ethical challenges." PLoS medicine 15.11 (2018): e1002689.
• Esteva, Andre, et al. "A guide to deep learning in healthcare." Nature medicine 25.1 (2019): 24-29.
• Ricci Lara, María Agustina, Rodrigo Echeveste, and Enzo Ferrante. "Addressing fairness in artificial intelligence for medical imaging." Nature communications 13.1 (2022): 4581.
• Bianchi, Federico, et al. "Easily accessible text-to-image generation amplifies demographic stereotypes at large scale." Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 2023.

47

Help us evaluate the Embedded Ethics Program!

https://tinyurl.com/embedethics

​

10-15 minute survey, taking it (or not) won’t impact your grade in the class in any way, and teaching team won’t know who participates or not.

​

Option to provide your email address to receive a $10 gift card, up to the first 800 participants. Compensation once per quarter (SUNet login required).

​

Questions? Email embeddedethics@stanford.edu