Data Ethics: Choices and Values
1
original slides & content by Kathleen Creel, Diana Acosta-Navas
CMSC 320 - Introduction to Data Science,
2025
2
BIG THANKS: Zico Kolter (CMU)
& Amol Deshpande (UMD) for the diagram
Ethics
What is right, what is fair, what is just.
3
In an ideal world
4
Established by a Society; allowed by the law
May not explicitly required by the law; align with principles of fairness, honesty, compassion, and respect for human dignity
In the real world
5
We will talk about
DATA ETHICS
6
The set of principles and processes that guide the ethical collection, processing, analysis, use and application of data having an effect on human lives and society
A branch of ethics that evaluates data practices
Ref: d’Aquin et al, Towards an “Ethics in Design” methodology for AI research projects, in AIES 2018
We use data to inform our decisions
7
In Data Science:
What can we learn from a data set?
8
What can we learn from a data set?
9
How to interrogate a data set to ethically find relevant elements?
Values in Design
Data is intrinsically values-laden
VALUES IN DESIGN
10
PROBLEM FORMULATION
DATA INTERPRETATION
DATA
COLLECTION
What are values?
value (n): an individual or community’s belief about what matters
Values express what we care about
Efficiency
Privacy
Truth
Security
Beauty
Fairness
…
Values reveal our assumptions about the world, people interacting with our designs, and how our choices affect them
11
Intentionality of values
Explicit values: Values that designers intend for their products to embody
Collateral values: Values that crop up as side effects of design decisions and the way users interact with those designs. These values are not intentionally designed into the system.
I.e: Social Media
12
Explicit Values
13
Contact-tracing
Health Safety Efficiency
Public interest
Collateral Values
14
https://www.theverge.com/2021/4/27/22405425/android-google-contact-tracing-bug-privacy
https://www.jdsupra.com/legalnews/class-action-filed-against-commonwealth-3421108/
“A better world won’t come about simply because we use data; data has its dark underside.”
― Mike Loukides, Ethics and Data Science
15
16
https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/?sh=48f798316668
Is it Legal?
Is it Ethical?
Remember, data can be powerful, yet it also has its challenges and ethical considerations. A important question that raised is how to respect individuals lives when using their data.
Why Ethics?
DALL-E 2 is an advanced AI capable of generating art, potentially raising questions about its impact on traditional artistic roles.
https://onsitego.com/blog/dall-e-2-tested-how-close-is-this-ai-to-replacing-artists/
Why Ethics?
"In 2017, the Rohingya were killed, tortured...and displaced in the thousands as part of the Myanmar security forces’ campaign of ethnic cleansing. In the months and years leading up to the atrocities, Facebook’s algorithms were intensifying a storm of hatred against the Rohingya which contributed to real-world violence,” said Agnès Callamard, Amnesty International’s Secretary General."
https://www.amnesty.org/en/latest/news/2022/09/myanmar-facebooks-systems-promoted-violence-against-rohingya-meta-owes-reparations-new-report/
Case Study: Spread of harmful anti-Rohingya content
Major Areas of Concern in Data Ethics
19
Data Ethics: Choices and Values
20
PROBLEM FORMULATION
LANGUAGE
BIAS AND REPRESENTATION
FAIRNESS
Why should we care? Who benefits?
Who’s harmed? What data?
Problem Formulation Statements
21
Ultimately the answers to these questions will depend on the values and assumptions of the people being asked.
Example: Unsubscribe feature
Potential design: Unsubscribe button on app homepage. Single click and user is unsubscribed.
Potential design: Unsubscribe button on app homepage 🡪 10 minute questionnaire 🡪 talk with a representative
22
Problem Formulation Statements
23
What is the problem to be solved?
Choice of Data
24
DATA COLLECTION
25
Data Collection
How can you be sure that your data has been ethically sourced?
26
“No data can be collected until explicit consent for that purpose has been given and that consent can be retracted at any time.”
Informed Consent and Transparency is important!
GDPR AND CONSENT
General Data Protection Regulation – new law in EU that recently went into play
Requires unambiguous consent
27
According to GDPR (General Data Protection Regulation)
“When collecting personal information, organisations must act transparently and with consent, collecting data only for the explicit purpose it’s needed.”
28
The GDPR is one of the most comprehensive and toughest data protection laws in the world and provides an overarching framework for the processing of personal data.Though it was drafted and passed by the European Union (EU), it imposes obligations onto organizations anywhere, so long as they target or collect data related to people in the EU. The regulation was put into effect on May 25, 2018.
There are more: California Consumer Privacy Act (CCPA): rights over personal information held by businesses; California Privacy Rights Act (CPRA); Health Insurance Portability and Accountability Act (HIPAA); Gramm-Leach-Bliley Act (GLBA): federal law that requires financial institutions to protect the privacy and security of consumers' personal financial information; Federal Trade Commission Act (FTCA); Children's Online Privacy Protection Act (COPPA) etc.
Case Study: It's Not OKCupid
The popular dating platform OkCupid was used in an experiment to determine if love is actually blind.
29
Data Ethics: Choices and Values
30
PROBLEM FORMULATION
LANGUAGE
BIAS AND REPRESENTATION
FAIRNESS
Descriptive: what is : Describes facts or observations without judgment.
Normative: what should be : Involves values or ethics about what is right or wrong.
Thick normative: both
how data can be framed or analyzed based on both observed facts and ethical considerations.
Language
31
Descriptive terms? Evaluative terms?
32
Descriptive vs. Normative Language
33
Descriptive language
90-minutes long”
Attendance: Not Mandatory
25
Textbook Required
Example: Descriptive Terms
Descriptive vs. Normative Language
Normative language:
35
AWESOME
27
GREAT TEACHER
Example: Normative Language
Descriptive language
Normative language:
28
38
CLEAR
EASY TO
LISTEN TO
38
Distinction between normative & descriptive language is not always clean-cut
Thick Normative Terms
Descriptive AND normative:
39
Example: toxic speech classification & context
40
Bears suck!!!
Ex: AI tools’ “racy” score & gender bias
41
Guardian. Feb 2023
Descriptive or Normative?
42
Does the program you are writing contain descriptive claims?
Do it contain normative claims or values?
How about thick normative terms?
Data Ethics: Choices and Values
43
PROBLEM FORMULATION
LANGUAGE
BIAS AND REPRESENTATION
FAIRNESS
Why should we care? Who benefits?
Who’s harmed? What data?
Descriptive: what is Normative: what should be Thick normative: both
What is Bias?
44
Statistical bias is the difference between measured results and “true” value.
□ This is the “neutral” or statistical meaning of the word bias. You will see it often in discussions of patterns in data.
Discrimination
45
Concern: Difference Type of Bias can
happen is data collection Step
Sampling Bias, Non Response Bias, Volunteer Bias, Social Desirability Bias, Measurement Bias, cultural bias or societal bias, many more ………
SAMPLING BIASES
Sampling effective at reducing the data you need to analyze
Ideally you want random sample
Bias in sampling: need to be very careful when generalizing inferences drawn from a sample
→ Even for random samples
Questions to ask: How was the sample selected? Was it truly random? Potential biases? How were questions worded? How is missing data/attrition handled? Was the sample size large enough?
47
SOME POTENTIAL SOURCES OF BIASES
Sample Bias
Survey/Response Bias
And more …..
48
Case Study: Sample Bias in Twitter Users
Twitter provides rich data, including network, text, and sociodemographic information. Valuable for observing behavior and opinions over time.
Scenario: suppose you want to predict the outcome of a US presidential race with two candidates: a democrat and a republican. A clever data science project is set up, and suppose you are able to get an accurate reflection of the political preference for all Twitter users as being a democrat (Y = 0) or a republican (Y = 1) voter.
Reference: Textbook DM
Case Study: Sample Bias in Twitter Users
Twitter users may not represent the broader population.
(1) overrepresentation in densely populated areas.
(2) gender bias, which is decreasing over time.
(3) non-representative racial and ethnic distributions.
According to “Research Center Study (2019)"
Reproducing social biases
57
BIAS AND REPRESENTATION
reproduces social biases:
(e.g. stereotypes)
often aligns most with dominant US culture
intellectual teacher
pleasant teacher
I have data about people! Now what?
52
Check for Statistical Bias
What correlations and patterns exist in my dataset?
In what ways do they fail to accurately represent the world?
Check for Discriminatory Bias
In what ways do the biases compound existing injustice?
Decide how to use the data given bias
Data Ownership
Unlawfully acquire photo from other person’s mobile without permission or consent.
53
Is it ethically correct?
Are this person owner now?
Who owns Data? Who has legal rights and complete control to make decisions about the data they generate or possess?
Data Privacy
How to protection of individuals' personal information and the control (privacy right) they have over its collection, use, and disclosure?
54
Is it ethical to disclose your photo to unauthorized parties?
Is it ethical to use your photo without your consent?
Data Anonymization
The trouble with anonymization:
For every amount of data you might release, there exists some set of priors that, if possessed by an attacker, will let them reverse your anonymization.
A survey on the patient’s medical condition
56
Which Dataset is maintaining Anonymity ?
Dataset (A)
Dataset (B)
Data Validity
The integrity and accuracy of the data being collected, analyzed, and used for various purposes.
57
Data Validity: Is collected data valid?
58
Surgery is always performed in a hospital and if the data shows otherwise, the data is invalid.
Ref: Source
Hospital Data Rules
CASE STUDY: NETFLIX PRIZE
Raw data: 100M dated ratings from 480K users to 18K movies
Why released: improve predicting ratings of unlabeled examples
How anonymized: exact details not described by Netflix
Attacks: dataset is claimed vulnerable [Narayanan Shmatikov 08]
Consequences: rich source of user data for researchers
A company uses an algorithm to screen job applicants for interviews
How about Algorithmic Fairness?
60
Result
Potential Candidates | ||
Name | Age | Skills |
John | 25 | … |
Nick | 25 | … |
Bob | 25 | … |
…. | …. | … |
Is this outcome of algorithm ethically fair?
Misuse of Statistics / Misrepresenting Data
The unethical practice of distorting or manipulating statistical information to support a particular agenda, mislead others, or draw false conclusions.
61
Example: Selective collection of positive customer feedback as representative of overall customer satisfaction.
62
A statistical anomaly in the form of an excess just below 0.05
SCIENTIFIC METHOD: STATISTICAL ERRORS
P values not as reliable as many scientists assume
p-hacking: cherry picking data points etc., to get the p-values; repeating experiments if they fail till you get the result
Much discussion/debate about this issue in recent years
63
Social Issues
AI and Alignment
Most people working in AI think there is about a 5% chance that an AI will kill a large portion of humanity in the next 50 years or so.
I've heard that number go as high as 90% from people who work in AI alignment.
Always Remember
It is not always simple to decide what's right or wrong.
Trolley Problem!
67
It is not always simple to decide what's right or wrong.
Decide Whom to save?
This example shows how complex ethical questions can be, and how changing details can make the decision even harder. This relates to data ethics because making choices about data use can also involve difficult decisions where the right answer isn't always clear.
Data ethics is about navigating these complex situations, making thoughtful choices, and considering the impact on people's privacy, fairness, and well-being.
Book Recommended to read
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens
Democracy by Cathy O'Neil
Data Science in Industry
(NOT NEEDED FOR THE COURSE)
WHAT IS A DATA SCIENTIST?
Many types of “data scientists” in industry …
70
Thanks to: Zico Kolter
KEY DIFFERENCES
Classical statistics vs machine learning approaches
Developing data science tools vs. doing data analysis
Working on a core business product vs more nebulous “identification of value” for the firm
71
FINDING A JOB
Make a personal website.
Highlight relevant coursework, open source projects, tangible work experience, etc
Highlight tools that you know (not just programming languages, but also frameworks like Pytorch, TensorFlow and general tech skills)
72
“REQUIREMENTS”
Data science job postings – and, honestly, CS postings in general – often have completely nonsense requirements
In most cases (unless the position is a team lead, pure R&D, or a very senior role) you can work around requirements:
73
INTERVIEWING
We saw that there is no standard for being a “data scientist” – and there is also no standard interview style …
… but, generally, you’ll be asked about the five “chunks” we covered in this class, plus core CS stuff:
Take-home data analysis project (YMMV): you may be given a take-home data analysis project as part of the interview process (vary company to company)
74
GRADUATE SCHOOL, ACADEMIA, R&D, …
Data science isn’t really an academic discipline by itself, but it comes up everywhere within and without CS
Academic work in the area:
75