Part IV: Fairness in �Machine Learning

Shahab Asoodeh (McMaster University)

Flavio P. Calmon (Harvard University)

Mario Diaz (Universidad Nacional Autónoma de México)

Haewon Jeong (UC Santa Barbara)

2022 IEEE International Symposium on Information Theory (ISIT)

Data-driven algorithms are increasingly applied to individual-level data to support decision-making in applications of individual-level consequence.

Recidivism prediction

Employment decisions/hiring

Personal finance

Data-driven algorithms are increasingly applied to individual-level data to support decision-making in applications of individual-level consequence.

Recidivism prediction

Employment decisions/hiring

Personal finance

Do these algorithms discriminate on race, sex, and/or some other protected attribute?

Discrimination in Machine Learning

Discrimination is the prejudicial treatment of an individual based on membership in a protected group (e.g., race or sex).

​

[Barocas and Selbst’16]

Discrimination in Machine Learning

Discrimination is the prejudicial treatment of an individual based on membership in a protected group (e.g., race or sex).

​

[Barocas and Selbst’16]

Google Translate

(recorded 06/22/22)

Discrimination in Machine Learning

Discrimination is the prejudicial treatment of an individual based on membership in a protected group (e.g., race or sex).

​

[Barocas and Selbst’16]

Google Translate

(recorded 06/22/22)

Discrimination in Machine Learning

ProPublica’16

How can information-theoretic tools help ensure �fair machine learning?

How can information-theoretic tools help ensure �fair machine learning?

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

How can information-theoretic tools help ensure �fair machine learning?

A case study in applying information-theoretic tools to fair ML:

• To split or not to split?

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

How can information-theoretic tools help ensure �fair machine learning?

A case study in applying information-theoretic tools to fair ML:

• To split or not to split?

Emerging aspects of fair ML:

• Predictive multiplicity
• Fair Use

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

How can information-theoretic tools help ensure �fair machine learning?

A case study in applying information-theoretic tools to fair ML:

• To split or not to split?

Emerging aspects of fair ML:

• Predictive multiplicity
• Fair Use

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

Setup

Model

(e.g. past grades, questionnaire answers)

(e.g. academic performance)

Disparate Treatment vs. Disparate Impact

Model

(e.g. past grades, questionnaire answers)

(e.g. academic performance)

Disparate treatment (direct discrimination): occurs when the protected attributes are used directly in decision making.

(e.g. sex, age, race)

[Barocas and Selbst’16]

Disparate Treatment vs. Disparate Impact

Model

(e.g. past grades, questionnaire answers)

(e.g. academic performance)

(e.g. sex, age, race)

Disparate impact: group attributes are not used directly, but reliance on variables correlated with them lead to different outcome distributions for different groups.

[Barocas and Selbst’16]

Disparate Treatment vs. Disparate Impact

Model

(e.g. past grades, questionnaire answers)

(e.g. academic performance)

(e.g. sex, age, race)

Changes in input distribution…

Disparate Treatment vs. Disparate Impact

Model

(e.g. past grades, questionnaire answers)

(e.g. academic performance)

(e.g. sex, age, race)

Changes in input distribution…

disparate

impact

Performance

Disparate Treatment vs. Disparate Impact

Model

(e.g. past grades, questionnaire answers)

(e.g. sex, age, race)

Changes in input distribution…

disparate

impact

Performance

Group fairness metrics

Disparate Treatment vs. Disparate Impact

Model

(e.g. past grades, questionnaire answers)

(e.g. sex, age, race)

Group fairness metrics

Group fairness metrics are usually defined in terms of differences between average outcomes and error rates across different populations

Building Group Fairness Metrics

Population

FPR

TPR

TNR

FNR

[Narayan, FAT* Tutorial 2017]

Building Group Fairness Metrics

Population

Population

FPR

TPR

TNR

FNR

FPR

TPR

TNR

FNR

Statistical parity

[Narayan, FAT* Tutorial 2017]

Building Group Fairness Metrics

Population

Population

FPR

TPR

TNR

FNR

FPR

TPR

TNR

FNR

Equalized odds

[Hardt, Price Srebro’16]

Building Group Fairness Metrics

Population

Population

FPR

TPR

TNR

FNR

FPR

TPR

TNR

FNR

Equal opportunity

[Hardt, Price Srebro’16]

Building Group Fairness Metrics

Population

Population

FPR

TPR

TNR

FNR

FPR

TPR

TNR

FNR

There are (combinatorically many) other metrics and trade-offs you can derive from this table

For trade-offs see, for example, [Lipton et al., 2018; Chouldechova, 2017; Kleinberg et al., 2016; Pleiss et al., 2017]

​

Building Group Fairness Metrics

Population

Population

FPR

TPR

TNR

FNR

FPR

TPR

TNR

FNR

There are (combinatorically many) other metrics and trade-offs you can derive from this table

For trade-offs see, for example, [Lipton et al., 2018; Chouldechova, 2017; Kleinberg et al., 2016; Pleiss et al., 2017]

​

Population

Population

FPR

TPR

TNR

FNR

FPR

TPR

TNR

FNR

There are (combinatorically many) other metrics and trade-offs you can derive from this table

Important: Several group fairness metrics can be written as linear constraints on the classifier

[Lemma 1 in Alghamdi et al. ISIT’20]

Group fairness violations

Group (un)fairness:

Average performances changes conditioned on a group attribute (e.g., race, age)

Facial recognition:

Healthcare

[Buolamwini, Gebru’18]

[Science’18]

Education

[The Guardian’20]

Ensuring group fairness (more on this later)

Group (un)fairness:

Average performances changes conditioned on a group attribute (e.g., race, age)

Pre-processing

[Hajian’13], [Zemel et al.’13], [Kamiran & Calders’12], [Hajian & Domingo-Ferrer’13], [Ruggieri’14],[C et. al’17], [Madras et al’18], [Ghassam et al.18], [Wang et al.’19] and more!

In-processing

[Fish et al.’16], [Zafar et al.’16],

[T. Kamishima, S. Akaho * J. Sakuma’11], [A. Agarwal et al.’18]

Post-processing

[Hardt, Price, Srebro’16], [Wei, Ramamurthy, C. 20], [Menon & Williamson’18], [Yang et al.’20], [Alghamdi et al.’20]

Ensuring group fairness (more on this later)

[Frideler et al.’19, “A Comparative Study on Fairness-Enhancing interventions in ML”]

Benchmarks!

Group (un)fairness:

Average performances changes conditioned on a group attribute (e.g., race, age)

“Individual” fairness

[Dwork et al. 2011]

​

Key idea: “similar” individuals are treated “similarly”.

Formulation: “Lispschitz” constraint on classifier output:

How can information-theoretic tools help ensure �fair machine learning?

Emerging aspects of fair ML:

• Predictive multiplicity
• Fair Use

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

A case study in applying information-theoretic tools to fair ML:

• To split or not to split?

How can information-theoretic tools help ensure �fair machine learning?

Emerging aspects of fair ML:

• Predictive multiplicity
• Fair Use

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

A case study in applying information-theoretic tools to fair ML:

• To split or not to split?

Motivation

Image is from: https://stanfordmlgroup.github.io/projects/chexnet/

Classifier

Classifier

🤔

Should I use the group attribute?

pneumonia

Motivation

Strategy 1: group-blind classifier

Training time

Strategy 1: group-blind classifier

features

predicted probability

Pneumonia positive: 10%

Testing time

Training time

Negative

​

Positive

​

Negative

Strategy 2: coupled classifier

group

attribute

Training time

Negative

​

Positive

​

Negative

Strategy 2: coupled classifier

features

group

attribute

M

Testing time

Training time

Negative

​

Positive

​

Negative

Pneumonia positive: 10%

predicted probability

Strategy 3: split classifiers

sex: F

sex: M

Training time

Split classifiers are also called decoupled classifiers in [Dwork et al., 2018] and [Ustun et al, 2019]

Strategy 3: split classifiers

Testing time

sex: F

sex: M

Training time

sex?

F

M

F

Split classifiers are also called decoupled classifiers in [Dwork et al., 2018] and [Ustun et al., 2019]

Positive

​

Negative

Positive

Negative

Pneumonia positive: 10%

Pneumonia positive: 15%

predicted probability

Comparison

group

attribute

​

Comparison

group

attribute

​

Comparison

group

attribute

​

Related work

group

attribute

​

Split classifiers are also called decoupled classifiers in [Dwork et al., 2018] and [Ustun et al, 2019]

How can a group attribute be used in ML systems given that it is ethical and legal to do so?

[Dwork et al., 2018]

This work

group

attribute

​

• Is it always beneficial to use a group attribute in the prediction task?
• When does using a group attribute benefit model performance the most?

Drawback of group-blind classifier

group

attribute

​

Different

probability

distributions

Drawback of split classifiers

group

attribute

​

Limited amount

of samples

-

+

+

-

+

+

+

+

-

-

-

-

+

+

+

-

-

-

Factors

group

attribute

​

• data distributions
• number of samples
• hypothesis class
• …

Goal

group

attribute

​

What is the gain of incorporating a group attribute in a classifier?

Notation

1

Notation: unlabeled distribution and labeling function

unlabeled

distribution

labeling

function

[Ben-David et al., 2010]

binary label

(e.g., pneumonia)

0

1

Notation: classifier and loss function

features

Pneumonia positive: 10%

probabilistic classifier

predicted probability

Benefit-of-splitting An information-theoretic quantify

worst-case loss

group 0

group 1

[Ben-David et al., 2010]

Loss function

Benefit-of-splitting An information-theoretic quantify

worst-case loss

group 0

group 1

Loss function

Benefit-of-splitting An information-theoretic quantify

group 0

group 1

Loss function

Benefit-of-splitting An information-theoretic quantify

group 0

group 1

Benefit-of-splitting: warm up

Information-theoretically, splitting classifiers never harms any group.

Benefit-of-splitting: warm up

Information-theoretically, splitting classifiers never harms any group.

When does splitting benefit model accuracy the most?

Bounding the benefit-of-splitting: two factors

Bounding the benefit-of-splitting: two factors

Disagreement between labeling functions

Similarity between unlabeled distributions

Bounding the benefit-of-splitting: taxonomy of splitting

Colors = subgroups (e.g. male/female)

+ / - = label

Bounding the benefit-of-splitting: taxonomy of splitting

Colors = subgroups (e.g. male/female)

+ / - = label

Bounding the benefit-of-splitting: taxonomy of splitting

Splitting does not

bring much benefit

Colors = subgroups (e.g. male/female)

+ / - = label

Bounding the benefit-of-splitting: taxonomy of splitting

Splitting does not necessarily

bring much benefit

Colors = subgroups (e.g. male/female)

+ / - = label

Bounding the benefit-of-splitting: taxonomy of splitting

Splitting benefits the most!

Colors = subgroups (e.g. male/female)

+ / - = label

Bounding the benefit-of-splitting: taxonomy of splitting

Splitting benefits the most!

Colors = subgroups (e.g. male/female)

+ / - = label

How can information-theoretic tools help ensure �fair machine learning?

A case study in applying information-theoretic tools to fair ML:

• To split or not to split?

Emerging aspects of fair ML:

• Fair Use
• Predictive Multiplicity

Fairness metrics widely used in the literature

• Group fairness
• Individual fairness

Fair Use of group attributes

​

Non-maleficience: do not harm

Beneficience: do good

​

[Ustun et al. ICML’19]

Users of a model should also be incentivized to report their data features truthfully.

When collecting a group attributes, we must ensure

Fair Use of group attributes

​

Non-maleficience: do not harm

Beneficience: do good

​

[Ustun et al. ICML’19]

Rationality

Envy-freeness

Users of a model should also be incentivized to report their data features truthfully.

When collecting a group attributes, we must ensure

Fair Use of group attributes

[Ustun et al. ICML’19]

Rationality

Envy-freeness

Fair Use of group attributes

[Ustun et al. ICML’19] , [Paes, Long, Ustun, Calmon’22]

Fair Use of group attributes

[Ustun et al. ICML’19] , [Paes, Long, Ustun, Calmon’22]

Rationality:

Envy-freeness:

Fair Use of group attributes

[Ustun et al. ICML’19] , [Paes, Long, Ustun, Calmon’22]

Rationality:

Envy-freeness:

• Less stringent fairness metric than group-fairness;
• Relevant to healthcare applications (e.g., personalized medicine) where collection of new data may be challenging;
• Might be impossible to verify in practice for many intersectional groups.

Fair Use of group attributes

[Ustun et al. ICML’19] , [Paes, Long, Ustun, Calmon’22]

Rationality:

• Less stringent fairness metric than group-fairness;
• Relevant to healthcare applications (e.g., personalized medicine) where collection of new data may be challenging;
• Might be impossible to verify in practice for many intersectional groups.

Minimax converse bound: at most 20 binary group attributes!

Predictive Multiplicity

Training time

Negative

​

Positive

​

Negative

Predictive Multiplicity

Training time

Negative

​

Positive

​

Negative

• Set random seed;
• Initialize weights;
• Set dropout parameters;
• Randomly shuffle the dataset and split in batches;
• Etc.

Predictive Multiplicity

Training time

Negative

​

Positive

​

Negative

Random Initialization

Predictive Multiplicity

Training time

Negative

​

Positive

​

Negative

features

predicted probability

Pneumonia positive: 10%

Testing time

Training time

Negative

​

Positive

​

Negative

features

predicted probability

Pneumonia positive: 10%

Testing time

features

predicted probability

Pneumonia positive: 10%

Testing time

Set of model parameters

Set of model parameters

Random Initialization

[Breiman’01, Fisher et al.’19, Semenova et al.’19, Marx, C, Ustun’20]

[Breiman’01, Fisher et al.’19, Semenova et al.’19, Marx, C, Ustun’20]

features

Pneumonia positive: 23%

Pneumonia positive: 62%

Pneumonia positive: 50%

Predictive Multiplicity: models with similar average performance may produce conflicting predictions on individual samples.

[Breiman’01, Fisher et al.’19, Semenova et al.’19, Marx, C, Ustun’20]

features

Pneumonia positive: 23%

Pneumonia positive: 62%

Pneumonia positive: 50%

Predictive multiplicity: models with similar average performance may produce conflicting predictions on individual samples.

[Hsu, C arxiv’22]

Predictive multiplicity: models with similar average performance may produce conflicting predictions on individual samples.

[Creel and Hellman’21]

• If predictive multiplicity is not accounted for, decisions supported by ML models may depend on arbitrary and unjustified choices (e.g., model initialization).

​

• In sectors dominated by a few algorithms (algorithmic leviathans used in credit scoring, government services), this may lead to arbitrary loss of opportunities to certain individuals

Where tools from information theory can help:

1. Discovering multiplicity

​

​

Can we delineate the Rashomon Set without (re)training thousands of models?

Where tools from information theory can help:

2. Reporting multiplicity

​

​

[Hsu and C’22]

Pneumonia positive: 23%

Pneumonia positive: 62%

Pneumonia positive: 50%

2. Reporting multiplicity

​

​

[Hsu and C’22]

[Hsu and C’22]

Channel capacity can be used to quantify multiplicity!

Rashomon Capacity:

[Hsu and C’22]

Channel capacity can be used to quantify multiplicity!

Distribution over models in the Rashomon Set

Rashomon Capacity:

[Hsu and C’22]

Channel capacity can be used to quantify multiplicity!

Distribution over models in the Rashomon Set

Rashomon Capacity:

[Hsu and C’22]

• Between 1 and c;
• 1 iff predictions of all models match;
• c iff there are models in the Rashomon Set; that assign each of the c classes;
• Monotonic in the size of the Rashomon Set.

Rashomon Capacity:

Where tools from information theory can help:

3. Resolving multiplicity

​

​

Which model should we use?

[Hsu and C’22]

Where tools from information theory can help:

3. Resolving multiplicity

​

​

[Hsu and C’22]

A direct application of Caratheodóry’s Theorem yields that for each sample at most c models capture score variation as measured by Rashomon Capacity.

Take-aways:

• Group-fairness is, at this point, the most used fairs metric.

​

​

​

​

• Fairness notions such as fair use and predictive multiplicity are still mostly unexplored, but very relevant to practice.

​

​

​

​

​

• Information-theoretic tools can be used to derive converse results on auditing and ensuring fairness, as well as limits on fairness-accuracy trade-offs.

​

Up next: ensuring fairness in practice

Thanks!