Individual Fairness for Graph Neural Networks: A Ranking based Approach
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
1Yaochen Zhu 1Yinhan He 2Jing Ma 1 Mengxuan Hu 1Sheng Li 1Jundong Li
1University of Virginia
2Case Western Reserve University
Causal Inference with Latent Variables:
Recent Advances & Future Prospectives
1
About This Tutorial
2
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Outline
3
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Background and Causal Inference Basics
Counterfactual Analysis
Challenges & Future Directions
Latent Confounding Analysis
Generalization to Graphs
Latent Mediation Analysis
Overview of Causal Inference
4
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Overview of Causal Inference
5
such as:
Medicine
Politics
Economy
Clinical trial
Geo-political influence
Econ-factors
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Overview of Causal Inference
6
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Overview of Causal Inference
7
attend this tutorial (T)
research productivity (Y)
For example, you may wonder about the following:
Jundong
Yaochen
* You
influence
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Overview of Causal Inference
8
attend this tutorial (T)
research productivity (Y)
Jundong
Yaochen
Hopefully, the causal effect of T on Y is positive…
influence
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
For example, you may wonder about the following:
* You
Causal Inference Tasks
9
Estimate the causal effect of a treatment variable T on an outcome Y
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference Tasks
10
Estimate the causal effect of a treatment variable T on an outcome Y
Jundong
Yaochen
attending this tutorial (T = 1)
ignore this tutorial (T = 0)
compare
with the same group
average research productivity
average research productivity
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference Tasks
11
Jundong
Yaochen
attending this tutorial (T = 1)
ignore this tutorial (T = 0)
compare
X = Ph.D. student
X = Professors
X = Area Chair
with the same sub-group
Estimate the causal effect of a treatment variable T on an outcome Y
average research productivity
average research productivity
X = Ph.D. student
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference Tasks
12
The causal effect of a treatment variable T on an outcome Y
mediated via some other variables M
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference Tasks
13
The causal effect of a treatment variable T on an outcome Y
mediated via some other variables M
attend this tutorial (T)
research productivity (Y)
Jundong
Yaochen
M1 = Jundong’s part
M2 = Yaochen’s part
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference Tasks
14
The causal effect of a treatment variable T on an outcome Y
mediated via some other variables M
attend this tutorial (T)
research productivity (Y)
Jundong
Yaochen
M1 = Jundong’s part
M2 = Yaochen’s part
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
CMA considers fine-grained causal effect along specific causal chains
Causal Inference Tasks
15
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The effects of treatment T on outcome Y had X been x’
Causal Inference Tasks
16
Jundong
Yaochen
attended this tutorial
What if I had not attended this tutorial?
(For T=1, what Y would be had T been 0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The effects of treatment T on outcome Y had X been x’
Causal Inference Tasks
17
Jundong
Yaochen
attended this tutorial
What if I had not attended this tutorial?
(For T=1, what Y would be had T been 0)
Similarly, we can study more fine-grained counterfactuals
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The effects of treatment T on outcome Y had X been x’
Causal Inference Tasks
18
Jundong
Yaochen
M1 = Jundong’s part
M2 = Yaochen’s part
attended this tutorial
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The effects of treatment T on outcome Y had X been x’
Causal Inference Tasks
19
Jundong
Yaochen
M1 = Jundong’s part
M2 = Yaochen’s part
What if Judea Pearl had given Yaochen’s part?
M2 = Judea Pearl
’
attended this tutorial
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The effects of treatment T on outcome Y had X been x’
Causal Inference Tasks
20
Jundong
Yaochen
M1 = Jundong’s part
M2 = Yaochen’s part
What if Judea Pearl had given Yaochen’s part?
M2 = Judea Pearl
’
attended this tutorial
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
your research
The effects of treatment T on outcome Y had X been x’
Fundamental Challenge of Causal Inference
21
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
Professors
22
Jundong
Yaochen
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
compare
with the same group
parallel universe 1
parallel universe 2
Ph.D. students
Area Chairs
Professors
Area Chairs
Ph.D. students
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e.,
Fundamental Challenge of Causal Inference
Professors
23
Jundong
Yaochen
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
compare
with the same group
parallel universe 1
parallel universe 2
Ph.D. students
Area Chairs
Professors
Area Chairs
Ph.D. students
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e.,
Fundamental Challenge of Causal Inference
Professors
24
Jundong
Yaochen
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
However, this is fundamentally impossible in real world…
compare
with the same group
parallel universe 1
parallel universe 2
Ph.D. students
Area Chairs
Professors
Area Chairs
Ph.D. students
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e.,
Fundamental Challenge of Causal Inference
25
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
26
real universe
treatment group (T=1)
Jundong
Yaochen
Ph.D. students
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
27
real universe
real universe
treatment group (T=1)
non-treatment group (T=0)
Jundong
Yaochen
Ph.D. students
Professors
Area Chairs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
28
Comparable?
Different units
real universe
real universe
treatment group (T=1)
non-treatment group (T=0)
Jundong
Yaochen
Ph.D. students
Professors
Area Chairs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
29
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
30
Y before
Y before
real universe
real universe
treatment group (T=1)
non-treatment group (T=0)
Jundong
Yaochen
Ph.D. students
Professors
Area Chairs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
31
Y after
Y before
Y before
Y after
real universe
real universe
treatment group (T=1)
non-treatment group (T=0)
Jundong
Yaochen
Ph.D. students
Professors
Area Chairs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fundamental Challenge of Causal Inference
32
Y after
Y before
Y before
Y after
real universe
real universe
treatment group (T=1)
non-treatment group (T=0)
Jundong
Yaochen
Ph.D. students
Professors
Area Chairs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
group-wise, attending tutorial helps your productivity (which makes us happy)
Fundamental Challenge of Causal Inference
33
Y after
Y before
Y before
Y after
unobserved
unobserved
comparison across group leads us to believe that tutorial hurts productivity
treatment group (T=1)
non-treatment group (T=0)
real universe
real universe
Jundong
Yaochen
Ph.D. students
Professors
Area Chairs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference in a Nutshell
34
to reason with causal relations
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal Inference in a Nutshell
35
to show that under certain conditions, causal reasoning can be derived from observed statistical relations
to reason with causal relations
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Common Definitions
36
the atomic research object in the study.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Common Definitions
37
(the cause we wanna study)
the atomic research object in the study.
an action that applies to a unit.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
attend causal inference tutorial
Common Definitions
38
(the cause we wanna study)
(the effect we are interested in)
the atomic research object in the study.
an action that applies to a unit.
response of the units after treatment.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
attend causal inference tutorial
I have a good idea
Rubin’s Causal Model (RCM)
39
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
40
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
41
Jundong
Yaochen
attend this tutorial (T = 1)
real universe for i
Ti = 1
Yi (1)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
42
Jundong
Yaochen
attend this tutorial (T = 1)
real universe for i
Ti = 1
Yi (1)
Jundong
Yaochen
attend this tutorial (T = 1)
real universe for i
leave half-way
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
43
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
def. the outcome Y if treatment t is applied on unit i.
Rubin’s Causal Model (RCM)
real universe for i
44
def. the outcome Y if treatment t is applied on unit i.
Jundong
Yaochen
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
i
i
Yi(1)
Yi(0)
parallel universe 2
e.g., in our previous example,
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
real universe for i
45
Jundong
Yaochen
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
i
i
parallel universe 2
potential outcomes Yi(1) and Yi(0) cannot be simultaneously observed
e.g., in our previous example,
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Yi(1)
Yi(0)
def. the outcome Y if treatment t is applied on unit i.
Rubin’s Causal Model (RCM)
46
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
def. the actual outcome Y on unit i under observed treatment.
Rubin’s Causal Model (RCM)
47
Jundong
Yaochen
i
i
Yi(0)
real universe for i
counterfactual universe for i
e.g., in our previous example, if unit i attends the tutorial
unobserved
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Yi
def. the actual outcome Y on unit i under observed treatment.
Rubin’s Causal Model (RCM)
48
Jundong
Yaochen
i
i
Yi(0)
real universe for i
counterfactual universe for i
Yi(t) and Yi reason with outcome from an individual perspective
e.g., in our previous example, if unit i attends the tutorial
unobserved
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Yi
def. the actual outcome Y on unit i under observed treatment.
Rubin’s Causal Model (RCM)
49
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
def. distribution of outcome Y if treatment t is uniformly applied
Rubin’s Causal Model (RCM)
50
Jundong
Yaochen
Y (1)
parallel universe 1
parallel universe 2
e.g., in our previous example,
Y (0)
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
def. distribution of outcome Y if treatment t is uniformly applied
Rubin’s Causal Model (RCM)
51
Jundong
Yaochen
Y (1)
parallel universe 1
parallel universe 2
Y(T=t) cannot be measured due to unobserved potential outcomes
e.g., in our previous example,
Y (0)
attend this tutorial (T = 1)
ignore this tutorial (T = 0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
def. distribution of outcome Y if treatment t is uniformly applied
Rubin’s Causal Model (RCM)
52
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
53
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
54
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
55
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
56
Question: How to estimate the above causal estimands?
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
57
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
58
Jundong
real universe
real universe
average Y for treated students
average Y for non-treated professors
Ph.D. students
Professors
Area Chairs
treatment group (T=1)
Yaochen
non-treatment group (T=0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Rubin’s Causal Model (RCM)
59
Jundong
real universe
real universe
average Y for treated students
average Y for non-treated professors
Ph.D. students
Professors
Area Chairs
treatment group (T=1)
Yaochen
non-treatment group (T=0)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
There are systematic differences between the treatment/non-treatment group
Rubin’s Causal Model (RCM)
60
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Jundong
real universe
real universe
average Y for treated students
average Y for non-treated professors
With no further assumptions, the naïve estimator is biased!!
Ph.D. students
Professors
Area Chairs
treatment group (T=1)
Yaochen
non-treatment group (T=0)
Rubin’s Causal Model (RCM)
61
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Treatment assignment cannot be deterministic
P(T = 1, 0|X = x) > 0 for all X = x
Rubin’s Causal Model (RCM)
62
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Treatment assignment cannot be deterministic
P(T = 1, 0|X = x) > 0 for all X = x
Jundong
real universe
Ph.D. students
Professors
Area Chairs
treatment group (T=1)
Yaochen
non-treatment group (T=0)
Bad universe that violates the positivity assumption!
real universe
Rubin’s Causal Model (RCM)
63
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Jundong
Yaochen
Ph.D. students
real universe
Professors
treatment group (T=1)
non-treatment group (T=0)
real universe
Area Chairs
Area Chairs
Professors
Treatment assignment cannot be deterministic
P(T = 1, 0|X = x) > 0 for all X = x
Ph.D. students
Rubin’s Causal Model (RCM)
64
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Jundong
Yaochen
Ph.D. students
real universe
Professors
treatment group (T=1)
non-treatment group (T=0)
real universe
Area Chairs
Area Chairs
Professors
Treatment assignment cannot be deterministic
P(T = 1, 0|X = x) > 0 for all X = x
Ph.D. students
Difference between two groups is alleviated, as student/AC/Prof. are in both groups
Rubin’s Causal Model (RCM)
65
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
PO R.V. Y(T) are independent of treatment assignment T
Y(t) ⊥ T, for all T = t
Rubin’s Causal Model (RCM)
66
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
real universe
treatment group (T=1)
non-treatment group (T=0)
real universe
PO R.V. Y(T) are independent of treatment assignment T
Y(t) ⊥ T, for all T = t
Ph.D. students
Area Chairs
Professors
Professors
Area Chairs
Ph.D. students
Jundong
Yaochen
Rubin’s Causal Model (RCM)
67
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
real universe
treatment group (T=1)
non-treatment group (T=0)
real universe
PO R.V. Y(T) are independent of treatment assignment T
Y(t) ⊥ T, for all T = t
Ph.D. students
Area Chairs
Professors
Professors
Area Chairs
Ph.D. students
Jundong
Yaochen
Difference between two groups is vanished!!
Rubin’s Causal Model (RCM)
68
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
real universe
treatment group (T=1)
non-treatment group (T=0)
real universe
PO R.V. Y(T) are independent of treatment assignment T
Y(t) ⊥ T, for all T = t
Ph.D. students
Area Chairs
Professors
Professors
Area Chairs
Ph.D. students
Jundong
Yaochen
Problem: too strong…conditional version introduced later…
Structural Causal Model (SCM)
69
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Structural Causal Model (SCM)
70
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. A direct acyclic graph (DAG) that encodes the assumed causal relations among variables of interest
Structural Causal Model (SCM)
71
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
(a) Chain
T
M
Y
tutorial
presenter
research productivity
Def. A direct acyclic graph (DAG) that encodes the assumed causal relations among variables of interest
Structural Causal Model (SCM)
72
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
(a) Chain
(b) Fork
T
M
Y
T
Y
C
tutorial
presenter
research productivity
tutorial
research productivity
knowledge level of audience
Def. A direct acyclic graph (DAG) that encodes the assumed causal relations among variables of interest
Spuriously correlates T and Y if C is NOT considered!!
Structural Causal Model (SCM)
73
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. A direct acyclic graph (DAG) that encodes the assumed causal relations among variables of interest
Spuriously correlates T and Y if M is conditioned on!!
(b) Fork
(c) V-structure
T
Y
C
Y
T
M
tutorial
research productivity
knowledge level of audience
tutorial
research productivity
# papers
(a) Chain
T
M
Y
tutorial
presenter
research productivity
Structural Causal Model (SCM)
74
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Variables out of the study that influence variables in the study
(i.e., endogenous variables)
Structural Causal Model (SCM)
75
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Variables out of the study that influence variables in the study
(a) population level
T
Y
C
tutorial
research productivity
knowledge level of audience
(i.e., endogenous variables)
Structural Causal Model (SCM)
76
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Variables out of the study that influence variables in the study
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
T
Y
C
tutorial
research productivity
knowledge level of audience
(b) individual level
(a) population level
U1
U2
U3
Structural Causal Model (SCM)
77
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Variables out of the study that influence variables in the study
U are important to consider for individual treatment effects!
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
T
Y
C
tutorial
research productivity
knowledge level of audience
(b) individual level
(a) population level
U1
U2
U3
Structural Causal Model (SCM)
78
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Quantify the causal influence of causal parents to the child node
Structural Causal Model (SCM)
79
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Quantify the causal influence of causal parents to the child node
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
U1
U2
U3
qualify causal relations
Structural Causal Model (SCM)
80
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Quantify the causal influence of causal parents to the child node
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
U1
U2
U3
qualify causal relations
quantify causal influence
Structural Causal Model (SCM)
81
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Quantify the causal influence of causal parents to the child node
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
U1
U2
U3
qualify causal relations
quantify causal influence
remark#1. structural equations represent the underlying physical world
Structural Causal Model (SCM)
82
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Quantify the causal influence of causal parents to the child node
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
U1
U2
U3
qualify causal relations
quantify causal influence
remark#2. structural equations usually needs to be estimated from data
Structural Causal Model (SCM)
83
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Study the causal influence of treatment node on the target node
Structural Causal Model (SCM)
84
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Study the causal influence of treatment node on the target node
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
U1
U2
U3
Structural Causal Model (SCM)
85
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Study the causal influence of treatment node on the target node
do(T=1)
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
U2
U3
your mood
U1
Structural Causal Model (SCM)
86
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Study the causal influence of treatment node on the target node
Y
C
research productivity
knowledge level of audience
your perseverance
your previous education
U2
U3
force everyone to attend the tutorial
your mood
U1
regardless of the mood, previous education, etc.
do(T=1)
tutorial
Structural Causal Model (SCM)
87
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Study the causal influence of treatment node on the target node
Y
C
research productivity
knowledge level of audience
your perseverance
your previous education
U2
U3
force everyone to attend the tutorial
your mood
U1
regardless of the mood, previous education, etc.
do(T=1)
tutorial
Latent Variables in Causal Inference
88
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Variables in Causal Inference
89
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Variables in Causal Inference
90
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between T and Y
Latent Variables in Causal Inference
91
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between T and Y
T
Y
C
tutorial
research productivity
knowledge level of audience
T
Y
tutorial
research productivity
(a) no confounder
(b) unobserved confounder
Latent Variables in Causal Inference
92
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level of audience
C may not be observed if we don’t have the attendant demographical data
T
Y
tutorial
research productivity
(a) no confounder
(b) unobserved confounder
confounders C leads to spurious correlation between T and Y
Latent Variables in Causal Inference
93
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between
T
M
Y
tutorial
different parts
research productivity
Latent Variables in Causal Inference
94
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between
T
M
Y
tutorial
different parts
research productivity
C
research interest
Determines whether or not you will attend it
Latent Variables in Causal Inference
95
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between
T
M
Y
tutorial
different parts
research productivity
C
research interest
Determines which part you pay most attention to
Determines whether or not you will attend it
Latent Variables in Causal Inference
96
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between
T
M
Y
tutorial
different parts
research productivity
C
research interest
Determines which part you pay most attention to
Determines also research productivity
Determines whether or not you will attend it
Latent Variables in Causal Inference
97
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounders C leads to spurious correlation between
T
M
Y
tutorial
different parts
research productivity
C
research interest
Determines which part you pay most attention to
Determines also research productivity
C may not be observed if we don’t have the attendant background data
Determines whether or not you will attend it
Latent Variables in Causal Inference
98
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Individual factors (i.e., exogenous variables) are usually not observed
Latent Variables in Causal Inference
99
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Individual factors (i.e., exogenous variables) are usually not observed
T
Y
C
tutorial
research productivity
knowledge level of audience
your perseverance
your previous education
your mood
Latent Variables in Causal Inference
100
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Variables in Causal Inference
101
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Show that under certain conditions, we can eschew the latent variables
Latent Variables in Causal Inference
102
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Show that under certain conditions, we can eschew the latent variables
T
Y
C
tutorial
research productivity
knowledge level of audience
T
Y
tutorial
research productivity
I
draw lots
e.g.,
Latent Variables in Causal Inference
103
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Since T is randomly assigned, C cannot be a causal parent of T
Show that under certain conditions, we can eschew the latent variables
T
Y
C
tutorial
research productivity
knowledge level of audience
T
Y
tutorial
research productivity
I
draw lots
e.g.,
Latent Variables in Causal Inference
104
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Infer the latent variables from other observed covariates
Latent Variables in Causal Inference
105
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Infer the latent variables from other observed covariates
T
Y
C
tutorial
research productivity
knowledge level of audience
E.g.,
Latent Variables in Causal Inference
106
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Infer the latent variables from other observed covariates
T
Y
C
tutorial
research productivity
knowledge level of audience
E.g.,
Age could be an indicator for the knowledge level!
Outline
Background and Causal Inference Basics
Counterfactual Analysis
Latent Confounding Analysis
107
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Challenges & Future Directions
Generalization to Graphs
Background Knowledge
108
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Covariates that simultaneously affect the treatment T and outcome Y
Background Knowledge
109
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Covariates that simultaneously affect the treatment T and outcome Y
T
Y
C
tutorial
research productivity
knowledge level of audience
Background Knowledge
110
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Covariates that simultaneously affect the treatment T and outcome Y
T
Y
C
tutorial
research productivity
knowledge level of audience
treatment group (T=1)
non-treatment group (T=0)
Background Knowledge
111
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Covariates that simultaneously affect the treatment T and outcome Y
T
Y
C
tutorial
research productivity
knowledge level of audience
treatment group (T=1)
non-treatment group (T=0)
lead to systematic difference between the treatment/non-treatment groups
Professors
Area chair
Area chairs
Professors
Ph.D. level
Ph.D. level
Traditional Methods
112
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
Traditional Methods
113
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
research productivity
knowledge level
(one possible case)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
Traditional Methods
114
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
research productivity
knowledge level
giving you an exam before you enter this tutorial
measured by
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
(one possible case)
Traditional Methods
115
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
(one possible case)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
Traditional Methods
116
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Wholistically, the treatment are not randomly assigned
T
Y
X
tutorial
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
(one possible case)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
Traditional Methods
117
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
For each sub-population, treatment are randomly assigned (same color)
T
Y
X
tutorial
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
(one possible case)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
Traditional Methods
118
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
For each sub-population, treatment are randomly assigned (same color)
(with possibly different probability of treatment assignment)
T
Y
X
tutorial
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
(one possible case)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
Traditional Methods
119
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Expected Y(0) for sub-treatment group should be same as sub-non-treatment group
T
Y
X
tutorial
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
(one possible case)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
E[Y(0)|x]
E[Y(0)|x]
Traditional Methods
120
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Expected Y(1) for sub-non-treatment group should be same as sub-treatment group
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
T
Y
X
tutorial
(one possible case)
knowledge level
research productivity
E[Y(1)|x]
E[Y(1)|x]
Traditional Methods
121
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
Traditional Methods
122
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
Traditional Methods
123
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
C
seniority level
Traditional Methods
124
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
Ph.D. level
Professors
Ph.D. level
Area chair
Area chairs
Professors
treatment group (T=1)
non-treatment group (T=0)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
C
seniority level
Traditional Methods
125
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
Ph.D. level
APs
Senior Ph.D.
Area chair
SAC
Full Prof.
treatment group (T=1)
non-treatment group (T=0)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
C
seniority level
Traditional Methods
126
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
X
tutorial
Ph.D. level
APs
Senior Ph.D.
Area chair
SAC
Full Prof.
treatment group (T=1)
non-treatment group (T=0)
Y(t) are independent of treatment T given observed covariates X
Y(t) ⊥ T | X = x, for all T = t, X=x
knowledge level
research productivity
C
seniority level
Conditional on X CANNOT eliminate the systematic difference between two groups
Traditional Methods
127
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Traditional Methods
128
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Controlling for the effects of observed confounders
Traditional Methods
129
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
Traditional Methods
130
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
missing data from T = 0
Controlling for the effects of observed confounders
Traditional Methods
131
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
missing data from T = 1
Controlling for the effects of observed confounders
Traditional Methods
132
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
Traditional Methods
133
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
(we first ignore the outer E, and focus on the inter with X = Ph.D. student)
Traditional Methods
134
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
missing data from T = 0
Controlling for the effects of observed confounders
(we first ignore the outer E, and focus on the inter with X = Ph.D. student)
Traditional Methods
135
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
missing data from T = 1
Controlling for the effects of observed confounders
Traditional Methods
136
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
Traditional Methods
137
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
missing data from T = 0
Controlling for the effects of observed confounders
Traditional Methods
138
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
Traditional Methods
139
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
missing data from T = 1
Controlling for the effects of observed confounders
Traditional Methods
140
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
Traditional Methods
141
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
Controlling for the effects of observed confounders
Traditional Methods
142
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
is just the average observed Y for T=1
Controlling for the effects of observed confounders
Traditional Methods
143
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
chairs
treatment group (T=1)
non-treatment group (T=0)
Prof.
Ph.D.
is just the average observed Y for T=0
Controlling for the effects of observed confounders
Traditional Methods
144
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Ph.D. effects
chairs effects
treatment group (T=1)
non-treatment group (T=0)
Prof. effects
（weighted average）
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
Controlling for the effects of observed confounders
Traditional Methods
145
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Ph.D. effects
chairs effects
treatment group (T=1)
non-treatment group (T=0)
Prof. effects
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
Causal estimands are reduced to correlations measurable in dataset
Controlling for the effects of observed confounders
（weighted average）
Traditional Methods
146
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
(a) linear regression
Causal estimands are reduced to correlations measurable in dataset
Controlling for the effects of observed confounders
Traditional Methods
147
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
(a) linear regression
(b) trees
Causal estimands are reduced to correlations measurable in dataset
Controlling for the effects of observed confounders
Traditional Methods
148
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（the definition of ATE）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（consistency）
(a) linear regression
(b) trees
(c) deep neural networks
Causal estimands are reduced to correlations measurable in dataset
Controlling for the effects of observed confounders
Traditional Methods
149
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Traditional Methods
150
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
Traditional Methods
151
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
i.e., the probability treatment is assigned for sub-population X = x
Traditional Methods
152
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
Traditional Methods
153
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
expected Y(T=1) for the whole population
treatment group (T=1)
non-treatment group (T=0)
Ph.D.
Area Chair
missing data from T = 0
Traditional Methods
154
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
reweighted observed Y for the treatment group
treatment group (T=1)
non-treatment group (T=0)
Ph.D.
Area Chair
Traditional Methods
155
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
Traditional Methods
156
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
treatment group (T=1)
non-treatment group (T=0)
Ph.D.
Area Chair
Profs.
e = 3/4
e = 1/3
e = 1/4
Traditional Methods
157
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
treatment group (T=1)
non-treatment group (T=0)
Ph.D.
Area Chair
Profs.
e = 3/4
e = 1/3
e = 1/4
reweight by 4/3
Traditional Methods
158
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
non-treatment group (T=0)
Ph.D.
Area Chair
Profs.
treatment group (T=1)
Traditional Methods
159
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Def. Propensity Score: e = p(T | X = x)
non-treatment group (T=0)
Ph.D.
Area Chair
Profs.
treatment group (T=1)
Traditional Methods
160
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（binary treatment + consistency）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（definition of propensity score）
（law of total expectation）
Def. Propensity Score: e = p(T | X = x)
non-treatment group (T=0)
Ph.D.
Area Chair
Profs.
treatment group (T=1)
Traditional Methods
161
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
（binary treatment + consistency）
（law of total expectation）
（conditional ignorability, Y(t) ⊥ T | X = x）
（definition of propensity score）
（law of total expectation）
Def. Propensity Score: e = p(T | X = x)
non-treatment group (T=0)
Ph.D.
Area Chair
Profs.
treatment group (T=1)
e can be estimated via linear regression, trees, and deep neural networks
Question
162
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e., some confounders are not observed in X
Question
163
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e., some confounders are not observed in X
x
treatment group (T=1)
non-treatment group (T=0)
age: young
age: old
age: young
age: old
observed covariates X
Question
164
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e., some confounders are not observed in X
The population
Ph.D.
Area Chair
Profs.
treatment group (T=1)
non-treatment group (T=0)
age: young
age: old
age: young
age: old
select
select
observed covariates X
true confounders C
Question
165
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
i.e., some confounders are not observed in X
The population
Ph.D.
Area Chair
Profs.
treatment group (T=1)
non-treatment group (T=0)
age: young
age: old
age: young
age: old
select
select
In each sub-population specified by X = x, systematic difference still exists
Circumvention-based Method
166
show that under certain conditions, we can eschew the latent variable
while obtain unbiased estimate of the causal estimands
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Circumvention-based Method
167
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Circumvention-based Method
168
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The naïve estimator ER[Y|T=1] - ER[Y|T=0] is unbiased for ATE
randomized data distribution
Circumvention-based Method
169
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The population
treatment group (T=1)
non-treatment group (T=0)
age: young
The naïve estimator ER[Y|T=1] - ER[Y|T=0] is unbiased for ATE
randomized data distribution
However, the size of randomized data could be very small:
age: old
age: old
age: young
Ph.D.
Area Chair
Profs.
Circumvention-based Method
170
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
However, the size of randomized data could be very small:
The population
treatment group (T=1)
non-treatment group (T=0)
random sample
random sample
The naïve estimator ER[Y|T=1] - ER[Y|T=0] is unbiased for ATE
randomized data distribution
age: young
age: young
age: old
age: old
Ph.D.
Area Chair
Profs.
Circumvention-based Method
171
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
However, the size of randomized data could be very small:
The population
treatment group (T=1)
non-treatment group (T=0)
random sample
random sample
The naïve estimator ER[Y|T=1] - ER[Y|T=0] is unbiased for ATE
randomized data distribution
age: young
age: young
age: old
age: old
Ph.D.
Area Chair
Profs.
Circumvention-based Method
172
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
However, the size of randomized data could be very small:
The population
treatment group (T=1)
non-treatment group (T=0)
random sample
random sample
The naïve estimator ER[Y|T=1] - ER[Y|T=0] is unbiased for ATE
randomized data distribution
age: young
age: old
age: young
age: old
The variance of the estimator will be very large…
Ph.D.
Area Chair
Profs.
Circumvention-based Method
173
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
A large observational data DO with unobserved confounders
A small randomized data DR
Problem Setting:
Circumvention-based Method
174
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
A large observational data DO with unobserved confounders
A small randomized data DR
randomized treatment group (T=1)
treatment group (T=1)
Problem Setting:
X = young
X = old
X = young
X = old
Circumvention-based Method
175
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
fit a base estimator y = fO(x, t)
A large observational data DO with unobserved confounders
A small randomized data DR
randomized treatment group (T=1)
treatment group (T=1)
Problem Setting:
X = young
X = old
X = young
X = old
Kallus, Nathan, Aahlad Manas Puli, and Uri Shalit. "Removing hidden confounding by experimental grounding." Advances in neural information processing systems 31 (2018).
Circumvention-based Method
176
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
A large observational data DO with unobserved confounders
A small randomized data DR
randomized treatment group (T=1)
treatment group (T=1)
Problem Setting:
X = young
X = old
X = young
X = old
due to selection on true confounder C
fit a base estimator y = fO(x, t)
It is biased for E [Y(t) | X=x]
Circumvention-based Method
177
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
evaluate the residual of fO(x, t) on DO
fit a base estimator y = fO(x, t)
It is biased for E [Y(t) | X=x]
A large observational data DO with unobserved confounders
A small randomized data DR
randomized treatment group (T=1)
treatment group (T=1)
Problem Setting:
X = young
X = old
X = young
X = old
due to selection on true confounder C
Circumvention-based Method
178
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
evaluate the residual of fO(x, t) on DO
fit fR(x, t) on the residual
A large observational data DO with unobserved confounders
A small randomized data DR
randomized treatment group (T=1)
treatment group (T=1)
Problem Setting:
X = young
X = old
X = young
X = old
fit a base estimator y = fO(x, t)
It is biased for E [Y(t) | X=x]
due to selection on true confounder C
Circumvention-based Method
179
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
fO(x, t) + fR(x, t) is an unbiased estimator for E [Y(t) | X=x]
fit fR(x, t) on the residual
A large observational data DO with unobserved confounders
A small randomized data DR
randomized treatment group (T=1)
treatment group (T=1)
Problem Setting:
X = young
X = old
X = young
X = old
fit a base estimator y = fO(x, t)
It is biased for E [Y(t) | X=x]
due to selection on true confounder C
evaluate the residual of fO(x, t) on DO
Circumvention-based Method
180
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
If residual is small, estimation variance can be substantially reduced…
fit fR(x, t) on the residual
randomized treatment group (T=1)
treatment group (T=1)
X = young
X = old
X = young
X = old
fit a base estimator y = fO(x, t)
It is biased for E [Y(t) | X=x]
due to selection on true confounder C
A large observational data DO with unobserved confounders
A small randomized data DR
Problem Setting:
evaluate the residual of fO(x, t) on DO
Circumvention-based Method
181
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Circumvention-based Method
182
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(i) has no confounding with the outcome Y
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Circumvention-based Method
183
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(i) has no confounding with the outcome Y
(ii) affects the treatment T (relevance)
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Circumvention-based Method
184
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(iii) affects the outcome 𝑌 only through 𝑇 (restriction)
(i) has no confounding with the outcome Y
(ii) affects the treatment T (relevance)
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Circumvention-based Method
185
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(iii) affects the outcome 𝑌 only through 𝑇 (restriction)
(i) has no confounding with the outcome Y
(ii) affects the treatment T (relevance)
T
Y
C
I
X
advisor randomly coerce Ph.D. to attend tutorial based on their mood
treatment
outcome
unobserved confounder
instrumental variable
observed covariates
IV (advisor coercion):
Circumvention-based Method
186
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(iii) affects the outcome 𝑌 only through 𝑇 (restriction)
(i) has no confounding with the outcome Y
(ii) affects the treatment T (relevance)
T
Y
C
I
X
treatment
outcome
unobserved confounder
instrumental variable
observed covariates
whether or not you wanna listen to them by attending this tutorial
T (attend the tutorial):
advisor randomly coerce Ph.D. to attend tutorial based on their mood
IV (advisor coercion):
Circumvention-based Method
187
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(iii) affects the outcome 𝑌 only through 𝑇 (restriction)
(i) has no confounding with the outcome Y
(ii) affects the treatment T (relevance)
T
Y
C
I
X
treatment
outcome
unobserved confounder
instrumental variable
observed covariates
whether or not you wanna listen to them by attending this tutorial
T (attend the tutorial):
advisor randomly coerce Ph.D. to attend tutorial based on their mood
IV (advisor coercion):
compliance to treatment (I -> T measures intention for treatment)
Circumvention-based Method
188
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
IV is a variable that
(iii) affects the outcome 𝑌 only through 𝑇 (restriction)
(i) has no confounding with the outcome Y
(ii) affects the treatment T (relevance)
T
Y
C
I
X
treatment
outcome
unobserved confounder
instrumental variable
observed covariates
whether or not you wanna listen to them by attending this tutorial
T (attend the tutorial):
advisor randomly coerce Ph.D. to attend tutorial based on their mood
IV (advisor coercion):
The effect between T and Y is what we really wanna estimate…
Circumvention-based Method
189
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The two stage algorithm
Circumvention-based Method
190
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
The two stage algorithm
Circumvention-based Method
191
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Y = f(X, T) + C
（additive confounders）
(1)
The two stage algorithm
Circumvention-based Method
192
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Y = f(X, T) + C
（additive confounders）
E[Y|X, I] = E[f(X, T)|X, I] +E[C|X]
(1)
（taking expectation of Eq. 1）
The two stage algorithm
Circumvention-based Method
193
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Y = f(X, T) + C
（additive confounders）
E[Y|X, I] = E[f(X, T)|X, I] +E[C|X]
(1)
（taking expectation of Eq. 1）
（law of total expectation）
The two stage algorithm
Circumvention-based Method
194
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Y = f(X, T) + C
（additive confounders）
E[Y|X, I] = E[f(X, T)|X, I] +E[C|X]
(1)
（taking expectation of Eq. 1）
（law of total expectation）
（change of name）
The two stage algorithm
Circumvention-based Method
195
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
treatment
outcome
unobserved confounder
I
instrumental variable
observed covariates
X
Y = f(X, T) + C
（additive confounders）
E[Y|X, I] = E[f(X, T)|X, I] +E[C|X]
(1)
（taking expectation of Eq. 1）
（law of total expectation）
（change of name）
The two stage algorithm
exactly the causal estimand we are interested in
Circumvention-based Method
196
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
From the above derivation we have:
a new T conditional on X, I
causal estimand
The two stage algorithm
observed Y
Circumvention-based Method
197
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The relation between Y, X, I, T are linear
From the above derivation we have:
a new T conditional on X, I
causal estimand
The two stage algorithm
observed Y
Circumvention-based Method
198
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The relation between Y, X, I, T are linear
From the above derivation we have:
a new T conditional on X, I
causal estimand
The two stage algorithm
observed Y
Circumvention-based Method
199
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The relation between Y, X, I, T are linear
From the above derivation we have:
observed Y
a new T conditional on X, I
causal estimand
The two stage algorithm
Circumvention-based Method
200
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The relation between Y, X, I, T are linear
From the above derivation we have:
observed Y
a new T conditional on X, I
causal estimand
The two stage algorithm
Circumvention-based Method
201
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The relation between Y, X, I, T are linear
From the above derivation we have:
observed Y
a new T conditional on X, I
causal estimand
The two stage algorithm
Issue #1: Instrumental variables are difficult to obtain
Circumvention-based Method
202
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The relation between Y, X, I, T are linear
From the above derivation we have:
observed Y
a new T conditional on X, I
causal estimand
The two stage algorithm
Issue #2: Weak IV leads to high estimation variance
This will be very small
Circumvention-based Method
203
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Circumvention-based Method
204
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
smoke
lung cancer
T
Circumvention-based Method
205
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
smoke
lung cancer
T
C
smoking gene
Circumvention-based Method
206
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
C
smoke
lung cancer
T
M
smoking gene
tar deposit
Circumvention-based Method
207
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
C
smoke
lung cancer
T
M
smoking gene
tar deposit
tar deposit satisfy the front-door criterion!
Circumvention-based Method
208
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
C
smoke
lung cancer
T
M
smoking gene
tar deposit
tar deposit satisfy the front-door criterion!
p(M | do(T)) = p(M | T)
causal influence from T to M:
Circumvention-based Method
209
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
C
smoke
lung cancer
T
M
smoking gene
tar deposit
tar deposit satisfy the front-door criterion!
p(M | do(T)) = p(M | T)
causal influence from T to M:
causal influence from M to Y:
As Y <- C -> T -> M forms a backdoor path
Circumvention-based Method
210
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
ATE can be identified with known, unconfounded, sufficient mediator
Y
C
smoke
lung cancer
T
M
smoking gene
tar deposit
tar deposit satisfy the front-door criterion!
the treatment effect can be calculated as
p(M | do(T)) = p(M | T)
causal influence from T to M:
causal influence from M to Y:
As Y <- C -> T -> M forms a backdoor path
Inference-based Method
211
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
show that under certain conditions, latent confounders can be inferred
from other observed variables
Inference-based Method
212
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Inference-based Method
213
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
unobserved confounders
confounder proxies
General analysis of proxy of confounders
Inference-based Method
214
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
unobserved confounders
confounder proxies
General analysis of proxy of confounders
age for knowledge level
Inference-based Method
215
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
unobserved confounders
confounder proxies
General analysis of proxy of confounders
ATE can be unbiasedly estimated if
p(T, Y, C, X) can be identified.
Inference-based Method
216
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
unobserved confounders
confounder proxies
（law of total expectation）
General analysis of proxy of confounders
ATE can be unbiasedly estimated if
p(T, Y, C, X) can be identified.
Inference-based Method
217
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
unobserved confounders
confounder proxies
（law of total expectation）
（conditional ignorability）
General analysis of proxy of confounders
ATE can be unbiasedly estimated if
p(T, Y, C, X) can be identified.
Inference-based Method
218
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
unobserved confounders
confounder proxies
（law of total expectation）
（conditional ignorability）
Both p(Y|X, T, C) and p(C, T|X) can be derived from p(T, Y, C, X)
General analysis of proxy of confounders
ATE can be unbiasedly estimated if
p(T, Y, C, X) can be identified.
Inference-based Method
219
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal effect variational auto-encoder (CEVAE)
Inference-based Method
220
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Non-treatment branch
Treatment branch
Covariate
branch
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
Treatment prediction
T
Y
C
X
Inference-based Method
221
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal effect variational auto-encoder (CEVAE)
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
T
Y
C
X
Inference-based Method
222
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
T
Y
C
X
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
Inference-based Method
223
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
Treatment prediction
T
Y
C
X
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
Inference-based Method
224
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Non-treatment branch
Treatment branch
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
Treatment prediction
T
Y
C
X
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
Inference-based Method
225
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Non-treatment branch
Treatment branch
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
Treatment prediction
T
Y
C
X
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
two-branch network to avoid ignoring the T information
Inference-based Method
226
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Non-treatment branch
Treatment branch
Covariate
branch
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
Treatment prediction
T
Y
C
X
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
Inference-based Method
227
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Non-treatment branch
Treatment branch
Covariate
branch
latent confounder C
P(T | C)
P(X | C)
P(Y | C, T=0)
P(Y | C, T=1)
Causal effect variational auto-encoder (CEVAE)
Treatment prediction
T
Y
C
X
p(T, Y, C, X) = p(C) ⨉ p(T | C) ⨉ p(Y | C, T) ⨉ p(X | C)
The generation network can be trained with observed (X, T, Y)
Inference-based Method
228
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal effect variational auto-encoder (CEVAE)
Q(C|T, Y, X) - > variational posterior
Inference-based Method
229
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounder proxies X
Q(T|X)
Q(C|Y, T=0, X)
Q(C|Y, T=0, X)
Non-treatment branch
Treatment branch
Causal effect variational auto-encoder (CEVAE)
Q(C|T, Y, X) - > variational posterior
two-branch network to avoid ignoring the T information
Q(Y|T=0, X)
Q(Y|T=1, X)
Inference-based Method
230
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
confounder proxies X
Q(T|X)
Q(Y|T=0, X)
Q(Y|T=1, X)
Q(C|Y, T=0, X)
Q(C|Y, T=0, X)
Cascade inference to allow estimation for test data with missing covariates
Non-treatment branch
Treatment branch
Causal effect variational auto-encoder (CEVAE)
Q(C|T, Y, X) - > variational posterior
Inference-based Method
231
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
General analysis of latent variables in X
Inference-based Method
232
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
General analysis of latent variables in X
Inference-based Method
233
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
reduce bias
General analysis of latent variables in X
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Inference-based Method
234
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
adjustors
reduce bias
reduce variance
General analysis of latent variables in X
(less variability of Y when A is controlled)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
I.V.
Inference-based Method
235
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
reduce bias
reduce variance
increase both bias and variance
General analysis of latent variables in X
(less variability of Y when A is controlled)
(may leave open backdoor path)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Inference-based Method
236
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
reduce bias
reduce variance
increase both bias and variance
General analysis of latent variables in X
(less variability of Y when A is controlled)
(may leave open backdoor path)
(imbalance treatment/non-treatment group)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Inference-based Method
237
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
reduce bias
reduce variance
increase both bias and variance
We should exclude latent IV from the control set
General analysis of latent variables in X
(less variability of Y when A is controlled)
(may leave open backdoor path)
(imbalance treatment/non-treatment group)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Inference-based Method
238
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
I.V.
Treatment effect estimation with disentangled latent factors (TEDVAE)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
Inference-based Method
239
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
I is predictive only for T
A is predictive only for Y
C is predictive for both T and Y
Treatment effect estimation with disentangled latent factors (TEDVAE)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Inference-based Method
240
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
I is predictive only for T
A is predictive only for Y
C is predictive for both T and Y
infer Z = {I, A, C} via CEVAE
max. p(T|C, I), p(Y|C, A)
Treatment effect estimation with disentangled latent factors (TEDVAE)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Inference-based Method
241
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
The constraint encourages the disentanglement of I, C, A
I is predictive only for T
A is predictive only for Y
C is predictive for both T and Y
Treatment effect estimation with disentangled latent factors (TEDVAE)
infer Z = {I, A, C} via CEVAE
max. p(T|C, I), p(Y|C, A)
T
Y
C
X
treatment
outcome
confounders
confounder proxies
I
A
adjustors
I.V.
Outline
242
Background and Causal Inference Basics
Counterfactual Analysis
Latent Confounding Analysis
Latent Mediation Analysis
Challenges & Future Directions
Generalization to Graphs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
243
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
good diet
health situation
244
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
good diet
health situation
245
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
good diet
health situation
246
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
good diet
health situation
247
emotions
Latent Mediation Analysis
(nutrition)
good diet
health situation
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
248
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
249
bad diet
emotions
(nutrition)
good diet
health
nested potential outcome
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
intervention T = 1 on path T -> M -> Y
intervention T = 0 on path T -> Y
Latent Mediation Analysis
250
intervention T = 1 on path T -> M -> Y
intervention T = 0 on path T -> Y
intervention T = 0 on path T -> M -> Y
intervention T = 0 on path T -> Y
bad diet
emotions
(nutrition)
health
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
bad diet
emotions
good diet
health
nested potential outcome
(nutrition)
Latent Mediation Analysis
251
intervention T = 0 on path T -> M -> Y
intervention T = 1 on path T -> Y
intervention T = 0 on path T -> M -> Y
intervention T = 0 on path T -> Y
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
(nutrition)
bad diet
emotions
(nutrition)
health
good diet
emotions
bad diet
health
252
T
M
good diet
health situation
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
253
T
M
stress level
emotions
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
good diet
health situation
254
T
Y
M
social support
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
good diet
health situation
255
T
Y
M
environmental factors
(e.g., pollution)
good diet
health situation
(nutrition)
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
Latent Mediation Analysis
256
(nutrition)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
good diet
health situation
emotions
Latent Mediation Analysis
257
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
258
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
259
control for observed confounders
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
260
Let T=1 affect Y through M
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
261
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Let T=1 affect Y through M
Let T=0 affect Y through M
Latent Mediation Analysis
262
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Let T=0 directly affect Y
Latent Mediation Analysis
263
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Let T=0 directly affect Y
Latent Mediation Analysis
264
Linear structural equation modeling (LSEM)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
265
Linear structural equation modeling (LSEM)
Regress:
(no M)
total effect
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
266
Linear structural equation modeling (LSEM)
Regress:
(no M)
total effect
effect of T on M
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
267
Linear structural equation modeling (LSEM)
Regress:
(no M)
total effect
effect of T on M
direct effect of T on M
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
268
Linear structural equation modeling (LSEM)
Regress:
(no M)
total effect
effect of T on M
direct effect of T on M
effect of M on Y
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
269
Linear structural equation modeling (LSEM)
Mediation Effect:
Regress:
(no M)
effect of T on M
effect of M on Y
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
270
Linear structural equation modeling (LSEM)
Mediation Effect:
Regress:
(no M)
total effect
direct effect of T on M
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
271
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
272
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Latent Mediation Analysis
273
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
274
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Inference-based Method for Latent CMA
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Inference-based Method for Latent CMA
275
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
276
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Inference-based Method for Latent CMA
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
277
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
(law of total expectation)
Inference-based Method for Latent CMA
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
278
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
(law of total expectation)
(remove do-calculus by no confoundedness)
Inference-based Method for Latent CMA
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Inference-based Method for Latent CMA
279
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal mediation analysis variational auto-encoder (CMAVAE)
Inference-based Method for Latent CMA
280
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
cascadedly infer t, M, y, Z to allow missing values for inference
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal mediation analysis variational auto-encoder (CMAVAE)
Inference-based Method for Latent CMA
281
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal mediation analysis variational auto-encoder (CMAVAE)
Inference-based Method for Latent CMA
282
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
two-branch network to avoid forgetting t
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal mediation analysis variational auto-encoder (CMAVAE)
Inference-based Method for Latent CMA
283
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal mediation analysis variational auto-encoder (CMAVAE)
Inference-based Method for Latent CMA
284
[1] Lu Cheng, Ruocheng Guo, and Huan Liu. 2022. Causal Mediation Analysis with Hidden Confounders. WSDM.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Causal mediation analysis variational auto-encoder (CMAVAE)
Latent Mediation Analysis – Part II
285
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
important mediator can also be difficult to measure
Latent Mediation Analysis – Part II
286
Example
emotions
good diet
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
health
important mediator can also be difficult to measure
Latent Mediation Analysis – Part II
287
Example
unknown mediator violates the measurable mediator assumption.
emotions
good diet
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
health
important mediator can also be difficult to measure
Latent Mediation Analysis – Part II
288
Example
unknown mediator violates the measurable mediator assumption.
approximate latent mediator with proxy variables.
emotions
good diet
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
health
important mediator can also be difficult to measure
289
emotions
Example
$money you make
Latent Mediation Analysis – Part II
unknown mediator violates the measurable mediator assumption.
approximate latent mediator with proxy variables.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
good diet
health
important mediator can also be difficult to measure
290
emotions
Example
…
#paper you publish
Latent Mediation Analysis – Part II
$money you make
unknown mediator violates the measurable mediator assumption.
approximate latent mediator with proxy variables.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
good diet
health
important mediator can also be difficult to measure
Inference-based Method for Latent CMA
291
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
Example
…
#paper you publish
$money you make
good diet
health
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
292
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
Example
…
#paper you publish
$money you make
good diet
health
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
293
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
Example
…
#paper you publish
$money you make
good diet
health
latent variables
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
294
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
Example
…
#paper you publish
$money you make
good diet
health
learnable parameters
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
295
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
Example
…
#paper you publish
$money you make
good diet
health
learnable parameters
EM-algorithm!
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
296
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
emotions
…
#paper you publish
$money you make
good diet
health
Example
^
^
^
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
297
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
emotions
…
#paper you publish
$money you make
good diet
health
Example
^
^
^
^
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
298
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
…
#paper you publish
$money you make
good diet
health
Example
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
299
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
…
#paper you publish
$money you make
good diet
health
(substitute with the estimated parameters)
Example
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
300
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
emotions
…
#paper you publish
$money you make
good diet
health
(Monte Carlo estimator)
(substitute with the estimated parameters)
Example
generalized structural equations model (GSEM)
Inference-based Method for Latent CMA
301
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
[2] Sun, Rongqian, Xiaoxiao Zhou, and Xinyuan Song. "Bayesian causal mediation analysis with latent mediators and survival outcome." Structural Equation Modeling: A Multidisciplinary Journal 28.5 (2021): 778-790.
…
…
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
In reality, there may exist multiple mediators either parallel or causally.
Inference-based Method for Latent CMA
302
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
[2] Sun, Rongqian, Xiaoxiao Zhou, and Xinyuan Song. "Bayesian causal mediation analysis with latent mediators and survival outcome." Structural Equation Modeling: A Multidisciplinary Journal 28.5 (2021): 778-790.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
…
…
In reality, there may exist multiple mediators either parallel or causally.
Inference-based Method for Latent CMA
303
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
[2] Sun, Rongqian, Xiaoxiao Zhou, and Xinyuan Song. "Bayesian causal mediation analysis with latent mediators and survival outcome." Structural Equation Modeling: A Multidisciplinary Journal 28.5 (2021): 778-790.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
…
…
In reality, there may exist multiple mediators either parallel or causally.
Inference-based Method for Latent CMA
304
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
[2] Sun, Rongqian, Xiaoxiao Zhou, and Xinyuan Song. "Bayesian causal mediation analysis with latent mediators and survival outcome." Structural Equation Modeling: A Multidisciplinary Journal 28.5 (2021): 778-790.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
…
…
In reality, there may exist multiple mediators either parallel or causally.
Inference-based Method for Latent CMA
305
[1] Albert, Jeffrey M., Cuiyu Geng, and Suchitra Nelson. "Causal mediation analysis with a latent mediator." Biometrical Journal 58.3 (2016): 535-548.
[2] Sun, Rongqian, Xiaoxiao Zhou, and Xinyuan Song. "Bayesian causal mediation analysis with latent mediators and survival outcome." Structural Equation Modeling: A Multidisciplinary Journal 28.5 (2021): 778-790.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
…
…
In reality, there may exist multiple mediators either parallel or causally.
Outline
306
Background and Causal Inference Basics
Counterfactual Analysis
Latent Confounding Analysis
Latent Mediation Analysis
Challenges & Future Directions
Generalization to Graphs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Counterfactual Analysis
307
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
Counterfactual Analysis
308
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
perseverance
your previous education
your mood
Counterfactual Analysis
309
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
perseverance
your previous education
your mood
if you attended the tutorial (T=1) and had high research productivity (Y=1),
Counterfactual Analysis
310
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
perseverance
your previous education
your mood
if you attended the tutorial (T=1) and had high research productivity (Y=1),
what would your research productivity be if you had not attended the tutorial?
Counterfactual Analysis
311
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
perseverance
your previous education
your mood
you might be in a good mood
if you attended the tutorial (T=1) and had high research productivity (Y=1),
what would your research productivity be if you had not attended the tutorial?
Counterfactual Analysis
312
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
perseverance
your previous education
your mood
you might be in a good mood
you might have high perseverance
if you attended the tutorial (T=1) and had high research productivity (Y=1),
what would your research productivity be if you had not attended the tutorial?
Counterfactual Analysis
313
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
tutorial
research productivity
knowledge level
perseverance
your previous education
your mood
you might be in a good mood
you might have high perseverance
Answer: your research productivity may drop, but is still higher than average
if you attended the tutorial (T=1) and had high research productivity (Y=1),
what would your research productivity be if you had not attended the tutorial?
Counterfactual Analysis
314
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
U1
U3
U2
Counterfactual Analysis
315
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
U ~ p(U), C = fc(U2)
T = ft(C, U1)
Y = fy(T, C, U3)
U1
U3
U2
Counterfactual Analysis
316
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
U ~ p(U), C = fc(U2)
T = ft(C, U1)
Y = fy(T, C, U3)
The population distribution of U
U1
U2
U3
Counterfactual Analysis
317
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
(1) reason with and (2) quantify the outcome of an individual had some observed variables X = x been x’
U1
U2
U3
U ~ p(U), C = fc(U2)
T = ft(C, U1)
Y = fy(T, C, U3)
Counterfactual Analysis
318
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
U1
U2
U3
due to abductive reasoning, counterfactual reasoning is a latent variable problem!
(1) reason with and (2) quantify the outcome of an individual had some observed variables X = x been x’
U ~ p(U), C = fc(U2)
T = ft(C, U1)
Y = fy(T, C, U3)
Counterfactual Analysis
319
Exogenous variables
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
intervention
observations
U1
U2
U3
U ~ p(U), C = fc(U2)
T = ft(C, U1)
Y = fy(T, C, U3)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Counterfactual Analysis
320
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
T
Y
C
U1
U2
U3
U ~ p(U), C = fc(U2)
T = ft(C, U1)
Y = fy(T, C, U3)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Counterfactual Analysis
321
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Y
C
U ~ p(U|Z, t), C = fc(U2)
T = t’
Y = fy(T, C, U3)
T
U1
U2
U3
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Counterfactual Analysis
322
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Do(T=t’)
Y
C
U2
U3
U ~ p(U|Z, t), C = fc(U2)
T = t’
Y = fy(T, C, U3)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Counterfactual Analysis
323
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Do(T=t’)
Y
C
U2
U3
U ~ p(U|Z, t), C = fc(U2)
T = t’
Y = fy(T, C, U3)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Counterfactual Analysis
Aim: latent exogenous variables may not be explicitly calculated if counterfactuals of interest do not need to be quantified.
324
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Counterfactual Analysis
325
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Treatment T = S, i.e., sensitive user features such as gender/age/race
machine learning predictor
Aim: latent exogenous variables may not be explicitly calculated if counterfactuals of interest do not need to be quantified.
Counterfactual Analysis
Aim: latent exogenous variables may not be explicitly calculated if counterfactuals of interest do not need to be quantified.
326
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Treatment T = S, i.e., sensitive user features such as gender/age/race
S
Y
C
U1
U3
Y
C
U3
do(S=s’)
U2
U2
s-abducted
s-abducted
factual world
counterfactual world
Counterfactual Analysis
327
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Example: Counterfactual Fairness
Counterfactual Analysis
328
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
only affect S and its causal descendent.
Example: Counterfactual Fairness
Counterfactual Analysis
329
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Quantify counterfactuals by inferring the latent exogenous variables
Counterfactual Analysis
330
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Problem Setting: predict the first-year average grade (FYA)
Counterfactual Analysis
331
race (R) and sex (S)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Problem Setting: predict the first-year average grade (FYA)
Counterfactual Analysis
332
race (R) and sex (S)
knowledge (K)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Problem Setting: predict the first-year average grade (FYA)
Counterfactual Analysis
333
race (R) and sex (S)
knowledge (K)
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Problem Setting: predict the first-year average grade (FYA)
observed covariates
Counterfactual Analysis
334
Fair Add: a fitting model for counterfactual fairness.
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
linear additive deterministic model.
Counterfactual Analysis
335
Fair Add: a fitting model for counterfactual fairness.
Kusner, Matt J., et al. "Counterfactual fairness." Advances in neural information processing systems 30 (2017).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
linear additive deterministic model.
Since the errors are exogenous, the prediction is unbiased
Counterfactual Analysis
336
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
337
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
338
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
dependence test to obtain skeleton
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
339
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
340
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
direction of other edges cannot be determined
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
341
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
342
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
Judge:
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
343
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
It’s a non-descendent of S
Judge:
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
344
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
It’s a non-descendent of S
Judge:
path 1
1
2
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
345
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
It’s a non-descendent of S
Judge:
path 2
1
2
3
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
346
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
It’s a non-descendent of S
Judge:
path 2
directed, not a chord!
1
2
3
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
347
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
Otherwise, we cannot be sure
Judge:
path 1
a chord!
2
1
3
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
348
Zuo, Aoqi, et al. "Counterfactual fairness with partially known causal graph." Advances in Neural Information Processing Systems 35 (2022): 1238-1252.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
determine a partial DAG based on V-structure.
pair-wise examine S and another node, e.g., Z1
Otherwise, we cannot be sure
Judge:
We should only include definite non-descendent of S for prediction
Fair Relax: counterfactual fairness with unknown SCM.
Counterfactual Analysis
349
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Path-specific counterfactuals
Counterfactual Analysis
350
race
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
culture
stereotype
Path-specific counterfactuals
purchases
Counterfactual Analysis
351
race
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
culture
stereotype
Path-specific counterfactuals
bias, should be avoided!
purchases
Counterfactual Analysis
352
race
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
culture
stereotype
Path-specific counterfactuals
bias, should be avoided!
diversity, should be preserved!
purchases
Counterfactual Analysis
353
race
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
factual value
purchases
culture
stereotype
counterfactual value
Path-specific causal effects
Counterfactual Analysis
354
Wu, Yongkai, et al. "Pc-fairness: A unified framework for measuring causality-based fairness." Advances in neural information processing systems 32 (2019)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Path-specific causal effects
culture
stereotype
purchases
factual value
Counterfactual Analysis
355
Wu, Yongkai, et al. "Pc-fairness: A unified framework for measuring causality-based fairness." Advances in neural information processing systems 32 (2019)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
PC-Fairness: bounding path-specific counterfactual bias.
Counterfactual Analysis
356
Wu, Yongkai, et al. "Pc-fairness: A unified framework for measuring causality-based fairness." Advances in neural information processing systems 32 (2019)
consider response functional
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
PC-Fairness: bounding path-specific counterfactual bias.
Counterfactual Analysis
357
Wu, Yongkai, et al. "Pc-fairness: A unified framework for measuring causality-based fairness." Advances in neural information processing systems 32 (2019)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
PC-Fairness: bounding path-specific counterfactual bias.
Counterfactual Analysis
358
Wu, Yongkai, et al. "Pc-fairness: A unified framework for measuring causality-based fairness." Advances in neural information processing systems 32 (2019)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
PC-Fairness: bounding path-specific counterfactual bias.
Outline
359
Background and Causal Inference Basics
Counterfactual Analysis
Challenges & Future Directions
Latent Confounding Analysis
Generalization to Graphs
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Generalization to Graphs
360
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
which have been extensively applied for real-world systems with connected units
Def. Units with interactions
Social Network
Citation Network
Knowledge Graph
For example, we have:
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
360
Generalization to Graphs
361
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Wear Mask
Infection
causes
Answer causal questions when units have interactions
Consider the following example:
Question: how does the usage of face mask influence COVID-19 infection?
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
361
Challenges for Causal Inference on Graph
362
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Wear mask
Infection
Treatment
Outcome
causes
Ordinary graph
Hyperedge
The treatment of a unit may causally affect the outcome of other units
Examples:
On normal graphs
On hyper-graphs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
362
363
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Wear mask
Infection
Treatment
Outcome
causes
Correlation
What if?
Confounder
Treatment
Outcome
Graph connections can also be confounding factor
Examples:
It is hard for traditional methods to control for these confounders
Challenges for Causal Inference on Graph
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
363
364
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Causal Effects on Graphs
Given: observational data {X, A, T, Y}
node features
graph structure
treatments
outcomes
Aim:
Wear mask
Infection
Treatment
Outcome
causes
ITE =
_
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
364
365
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Causal Effects on Graphs
Key idea: Graphs serve as “proxies” for these hidden confounders
E.g., one’s economic status can often be reflected by their social network
Confounder representations
Graph data as confounder proxy
Effective deep representation learning
ITE Estimation based on confounder repr.
Overall Framework
ITE =
_
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
365
366
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Causal Effects on Graphs
- Network Deconfounder
Figure: The workflow of network deconfounder [1]
GCN layers
Wasserstein-1
distance
Loss function
Outcome prediction loss
balancing
regularization
Confounder representation
Representation balancing
Outcome prediction
Guo, Ruocheng, et al. "Learning individual causal effects from networked observational data." WSDM. 2020.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
366
367
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Causal Effects on Graphs
- DNDC for Dynamic Networks
Time
…
Key idea: Capture hidden confounders dynamic graph data
confounder representation learning
History embedding
Graph structure
Graph neural network
Ma, Jing, et al. "Deconfounding with networked observational data in a dynamic environment." ACM WSDM. 2021.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
367
368
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Causal Effects on Graphs
- Instrumental Variable
Key idea: Graph topology may be viewed as an instrumental variable (IV)
treatment
outcome
graph topology
hidden confounders
features
A valid IV must:
Recall that
Ma, Jing, et al. "A Look into Causal Effects under Entangled Treatment in Graphs: Investigating the Impact of Contact on MRSA Infection." KDD. 2023.
Problem setting
Two-stage algorithms can be used to eliminate the confounding bias
In-room contacts
Room sharing network in hospital can serve as an IV
MRSA Infection
cause
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
368
369
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Counterfactual Analysis on Graphs
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
369
370
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Counterfactual Analysis on Graphs
Prediction
Graph
Graph ML model
How to mitigate biases towards underrepresented groups in graph ML?
(e.g., race, gender)
A natural question of fairness – What if ?
Will my application get approved if my gender/race/age had been different?
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
370
371
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Counterfactual Analysis on Graphs
Ma, Jing, et al. Learning Fair Node Representations with Graph Counterfactual Fairness. WSDM, 2022.
Aim: (1) learn counterfactually fair node representations to mitigate bias from sensitive features of each node and their neighbors, (2) maintain good prediction performance
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
371
particularly useful if impossible to directly manipulate the factors of interest
372
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Counterfactual Analysis on Graphs
Work experience: N/A
Work experience: 5 years
Original
Counterfactual
Aim: promotes model explainability by answering the key question:
An example of counterfactual explanation
how can I change to achieve a specific purpose.
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
372
373
Fairness in Graph Machine Learning: Recent Advances and Future Prospectives
Counterfactual Analysis on Graphs
Aim: Generate counterfactual explanations (a graph slightly different from the original input) that lead to a desired output in graph-related predictors
Strategy: VAE with classifier guided generation
[1] Ma, Jing, et al. "CLEAR: Generative Counterfactual Explanations on Graphs." NeurIPS (2022).
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
373
Outline
374
Background and Causal Inference Basics
Counterfactual Analysis
Challenges & Future Directions
Latent Confounding Analysis
Generalization to Graphs
Latent Mediation Analysis
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
Future Directions
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
375
On Theories and Model Design
| | | |
| | | |
| | | |
| | | |
| | | |
Learn causal relationships and representations for high-level causal concepts from data
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
376
On Theories and Model Design
Visual
Language
Spatial
Audio
Temporal
Leverage causal knowledge from multiple modalities
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
377
On Theories and Model Design
Promote human understanding of causal models
Use causal knowledge to improve model explanation
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
378
On Theories and Model Design
Quantify the uncertainty of causal inference and causal ML models
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
379
Large Language Model (LLM)
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
380
Large Language Model (LLM)
LLM
Use the inference capability and the common knowledge embedded in LLMs to identify unknown latent variables
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
381
Large Language Model (LLM)
LLM
Reason with new strategies to circumvent or infer the variables from proxy
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
382
Large Language Model (LLM)
LLM
Design prompts to improve the understanding of causal inference for LLMs
LLMs provide natural language to explain causal inference process
Causal Inference with Latent Variables: Recent Advances and Future Prospectives
383
Q&A
03/08/2021
Thanks for listening!
Causal Inference with Latent Variables: Recent Advances and Future Prospectives