Interpretable Machine Learning

Correlation vs. Causation: Does X influence Y?

X

Y

X

Y

vs.

X

Y

X

Y

vs.

vs.

vs.

Pr(Y=1|X)

Pr_X=1(Y=1|X)

Measures Correlation

Measures Causation

Interpretable Correlation: Can humans interpret how X correlates with Y?

Positive Linear Correlation: Pr(Y=1|X) = link_function(<A, X> + b)

Another Positive Linear Correlation: Pr(Y=1|X) = link_function(<A, higher_order(X)> + c)

X

=

[x₁, x₂, ………., x_n]

1. Do Decision Trees Model Linear or Non Linear Correlations?
2. Are Linear Correlation Models Interpretable?
3. Are Neural Networks Linear Correlation Models?

Interpretability - Necessary Condition: Tractable Linear Models - Enough to track a small set of linear mappings derived from interactions among the [x₁, x₂, ………., x_n] )

Interpretable Correlation: Can humans interpret how X correlates with Y?

X

=

[x₁, x₂]

x₁

x₂

z

x₁

x₂

z₁

z₂

z

𝛔

𝛔

Y

x₁

x₂

x₂

Y

Y

Y

Y

𝛔

Y

Y

=

x₁x₂ + x₁x₂ + x₁x₂ + x₁x₂

Interpretable Correlation: Can humans interpret how X correlates with Y?

Image Source For Tree��Image Source For Neural Network

Interpretability vs. Approximation Power Trade-off

Interpretable Machine Learning

Universally Approximate Machine Learning

Image Source

Interpretable Correlation: Can humans interpret how X correlates with Y?

Positive Linear Correlation: Pr(Y=1|X) = link_function(<A, X> + b)

Another Positive Linear Correlation: Pr(Y=1|X) = link_function(<A, higher_order(X)> + c)

X

=

[x₁, x₂, ………., x_n]

• Do Decision Trees Model Linear or Non Linear Correlations?
• Are Linear Correlation Models Interpretable?
• Are Neural Networks Linear Correlation Models?

Interpretability - Necessary and Sufficient Condition: Human interpretable and Tractable Linear Models - Enough to track linear relationships among a small set of human interpretable mappings derived from interactions among the [x₁, x₂, ………., x_n])

Interpretable Correlation: Can humans interpret how X correlates with Y?

LIME Image Source

Universally Approximate Machine Learning

Interpretable Machine Learning

Interpretable Correlation: Can humans interpret how X correlates with Y?

Locally a Linear Model

​

Still, what if dimensionality of

​

is too high?

​

i.e., not a tractable linear model

X

=

[x₁, x₂, ………., x_n]

X

NOT THE SAME AS LOCALLY INTERPRETABLE!

Interpretable Correlation: Can humans interpret how X correlates with Y?

Open Research - Mappings: Constructing the human interpretable mappings derived from interactions among the [x₁, x₂, ………., x_n])

​

There can be many mappings that achieve the same accuracy (all high performance)

Universally Approximate Machine Learning

Interpretable Machine Learning

Mapping Guided by Knowledge?

LIME: Mapping Guided by Features + First-Order Geometry

​

LIME Image Source

Universally Approximate Machine Learning

Interpretable Machine Learning

SHAP: Mapping Guided by Features + Higher-Order Geometry

​

Image Source

Revisiting: Mapping Guided by Knowledge

Relying on Geometry Requires Large Amounts of Data

Open Research - Mappings: Constructing the human interpretable mappings derived from interactions among the [x₁, x₂, ………., x_n])

​

There can be many mappings that achieve the same accuracy (all high performance)

Universally Approximate Machine Learning

Interpretable Machine Learning

Mapping Guided by Knowledge?

Revisiting: Mapping Guided by Knowledge

Knowledge Encodes Human Interpretable Features along with Human Interpretable Geometry

Open Research - Mappings: Constructing the human interpretable mappings derived from interactions among the [x₁, x₂, ………., x_n])

​

There can be many mappings that achieve the same accuracy (all high performance)

Universally Approximate Machine Learning

Interpretable Machine Learning

Mapping Guided by Knowledge?