Introduction to Interpretable Machine Learning (IML) and Explainable AI (XAI) with SHAP
1
Machine Learning’s Black Box Problem
2
Machine Learning’s Black Box Problem
THIS IS MERELY ANECTODAL AND ISN’T SCIENTIFICALLY VERIFIED
3
Machine Learning’s Black Box Problem
4
As scientists, we are interested in explaining reality, rather than simply predicting it.
How do we then use machine learning to understand the “why” of the phenomena that we study, rather than just the what?
IML and XAI: why is it important?
5
Interpretable Machine Learning
Any model whose behavior a human operator can understand by (1) understanding its methods and (2) viewing its trained parameters.
Explainable Artificial Intelligence
Any method that allows a human operator to understand why a trained model made certain predictions, both in specific instances and in general.
IML and XAI: what are the definitions?
… there actually isn’t consensus on that question, yet…
It’s a spectrum, without a clear dividing line…
6
If the coalition wins, how should we allocate the winnings?
XAI: how can we do it?
7
Game theory:
Shapley Values for cooperative games
8
Game theory:
Shapley Values for cooperative games
9
Game theory:
Shapley Values for cooperative games
10
Game theory:
Shapley Values for cooperative games
11
In the Glove Game, we have a team N of three players, named 1, 2, and 3. Each gets a right-handed glove or a left-handed glove, but no player gets more than one glove.
The goal of the game is to create as many complete pairs of gloves as possible. For each pair of gloves, the team is awarded one point.
In our game, players 1 and 2 each get a right-handed glove. Player 3 gets a left-handed glove.
Given that, logically, the team can only form one complete pair, what are the respective contributions of each player to the team?
Intro to Shapley Values: the Glove Game
Right-handed glove
Right-handed glove
Left-handed glove
12
The Glove Game: value function
Right-handed glove
Right-handed glove
Left-handed glove
13
The Glove Game: marginal contributions
14
The Glove Game: player values
Player 2’s value
Player 1’s value
Player 3’s value
15
Shapley Additive Explanations XAI toolkit
In machine learning, the features are the players from Shapley’s game theory framework!
16