1 of 22

Mountaineer: Topology-Driven Visual Analytics for Comparing Local Explanations

Parikshit Solunke

Committee:

Claudio Silva (advisor)

Juliana Freire

Luis Gustavo Nonato

2 of 22

Agenda

  1. Motivation

​

  • Related Work

​

  • Background: Topological Data Analysis�
  • Requirements

​

  • Mountaineer�
  • Case Studies�
  • Expert Interviews�
  • Conclusion

3 of 22

Why do we need Explanations for Black-box Machine Learning Models?

4 of 22

Why xAI?

  • Interpretability and Transparency�
  • Bias Detection�
  • Legal and Ethical Compliance�
  • Model Debugging

​

5 of 22

Local Explanation Methods for Black-box ML Models

6 of 22

Classifier Task: Binary - “Does the person with the given features earn more than 100K ?

�Explainer Task: “What features contributed to the classifier’s prediction and how much significant was the contribution?”

Local Explanation Methods - Attributions

7 of 22

Local Explanation Methods - Problems

Explanation Results are difficult to compare and evaluate!�

  • Variety of methods and techniques�
  • Attributions are on different scales and are multi-dimensional�
  • Some methods have their own hyperparameters and baselines

​

​

​

IntGrad

SHAP

LIME

DeepLIFT

Anchors

8 of 22

Local Explanation Methods - Disagreement

  • As a result of the variety of approaches and the randomness associated with some explanation techniques, they often disagree.����������

�Problem: Given multiple explanation results:�� 1) How do you compare them - locally AND globally?

2) How do you decide which ones to trust?

9 of 22

Proposal: Leverage Topological Data Analysis (TDA) to help understand and compare explanation results on a structural level.

10 of 22

Geometrically different but topologically equivalent

Background: Topology

​

​

  • Branch of Mathematics that deals with shapes of objects

​

  • Unconcerned with geometrical attributes�
  • Concerned only with connectivity details within different parts of objects

11 of 22

Background - TDA

�

  • TDA leverages topology to study the shape, structure, and topological features of data, even when the data is high-dimensional and noisy.

�

  • Core idea: convert point clouds => graph that summarizes the shape of the data.

​

12 of 22

Mapper Algorithm and Bottleneck Distance

​

  • The Mapper Algorithm uses algebraic topology to construct a network like structure from the data points, summarizing the overall topological “shape” of the data�
  • Mapper algorithm can be thought of as a form of informed overlapping clustering’.�
  • Bottleneck distance is a metric used �to measure the similarity of two�Mapper graphs, with a lower �distance indicating similar topological�structure.

​

13 of 22

Creating topological representations of Explanations

Explanation Output

14 of 22

Creating topological representations of Explanations

Predicted Probabilities

15 of 22

Creating topological representations of Explanations

Overlapping Clustering

16 of 22

Creating topological representations of Explanations

Topological Graph

17 of 22

Why TDA helps?

​

  • Extracts data shapes (i.e. patterns)�
  • Graph like skeleton fits well in a visual analytics framework�
  • Works across datasets with differing scales

18 of 22

Workflow

​

19 of 22

Workflow

20 of 22

Workflow

21 of 22

Workflow

22 of 22

Workflow

Interactions and Linked Views