1 of 41

Visualization for human-ai interaction

1

See quiz at humanaiclass.org

Human - AI

Interaction

Human - AI

Interaction

Human - AI

Interaction

​

​

Chinmay Kulkarni and Mary Beth Kery

Fall 2019, Human-Computer Interaction Institute, Carnegie Mellon University

Human-AI Interaction Fall 19 .

2 of 41

Assignment #1 review: whoa!

  • from sklearn.neural_network import MLPClassifier ???
  • from sklearn.ensemble import RandomForestClassifier
  • from sklearn import tree

clf = tree.DecisionTreeClassifier()

  • income_loan_ratio = df_new['loan_amount_000s'] / df_new['applicant_income_000s']

​

2

Human-AI Interaction Fall 19 .

3 of 41

Assignment #1 review: common confusions

You can google terms

  • Training set, validation set, test set difference
  • What is one-hot encoding doing?
  • What does it mean to replace nan values?
  • Why did Logistic Regression perform the same as Dummy Classifier?

3

Human-AI Interaction Fall 19 .

4 of 41

Why do we care about Visualization in HAII?

  • Good visualization (maybe) eliminates need for AI
    • augmented sensemaking�
  • AI helps create better visualizations
    • with lots of complex data, difficult to do manually�
  • Visualizing Models
    • “Understanding” models for developers
    • “Understanding” models for users
      • Understanding is too vague. Think about maybe describing, predicting outcomes, etc.

4

Human-AI Interaction Fall 19 .

5 of 41

5

Human-AI Interaction Fall 19 .

6 of 41

6

Human-AI Interaction Fall 19 .

7 of 41

7

Human-AI Interaction Fall 19 .

8 of 41

Hairballs

8

Human-AI Interaction Fall 19 .

9 of 41

Today’s Agenda

  • Visualization of information:
    • Can be used to manipulate; enlighten; explore
    • Visualization to communicate versus hide uncertainty: why hide uncertainty? Why show it?
    • When to visualize, when not to visualize?
  • For whom? User-facing visualizations in AI/ML Systems
    • Tasks to support [quantified self/charts, qualifying a result: medical decisions, is a graphical thing the best form?]
    • Different forms of showing data or results to users
  • For whom? Visualization for development of AI/ML Systems
    • Tasks to support
    • Understanding what’s in your data: Fairvis, What-if tool

9

Human-AI Interaction Fall 19 .

10 of 41

The process of visualization

Choose Data -> Transform -> map to visual attributes

E.g.

Student grades -> Group grades (+/-10 points) -> map each group to a bar, and number of students in group to height of bar chart

10

Human-AI Interaction Fall 19 .

11 of 41

Choosing the right data

11

Electoral college map

Popular vote map

Human-AI Interaction Fall 19 .

12 of 41

Choosing the right transformation

12

Human-AI Interaction Fall 19 .

13 of 41

Perils of transforming data poorly

Scales are so easy to abuse

13

Human-AI Interaction Fall 19 .

14 of 41

Perils of transforming data poorly

Scales are so easy to abuse

14

Human-AI Interaction Fall 19 .

15 of 41

Perils of transforming data poorly

Percentages make sense for proportions only

​

​

​

PS: Did Internet traffic from browsers grow faster 2000-2005 than 2005-2010?

15

Human-AI Interaction Fall 19 .

16 of 41

Perils of transforming data poorly

Percentages make sense for proportions only

A linear scale for exponentially distributed quantities makes it hard to compare rates

16

Human-AI Interaction Fall 19 .

17 of 41

Perils of encoding data poorly

What fraction of men can read the pie chart?

How much larger is “Don’t have it” to “have it”?

Is ALDE > (EPP + Non-inscrits + EUL-NGL)?

​

17

Human-AI Interaction Fall 19 .

18 of 41

Perils of encoding data poorly

What fraction of men can read the pie chart?

  • Reading angles is hard
  • Reading angles in perspective is harder

How much larger is “Don’t have it” to “have it”?

  • Comparing angles is harder

Is ALDE > (EPP + Non-inscrits + EUL-NGL)?

  • Angle math is really hard!

​

18

Human-AI Interaction Fall 19 .

19 of 41

Encoding: Which is the bigger square?

19

Human-AI Interaction Fall 19 .

20 of 41

Encoding: Which is the bigger square?

20

Human-AI Interaction Fall 19 .

21 of 41

Encoding: How much bigger is circle on right?

1.5x?

2x?

3x?

21

Human-AI Interaction Fall 19 .

22 of 41

From Jeff Heer: https://courses.cs.washington.edu/courses/cse442/17au/lectures/CSE442-VisualEncoding.pdf

22

Human-AI Interaction Fall 19 .

23 of 41

23

Human-AI Interaction Fall 19 .

24 of 41

24

Human-AI Interaction Fall 19 .

25 of 41

Why is visualization different for AI-infused products?

  • Data is often output of algorithms -- so it is uncertain
  • Inputs are also sometimes uncertain

​

25

Human-AI Interaction Fall 19 .

26 of 41

Visualizing for the user of an AI/ML system

26

Human-AI Interaction Fall 19 .

27 of 41

Visualizing uncertainty for users

27

Human-AI Interaction Fall 19 .

28 of 41

What does visualization give users? Is it worth it?

28

Human-AI Interaction Fall 19 .

29 of 41

29

Human-AI Interaction Fall 19 .

30 of 41

Visualizing similar objects AI/ML

Where is the visualization in this app?

What does visualization give users here?

30

Human-AI Interaction Fall 19 .

31 of 41

Visualization is good: Quantified self

Shown: Apple health visualization

What do you think of these plots? What is unusual about them?

31

Human-AI Interaction Fall 19 .

32 of 41

What does visualization give users? Is it worth it?

32

https://washuvis.github.io/admissions/process-book/

Human-AI Interaction Fall 19 .

33 of 41

Visualization for developing AI/ML systems

33

Human-AI Interaction Fall 19 .

34 of 41

Tasks for visualization in AI/ML Development

Understanding data that goes into models

  1. Understanding what a dataset contains
  2. Visualize for data cleaning
  3. Visualize for feature creation

Monitoring models

  • Visualize performance on different data groups
  • Understand model output
  • Visually search for possible negative bias or discrimination

34

Human-AI Interaction Fall 19 .

35 of 41

What is in this dataset (at scale)?

What do you think of this hairball?

35

Human-AI Interaction Fall 19 .

36 of 41

Exploring desirable outcomes of a model

What do you think of these visualizations?

Where you able to gain insights from them, why or why not?

36

Human-AI Interaction Fall 19 .

37 of 41

Reading comments

37

Human-AI Interaction Fall 19 .

38 of 41

“However, after reading this article I almost think usability has to be higher for applications that have real life impacts, like in the medical field, because there is very little room for error”

38

Human-AI Interaction Fall 19 .

39 of 41

​

​

Beyond this, there's still the issues of what the sliders will be among the thousands of features encoded by the model.

39

Human-AI Interaction Fall 19 .

40 of 41

Did I overthink the problem or did the algorithm went wrong? In my own mental model, I always think towards the way that let the algorithm build better models and produce better performance. But in the case of medical decisions, it is certainly not the appropriate way.

40

Human-AI Interaction Fall 19 .

41 of 41

“I thought the discussion at the end about the precision vs. glanceability tradeoff was really similar to how the class was divided on Tuesday about which type of user interface they would want to see”

41

Human-AI Interaction Fall 19 .