1 of 40

XDesign: Integrating Interface Design into Explainable AI Education

Hyungyu Shin

Nabila Sindi

Yoonjoo Lee

Jaeryoung Ka

Juho Kim

Jean Y. Song

2 of 40

Explainable AI (XAI)

2

The value of “Consolidated risk markers” is 65. It needs to be around 72 for the application to be approved.

Loan application outcome

(AI Explainability 360)

Image classification

(Ribeiro et al., 2016)

[1] Ribeiro et al., "Why should i trust you?" Explaining the predictions of any classifier, KDD 2016

Predicted label: Labrador

3 of 40

Explanations need to be usable �by effectively addressing user needs

3

“fur coat”

“Airliner”

[1] Ribeiro et al., "Why should i trust you?" Explaining the predictions of any classifier, KDD 2016

4 of 40

Explanations need to be usable �by effectively addressing user needs

4

“fur coat”

“Airliner”

Can you show me more superpixels?

[1] Ribeiro et al., "Why should i trust you?" Explaining the predictions of any classifier, KDD 2016

5 of 40

Explanations need to be usable �by effectively addressing user needs

5

“fur coat”

“Airliner”

Can you show me more superpixels?

What if I remove the background?

[1] Ribeiro et al., "Why should i trust you?" Explaining the predictions of any classifier, KDD 2016

6 of 40

Research argues the importance of taking �user-centered design process for creating explanations

6

Step 1

Step 2

Mapping questions to modeling solutions

Iterative design and evaluation

Question analysis

Question elicitation

Step 3

Step 4

Question-driven design process (Liao et al., 2021)

[2] Liao et al., Question-driven design process for explainable AI user experiences, arXiv 2021

7 of 40

Research argues the importance of taking �user-centered design process for creating explanations

7

Step 1

Step 2

Mapping questions to modeling solutions

Iterative design and evaluation

Question analysis

Question elicitation

Step 3

Step 4

Question-driven design process (Liao et al., 2021)

[2] Liao et al., Question-driven design process for explainable AI user experiences, arXiv 2021

Existing learning materials on explainable AI mostly focus on how to generate explanations and its ethical discussion

8 of 40

Can we guide learners to follow a user-centered �design process for creating usable explanations?

8

9 of 40

XDesign: an interactive platform that guides learners �to create an interactive UI prototype as usable explanations

9

As a solution, we built XDesign, an interactive platform that guides learners to create an interactive UI prototype as usable explanations by allowing learners to follow a user-centered design process. XDesign provides a step-by-step guidance on how to design usable explanation. In summary, / it first ask learners to use an explainable AI technique. What I mean by use is basically to generate explanations for many data points. Then the system asks learners to / analyze results, and make observations on notable limitations of the model. From that limitation, the system ask learners to / list up representative questions that need to be answered from an UI prototype, and ask learners to / actually create such interactive prototype. Through this process, learners are able to create an interactive prototype as a more usable explanation that can answer a set of representative questions that arises while using an explanation. Let’s go to more detailed walkthrough.

10 of 40

XDesign: an interactive platform that guides learners �to create an interactive UI prototype as usable explanations

10

As a solution, we built XDesign, an interactive platform that guides learners to create an interactive UI prototype as usable explanations by allowing learners to follow a user-centered design process. XDesign provides a step-by-step guidance on how to design usable explanation. In summary, / it first ask learners to use an explainable AI technique. What I mean by use is basically to generate explanations for many data points. Then the system asks learners to / analyze results, and make observations on notable limitations of the model. From that limitation, the system ask learners to / list up representative questions that need to be answered from an UI prototype, and ask learners to / actually create such interactive prototype. Through this process, learners are able to create an interactive prototype as a more usable explanation that can answer a set of representative questions that arises while using an explanation. Let’s go to more detailed walkthrough.

11 of 40

XDesign: an interactive platform that guides learners �to create an interactive UI prototype as usable explanations

11

As a solution, we built XDesign, an interactive platform that guides learners to create an interactive UI prototype as usable explanations by allowing learners to follow a user-centered design process. XDesign provides a step-by-step guidance on how to design usable explanation. In summary, / it first ask learners to use an explainable AI technique. What I mean by use is basically to generate explanations for many data points. Then the system asks learners to / analyze results, and make observations on notable limitations of the model. From that limitation, the system ask learners to / list up representative questions that need to be answered from an UI prototype, and ask learners to / actually create such interactive prototype. Through this process, learners are able to create an interactive prototype as a more usable explanation that can answer a set of representative questions that arises while using an explanation. Let’s go to more detailed walkthrough.

12 of 40

XDesign: an interactive platform that guides learners �to create an interactive UI prototype as usable explanations

12

As a solution, we built XDesign, an interactive platform that guides learners to create an interactive UI prototype as usable explanations by allowing learners to follow a user-centered design process. XDesign provides a step-by-step guidance on how to design usable explanation. In summary, / it first ask learners to use an explainable AI technique. What I mean by use is basically to generate explanations for many data points. Then the system asks learners to / analyze results, and make observations on notable limitations of the model. From that limitation, the system ask learners to / list up representative questions that need to be answered from an UI prototype, and ask learners to / actually create such interactive prototype. Through this process, learners are able to create an interactive prototype as a more usable explanation that can answer a set of representative questions that arises while using an explanation. Let’s go to more detailed walkthrough.

13 of 40

XDesign: an interactive platform that guides learners �to create an interactive UI prototype as usable explanations

13

As a solution, we built XDesign, an interactive platform that guides learners to create an interactive UI prototype as usable explanations by allowing learners to follow a user-centered design process. XDesign provides a step-by-step guidance on how to design usable explanation. In summary, / it first ask learners to use an explainable AI technique. What I mean by use is basically to generate explanations for many data points. Then the system asks learners to / analyze results, and make observations on notable limitations of the model. From that limitation, the system ask learners to / list up representative questions that need to be answered from an UI prototype, and ask learners to / actually create such interactive prototype. Through this process, learners are able to create an interactive prototype as a more usable explanation that can answer a set of representative questions that arises while using an explanation. Let’s go to more detailed walkthrough.

14 of 40

14

15 of 40

15

16 of 40

16

17 of 40

17

18 of 40

18

19 of 40

19

20 of 40

20

21 of 40

21

22 of 40

22

23 of 40

23

24 of 40

24

25 of 40

25

26 of 40

26

Our poster doesn’t include evaluation results in detail, but we actually deployed our system to a Human-AI interaction course offered at KAIST where two of our authors, Jean and Juho, were instructors, and I was TA. We issued an assignment, which asked students to use XDesign to design usable explanation. And here we get some interesting explanations.

This example shows not just parts of images but what the parts actually are, such as tongue and nose. It also presents similar data points in the training data so that users can get some insights on why such prediction results came out.

Next example shows a heatmap that visualizes the importance of each superpixels.

This example actually supports some image manipulation features that allows users to test some hypotheses on explanation generation algorithms.

The last example shows a conversational agent where users can ask for more informations. Of course, these were all prototype, so actual technical feasibility wasn’t considered.

27 of 40

27

Our poster doesn’t include evaluation results in detail, but we actually deployed our system to a Human-AI interaction course offered at KAIST where two of our authors, Prof. Jean Song and Prof. Juho Kim, were instructors, and I was TA. We issued an assignment, which asked students to use XDesign to design usable explanation. And here we get some interesting explanations.

This example shows not just parts of images but what the parts actually are, such as tongue and nose. It also presents similar data points in the training data so that users can get some insights on why such prediction results came out.

Next example shows a heatmap that visualizes the importance of each superpixels.

This example actually supports some image manipulation features that allows users to test some hypotheses on explanation generation algorithms.

The last example shows a conversational agent where users can ask for more informations. Of course, these were all prototype, so actual technical feasibility wasn’t considered.

28 of 40

28

Our poster doesn’t include evaluation results in detail, but we actually deployed our system to a Human-AI interaction course offered at KAIST where two of our authors, Prof. Jean Song and Prof. Juho Kim, were instructors, and I was TA. We issued an assignment, which asked students to use XDesign to design usable explanation. And here we get some interesting explanations.

This example shows not just parts of images but what the parts actually are, such as tongue and nose. It also presents similar data points in the training data so that users can get some insights on why such prediction results came out.

Next example shows a heatmap that visualizes the importance of each superpixels.

This example actually supports some image manipulation features that allows users to test some hypotheses on explanation generation algorithms.

The last example shows a conversational agent where users can ask for more informations. Of course, these were all prototype, so actual technical feasibility wasn’t considered.

29 of 40

29

Our poster doesn’t include evaluation results in detail, but we actually deployed our system to a Human-AI interaction course offered at KAIST where two of our authors, Prof. Jean Song and Prof. Juho Kim, were instructors, and I was TA. We issued an assignment, which asked students to use XDesign to design usable explanation. And here we get some interesting explanations.

This example shows not just parts of images but what the parts actually are, such as tongue and nose. It also presents similar data points in the training data so that users can get some insights on why such prediction results came out.

Next example shows a heatmap that visualizes the importance of each superpixels.

This example actually supports some image manipulation features that allows users to test some hypotheses on explanation generation algorithms.

The last example shows a conversational agent where users can ask for more informations. Of course, these were all prototype, so actual technical feasibility wasn’t considered.

30 of 40

30

Our poster doesn’t include evaluation results in detail, but we actually deployed our system to a Human-AI interaction course offered at KAIST where two of our authors, Prof. Jean Song and Prof. Juho Kim, were instructors, and I was TA. We issued an assignment, which asked students to use XDesign to design usable explanation. And here we get some interesting explanations.

This example shows not just parts of images but what the parts actually are, such as tongue and nose. It also presents similar data points in the training data so that users can get some insights on why such prediction results came out.

Next example shows a heatmap that visualizes the importance of each superpixels.

This example actually supports some image manipulation features that allows users to test some hypotheses on explanation generation algorithms.

The last example shows a conversational agent where users can ask for more informations. Of course, these were all prototype, so actual technical feasibility wasn’t considered.

31 of 40

Discussion

31

How to help students explore more diverse kinds of images?�
Can we generalize this approach to other types of ML tasks and XAI techniques?�
Would there be some UI design patterns for creating explanations?

From our experience, we have the following discussion points. First, how to help students explore more diverse kinds of images? / We found that students actually try a narrowed space of images. Like, students who’re trying dog images / just keep trying some similar dog images only. Even within the dogs, there can be diverse kinds of images such as a dog standing on a chair, a dog running, or multiple dogs. Since having diverse observations is important when identifying user needs, how can we systematically encourage students explore diverse kinds of images?

The second point is about generalizability. Can we generalize this approach to other types of ML tasks and XAI techniques? We only focused on image classification task, but we can think of applying the same design process for other dimensions like fairness. There could be some specific needs when an explanation is saying this model is biased for asian. There can be some follow-up questions from users, which can be ingredients to design more usable explanations.

The last question is about UI design; would there be some UI design patterns for creating explanations? From students’ UI design, we observed some common UI design patterns such as presenting heatmap or presenting similar data points in the training data. It means, it might be possible to easily design interactive explanations if we can support such popular UI design patterns for explanations.

32 of 40

How to help students explore more diverse kinds of images?

32

From our experience, we have the following discussion points. First, how to help students explore more diverse kinds of images? / We found that students actually try a narrowed space of images. Like, students who’re trying dog images / just keep trying some similar dog images only. Even within the dogs, there can be diverse kinds of images such as a dog standing on a chair, a dog running, or multiple dogs. Since having diverse observations is important when identifying user needs, how can we systematically encourage students explore diverse kinds of images?

The second point is about generalizability. Can we generalize this approach to other types of ML tasks and XAI techniques? We only focused on image classification task, but we can think of applying the same design process for other dimensions like fairness. There could be some specific needs when an explanation is saying this model is biased for asian. There can be some follow-up questions from users, which can be ingredients to design more usable explanations.

The last question is about UI design; would there be some UI design patterns for creating explanations? From students’ UI design, we observed some common UI design patterns such as presenting heatmap or presenting similar data points in the training data. It means, it might be possible to easily design interactive explanations if we can support such popular UI design patterns for explanations.

33 of 40

How to help students explore more diverse kinds of images?

33

34 of 40

Discussion

34

How to help students explore more diverse kinds of images?�
Can we generalize this approach to other types of ML tasks and XAI techniques?�
Would there be some UI design patterns of usable explanations?

35 of 40

Discussion

35

How to help students explore more diverse kinds of images?�
Can we generalize this approach to other types of ML tasks and XAI techniques?�
Would there be some UI design patterns of usable explanations?

Fairness

36 of 40

Discussion

36

How to help students explore more diverse kinds of images?�
Can we generalize this approach to other types of ML tasks and XAI techniques?�
Would there be some UI design patterns of usable explanations?

37 of 40

Discussion

37

How to help students explore more diverse kinds of images?�
Can we generalize this approach to other types of ML tasks and XAI techniques?�
Would there be some UI design patterns of usable explanations?

38 of 40

Discussion

38

How to help students explore more diverse kinds of images?�
Can we generalize this approach to other types of ML tasks and XAI techniques?�
Would there be some UI design patterns of usable explanations?

39 of 40

XDesign: Integrating Interface Design into Explainable AI Education

Hyungyu Shin

Nabila Sindi

Yoonjoo Lee

Jaeryoung Ka

Juho Kim

Jean Y. Song

https://xdesign.kixlab.org

40 of 40

XDesign: Integrating Interface Design into Explainable AI Education

Hyungyu Shin

Nabila Sindi

Yoonjoo Lee

Jaeryoung Ka

Juho Kim

Jean Y. Song

https://xdesign.kixlab.org