Human-Computer Interaction

saadh.info/hci

​

Week 13 (Thursday): Designing Human-AI Interaction

1

Attendance and Agenda

1. Design for Human-AI Interaction

2

Announcements

• Assignment 3 overdue. 7 missing…
• Continue working on milestone 2! Due December 6
• Assignment 4 (due December 2) is conducting heuristic evaluation of one screen flow
• Discuss dividing among team members
• Test 2 on Nov 21
• Same format as test 1
• Read chapters 4-7 of Human-Computer Interaction: An Empirical Research Perspective by MacKenzie, I. Scott, if you can

​

​

3

Test 2 Next Thursday

4

Cognitive Walkthroughs

• Walkthroughs are methods where an expert (that would be you, the designer) defines tasks. Afterward, rather than testing those tasks with real people, you walk through each step of the task and verify that a user would:
• know to do the step,
• know how to do the step,
• would successfully do the step, and
• would understand the feedback the design provided.
• If you go through every step and check these four things, you’ll find most problems with a design.

Polson, P. G., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive walkthroughs: a method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies.

5

Performing Cognitive Walkthrough

• Select a task to evaluate (probably a frequently performed important task that is central to the user interface’s value). Identify every individual action a user must perform to accomplish the task with the interface.
• Obtain a prototype of all of the states necessary to perform the task, showing each change. This could be anything from a low-fidelity paper prototype showing each change along a series of actions, or it might be a fully-functioning implementation.
• Develop or obtain persona of representative users of the system. You’ll use these to help speculate about user knowledge and behavior.

​

6

Identifying Design Flaws using Walkthrough

• Will the user try to achieve the right effect?
• Would the user even know that this is the goal they should have?
• Will the user notice that the correct action is available?
• If they wouldn’t notice, you have a design flaw.
• Will the user associate the correct action with the effect that the user is trying to achieve?
• Even if they notice that the action is available, they may not know it has the effect they want.
• If the correct action is performed, will the user see that progress is being made toward the solution of the task?
• Is there feedback that confirms the desired effect has occurred?

7

GenderMag Walkthrough

• Four customizable personas to cover:
• A user’s motivations for using the software.
• A user’s information processing style (top-down, which is more comprehensive before acting, and bottom-up, which is more selective.)
• A user’s computer self-efficacy (their belief that they can succeed at computer tasks).
• A user’s stance toward risk-taking in software use.
• A user’s strategy for learning new technology

8

Usability Evaluation Steps

1. Plan and prepare
2. Conduct the test
3. Collect data
4. Analyze data
5. Draw conclusions
6. Document results
7. Repeat step

9

10

1. Perception and Cognition

11

2. User Research Methods & Qualitative Analysis

12

Interviews

Contextual Inquiry

Think-out Aloud

3. Experimental Research in HCI

Error bars show

±1 standard deviation

4. Analytical Evaluations

Next week!

Last Week

5. Modeling Interactions

Units: bits

RT = a + b log₂(n + 1)

Fitts’ Law

Hick-Hyman’ Law

6. Designing for Human-AI Interaction

What we know about design so far

17

Gestalt Principles

Visual Design

Norman’s Design Principles

Nielsen's Heuristics

Can we apply these to Human-AI Interactions?

• AI-infused systems can violate established usability guidelines of traditional user interface design
• Inherently inconsistent due to poorly understood underlying probabilistic systems and blackbox implementations
• Change over time, e.g., learn more, learn false information
• React differently in different conditions
• Behave differently from one user to next (e.g., browsers due to personalization)

​

18

Shneiderman-Maes Debate

19

Ben Shneiderman and Pattie Maes. 1997. Direct manipulation vs. interface agents. interactions 4, 6 (Nov./Dec. 1997), 42–61. https://doi.org/10.1145/267505.267514 (569 Citations)

This week: Meredith Morris & Michael Bernstein Vs. Andrés Monroy-Hernández & Jeff Bigham

20

Guidelines for Human-AI Interaction

21

Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 3, 1–13. https://doi.org/10.1145/3290605.3300233 (1604 Citations)

Guidelines for Human-AI Interaction

22

Guidelines for Human-AI Interaction

23

Guidelines for Human-AI Interaction

24

Implications throughout the design cycle…

25

Google People+AI Research Guidelines

26

https://pair.withgoogle.com/guidebook/

Human-AI Interaction

• AI is better at some things than others. Make sure that it’s the right technology for the user problem you’re solving.
• Three Four fundamental questions
• How do I get started?
• How do I onboard new users?
• How to explain AI performance?
• How do I help build and calibrate trust in my product?

​

27

How to get started?

28

Determine if AI adds value

• When AI is probably better
• Recommending different content to different users, such as movie suggestions
• Predicting future events, such as weather events or flight price changes
• Natural language understanding
• Image recognition

​

​

• A heuristic-based solution is better
• Maintaining predictability is important, e.g., scheduling
• Users, customers or developers need complete transparency, e.g., credit card scoring?
• People don’t want a task automated

​

29

Automation vs. Augmentation

30

Augment: When a machine, software, or function extends a person’s abilities or potential while maintaining their agency.

​

Automate: When a machine, software, or function performs a task without user involvement.

Determine if AI adds value

31

Don’t use AI just because you can. Heuristics or manual control can often create better experiences. Here, using music preferences to suggest workouts will likely lead to a worse experience than letting people manually choose workouts.

Setting the Right Expectations

32

Avoid suggesting that the technology works perfectly in high-stakes situations if the tech isn’t yet reliable.

Be accountable for errors

33

Providing access to a person can be one way to make sure users’ concerns and problems are directly addressed. Sometimes the user’s error can’t be directly remedied but actions can be taken to make sure other users don’t encounter the same problem.

Invest early in good data practices

34

• The better your data planning and collection processes, the higher quality your end output.
• Collect data in batches.
• Embrace “noisy” data
• Plan for data maintenance
• Partner with domain experts

​

How to best do data collection?

35

Embrace Noisy Data

Design for data labelers (supervised learning)

Learn from disagreements

Make Precision and Recall Trade-offs Carefully

36

Precision

No false positives are classified, but some true positives are missed.

​

Recall

All true positives are classified, but some false positives are captured.

​

What if tool

37

https://pair-code.github.io/what-if-tool

Make Precision and Recall Trade-offs Carefully

38

Enable users to include results (true positives) that may have been excluded.

Enable users to exclude results (false positives) that may have been included.

How do I onboard new users?

39

Explain the benefit, not the technology

40

Emphasize how the app will benefit users. Avoid emphasizing the underlying technology.

Anchor on familiarity

41

Use familiar concepts from your product’s domain to help users set expectations and feel comfortable with the material. Avoid using clever and novel solutions just for the sake of it when a familiar solution will be more effective.

​

​

​

​

​

Automate in phases

42

As you design your product, think critically about the balance of automation and control that you need to offer your users for them to use your product successfully

How to explain AI performance?

43

Determine how to show model confidence, if at all

44

Show confidence in a way that is easier to interpret and understand when making a decision. Provide recourse for when the system is less than fully confident. Don’t user numeric numbers

Explain for understanding, not completeness

45

Don't try to explain the entire system, especially when the rationale is complex or unknown.

Go beyond in-the-moment explanations

46

Help users better understand your product with deeper explanations outside immediate product flows.

How do I help users build and calibrate trust in my product?

47

Setting the Right Expectations

48

Avoid suggesting that the tech works perfectly in high-stakes situations if the tech isn’t yet reliable.

Be transparent about privacy and data settings

49

Communicate what data is being collected and shared, and give users the ability to control their preferences.

Add context from human sources

50

Third Party Experts

Social Proofs

Let users give feedback

51

Don’t just thank users—reveal how feedback will benefit them. They’ll be more likely to give feedback again. Let users know what adjustments would happen.

Let users supervise automation

52

Avoid automating without giving users a way to undo, or allow users to make a choice in the first place.

Let users supervise automation

53

Be more proactive with automation when failure tolerance is higher.

Avoid automating without user control in high-stakes situations.

Give control back to users when automation fails

54

Help users to take over when automation fails.

Read more

55

Attendance & Next Time

• Presentations for milestone 1 review
• Test 2 Review

56