1 of 37

05: Primer on HCI (part 1)

Juho Kim & Jean Young Song

Human-AI Interaction KAIST Fall 2020 | kixlab.org/courses/human-ai

2 of 37

Administrative Notes

  • Assignment #1 will be announced on Thu.
    • “Detecting & Mitigating a bias in abusive language detection”
    • You’ll:
      • explore the given datasets.
      • write code for detecting & mitigating bias.
      • discuss implications for human-AI interaction.
    • Detailed description in class on Thu.
    • There will be office hours.
    • Due: 11:59pm Sep 29 (Tue)

3 of 37

Previously on CS492F...

4 of 37

Today’s Learning Objectives

After today’s class, you should be able to...

  • Understand the basics of HCI concepts and heuristic evaluation.
  • Apply HAI guidelines to a real-life example.
  • Suggest improvements to HAI guidelines for a given product.

5 of 37

6 of 37

Reflection on last week’s in-class activity

7 of 37

Activity Instruction Review

For the following three scenarios, please discuss the role division for AI and human.

S1. Bank deposits: Customer takes an image of their check with their phone, and uploads it to the bank. Bank deposits money.

S2. Concierge at expensive resort: Guests text in what they are looking for and the concierge recommends activities to do (ideally, activity is fun, and makes resort money)

S3. Admissions committee: who should be let into the masters program?

​

8 of 37

What roles AI should/should not take?

Q1. What human role can be useful for each scenario?

Q2. What machine role can be useful for each scenario?

Q3-1. What roles should AI not take if possible?

Q3-2. What role should human not take if possible?

9 of 37

Activity Summary: human’s role

  • You preferred human to feed data, backup machines, or manage the overall task (including triagers, decision makers, and judge).

10 of 37

Activity Summary: machine’s role

  • No one wanted AI to manage humans. You preferred machines to do things that human instruct (worker), to suggest actions (advisor, recommender), or to ask questions to re-think some aspects of the task.

11 of 37

Activity Summary: what AI shouldn’t do

  • appeals judge
  • final decision maker
  • manager
  • orchestrator

​

Quotes from you:

  • Any task that involves liability, in case of selection bias / misinterpretation of the data.
  • Tasks that interactions among human are highly valued.
  • Tasks with high accountability, important decisions which cannot be undone
  • Tasks involving social interactions and understanding
  • Ill-defined tasks (where there’s no definite answers)
  • And many more

12 of 37

Activity Summary: what human shouldn’t do

  • lifeguard
  • worker
  • calculation
  • data entry
  • routine tasks

​

Quotes from you:

  • Time consuming tasks which involves a lot of data processing (prone to make mistakes)
  • Well-defined tasks requiring mechanical actions mainly
  • If a lot of data is necessary for high accuracy
  • If the task needs to be done extremely quickly
  • And many more

​

13 of 37

HCI 101 in 10 mins

14 of 37

Human-Computer Interaction is about making computers that are…

14

useful

usable

15 of 37

HCI accomplishes the goal by designing and building better…

15

interaction

16 of 37

What happens when the computer is AI?

What’s different about it?

probabilistic & statistical�hard to predict�autonomous�adaptable

accurate, cheap, efficient�objective & biased

human-like�doesn’t involve humans�replaces or scares humans�...

​

16

interaction

17 of 37

18 of 37

Example from https://medium.com/@vinoth3.141/mental-model-ux-64e7a9d2a03f

19 of 37

https://www.youtube.com/watch?v=rxdyp_dyYZU

​

20 of 37

Whose fault is it?

“Stupid users keep making mistakes �when using this simple feature.”

“I built this really cool thing. �How come nobody uses it?”

Human Error?

No, it’s BAD DESIGN.

​

21 of 37

You’re not the user.

UI is about communicating with users.

Users are NOT LIKE YOU.

​

The user is ALWAYS RIGHT.

Usability problems are the designer’s fault.

​

​

22 of 37

Usability

how well users can �use the system’s functionality

Learnability: is it easy to learn

Efficiency: once learned, is it fast to use?

Safety: are errors few and recoverable?

​

​

23 of 37

24 of 37

User-Centered Design Process

http://www.spring2innovation.com/2019/04/design-thinking-vs-user-centred-design/

25 of 37

Low ⇒ High Fidelity Prototyping

26 of 37

Usability Testing Methods in HCI

  • Formative Evaluation
  • Field Study
  • Controlled Experiment

​

  • Heuristic Evaluation

Testing with users

Testing with experts

27 of 37

Heuristic Evaluation

  • small (3-5) group of UI evaluators (normally experts)
  • apply usability principles �(heuristics that best capture context)
  • identify usability issues w/ ref to heuristics�

​

28 of 37

Nielsen’s Heuristics

​

​

  • Match the real world
  • Consistency & standards
  • Help & documentation
  • User control & freedom
  • Visibility of system status
  • Flexibility & efficiency
  • Error prevention
  • Recognition, not recall
  • Error reporting, diagnosis, and recovery
  • Aesthetic & minimalist design

​

29 of 37

How to do Heuristic Evaluation

  • Justify every problem with a heuristic.
    • Avoid emotional reactions: “I don’t like it.”
  • List every problem.
  • Go through the interface at least twice.
    • Once to get the feel of the system
    • Again to focus on particular interface elements
  • Don’t have to limit to the 10 Nielsen heuristics.

​

30 of 37

Heuristic Evaluation vs Usability Testing

  • Evaluator is not the user either
    • Maybe closer to being a typical user than you are, though
  • Analogy: code inspection vs. testing
  • HE finds problems that UT often misses
  • But UT is the gold standard for usability.

​

​

31 of 37

“Guidelines for �Human-AI Interaction”

​

32 of 37

Main Strengths from RR

  • Practical for both designers and developers
  • Comprehensive
  • Concrete examples
  • Thorough methodology to design the guidelines

​

33 of 37

Main Criticisms from RR

  • Weak ethics considerations ⇒ Upcoming classes
  • Limited implications for the end-to-end design/development cycle ⇒ Upcoming classes
  • Centered around GUIs ⇒ Today’s activity
  • Some abstract guidelines esp. around social

34 of 37

ACTIVITY Part 1: Heuristic Evaluation

  • Now that you’re a Human-AI Interaction expert, let’s class-source heuristic evaluation of two AI-infused products: mobile voice assistant & FB News Feed
  • Use the 18 guidelines from the paper to perform heuristic evaluation.
  • Submit at least two usability issues for each product (2x2) using the form below. (Just reopen the link each time to add a new issue.) [10 mins]

yellkey.com/inside

35 of 37

ACTIVITY Part 2: Heuristic Evaluation

  • Let’s browse through all the issues that are collected.
  • What interesting patterns do you notice?
  • Any notable differences between the two products?
  • What could be said about the usability of each product?
  • [10 mins]

​

​

yellkey.com/best

36 of 37

ACTIVITY Part 3: Improving the Guidelines

  • Let’s think about ways to design more specific Human-AI Interaction guidelines for voice assistants.
  • [15 mins] in groups of 3-4

​

​

yellkey.com/increase

37 of 37