1 of 111

01: Introduction to Human-AI Interaction

Juho Kim & Jean Young Song

Human-AI Interaction KAIST Spring 2021 | human-ai.kixlab.org

2 of 111

Zoom Rules

  • Turn off audio (mute) & turn on video.
  • Find a quiet place (avoid crowded places like a café).
  • Use headphones or earphones.
  • Use the Zoom desktop app.
  • Emergency communication: Zoom chat > Campuswire > email
  • Interaction
    • Use the chat actively! We’re monitoring.
    • Raise your hand > Get addressed > Unmute > speak

​

3 of 111

Zoom Breakout Rooms

  • Used for group discussion and activity.
  • You will be randomly partnered with classmates.
  • Course staff will be lurking in the breakout rooms, so don’t be surprised!
  • Let’s practice.
  • With classmates in your breakout room, spend 3 mins introducing yourselves & talking why you’re in this class.

​

4 of 111

Class Atmosphere

  • Use chat anytime for questions & comments
    • Instructor & TA will address in real-time.
  • Discussion rather than lecture

5 of 111

Class Atmosphere

  • Use chat anytime for questions & comments
    • Instructor & TA will address in real-time.
  • Comment on the slides in real-time (actively please!)
    • We’ll address them after class.
    • Typos, clarification requests, additional references, ideas for better presentation, improvements, etc. are all welcome!

6 of 111

“The two hardest problems in computer science are: (i) people, (ii) convincing computer scientists that the hardest problem in computer science is people.”

- Jeff Bigham (CMU)

7 of 111

“Artificial Intelligence is �inspired by human intelligence, �made powerful by human data, and �ultimately only useful in �how it positively affects the human experience.”

- Jeff Bigham (CMU)

8 of 111

9 of 111

Computer Science is about �making computers that are…

Fast

Secure

Intelligent

Power-efficient

Error-free

Maintainable

Cheap

Small

Reliable

Standard-compliant

Modular

9

10 of 111

Human-Computer Interaction is about making computers that are…

10

useful

usable

11 of 111

HCI accomplishes the goal by designing and building better…

11

interaction

12 of 111

What happens when the computer is AI?

What’s different about it?

accurate�efficient�objective�cheap�doesn’t involve humans�replaces humans�enables new interactions

...

​

12

interaction

13 of 111

13

Ubiquitous Computing, IoT

14 of 111

14

Social Computing, Crowdsourcing

15 of 111

Interaction at Scale

15

16 of 111

Learning Objective

“You’ll master �design principles & technical methods�to design user-centered AI systems�that provide value beyond what �humans or AI can achieve alone.”

16

17 of 111

AI determines �user experience.

18 of 111

AI moderates social interaction.

  • Facebook, Twitter, YouTube, Netflix, Instagram, TikTok, ...

19 of 111

AI manager gives directions to human drivers.

  • Uber, Lyft, Kakao Taxi

​

20 of 111

Humans and AI collaborate and compete.

  • Algorithmic trading, AlphaGo, Chatbot
  • Sometimes humans don’t even realize AI exists in the system.

21 of 111

22 of 111

COMPAS Algorithm: Jailed or allowed out?

https://www.technologyreview.com/s/613508/ai-fairer-than-judge-criminal-risk-assessment-algorithm/

23 of 111

https://twitter.com/alessabocchi/status/1156513770254012416

24 of 111

https://twitter.com/alessabocchi/status/1156513770254012416

25 of 111

https://twitter.com/alessabocchi/status/1156513770254012416

26 of 111

https://twitter.com/FastCompany/status/1176634667660955648

27 of 111

Three troubling trends about how AI is portrayed to the public

28 of 111

Trend 1: Humans are hidden behind AI.

29 of 111

  • Bedrock of many modern AI systems
  • Publicly available image dataset
  • 14M images and 22K visual categories

​

https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/

ImageNet

30 of 111

https://www.oreilly.com/library/view/deep-learning-for/9781788295628/3052765c-f7cf-450a-a6bd-281750e27e8f.xhtml

Error rate

ImageNet Visual Recognition Challenge

31 of 111

Driving Force for Deep Learning Revolution

32 of 111

https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/

ImageNet Visual Recognition Challenge

33 of 111

  • Collect, Clean, process, label, verify…

https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/

How to create such a dataset? Crowdsourcing to the rescue

34 of 111

http://www.merriam-webster.com/dictionary/crowdsourcing

outcome

task

scale

undefined crowd

open call

35 of 111

Why use crowdsourcing?

  • Cost
  • Limited availability of experts
  • The crowd might actually outperform experts
  • Open call: diversity, error-tolerant
  • Parallel work possible: higher efficiency

​

36 of 111

Crowdsourcing Marketplace

37 of 111

Artificial Intelligence

Copper engraving from the book: Karl Gottlieb von Windisch, Briefe über den Schachspieler �des Hrn. von Kempelen, nebst drei Kupferstichen die diese berühmte Maschine vorstellen. 1783.

38 of 111

“Artificial” Artificial Intelligence

Copper engraving from the book: Freiherr Joseph Friedrich zu Racknitz, �Ueber den Schachspieler des Herrn von Kempelen, Leipzig und Dresden 1789.

39 of 111

40 of 111

41 of 111

“These people doing �‘ghost work’ make

the internet seem smart.”

​

  • Generating training data
  • Flagging bad content
  • Manually fixing AI errors

​

~8% of Americans have contributed to �“ghost economy”.

​

https://ghostwork.info/ghost-work/

42 of 111

What about AI hiding behing humans?

​

43 of 111

Who wins in the era of AI?

  • Platforms and marketplaces serve as power plants and mines.

44 of 111

Who made money during the Gold Rush?

  • Miners?

​

  • “Recent scholarship �confirms that merchants �made far more money than �miners during the Gold Rush.”�[https://en.wikipedia.org/wiki/California_Gold_Rush]

​

  • Supply stores, lodging, transportation

Image by G.F. Nesbitt & Co., printer

45 of 111

Who wins in the era of AI?

  • Model builders?

​

  • Dataset providers
  • GPU manufacturers
  • Cloud computing�service providers

http://cs231n.stanford.edu/slides/winter1516_lecture1.pdf

46 of 111

Trend 2: Hidden �social cost of AI

47 of 111

Intelligent Image Search

  • Google Photos: Automatic object recognition and tagging

48 of 111

Intelligent Image Search: �What could possibly go wrong?

http://mashable.com/2015/07/01/google-photos-black-people-gorillas/#v3QwN6bx1uqX

49 of 111

Recommender Systems

50 of 111

Filter Bubble: Seeing only what I’d like to see.

https://www.ted.com/talks/eli_pariser_beware_online_filter_bubbles

51 of 111

Do I access the same truth as you?

https://www.trendhunter.com/trends/blue-feed-red-feed

52 of 111

Emily/Brendan vs Lakisha/Jamal?

  • If Emily and Brendan needed to send out 10 resumes on average to get one response, how many resumes did Lakisha and Jamal need to send?
  • Emily and Brendan got 30% more callbacks when they sent out resumes listing high qualifications compared to when they sent out resumes with low qualifications. How much did Lakisha and Jamal benefit from getting higher qualifications?

Bertrand, Marianne, and Sendhil Mullainathan. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. No. w9873. National Bureau of Economic Research, 2003.

15

9%

53 of 111

Emily/Brendan vs Lakisha/Jamal?

  • If Emily and Brendan needed to send out 10 resumes on average to get one response, how many resumes did Lakisha and Jamal need to send?

Bertrand, Marianne, and Sendhil Mullainathan. "Are Emily and Greg more employable than Lakisha and Jamal?

A field experiment on labor market discrimination." American economic review 94.4 (2004): 991-1013.

15

54 of 111

Learning from Biased Data in �Word Embedding

55 of 111

Learning from Biased Data in �Word Embedding

Bolukbasi, Tolga, et al. "Man is to computer programmer as woman is to homemaker? debiasing word embeddings."

Advances in Neural Information Processing Systems. 2016.

56 of 111

Why is this problematic?

  • It’s not simply inheriting human biases; it can amplify really easily at scale.
  • People often believe AI is objective and fair.
  • What if such embedding was used to make hiring decisions?

57 of 111

Garbage in, garbage out

  • Technical pipelines involving data and AI are not neutral.
  • Various sources of bias and political & value-sensitive decisions

​

  • ImageNet
    • Where do categories come from?
    • Bias toward something visual
    • Insufficient representation across dimensions

Image from https://www.semantics3.com/blog/thoughts-on-the-gigo-principle-in-machine-learning-4fbd3af43dc4/

58 of 111

“programmer”

Image from http://image-net.org/update-sep-17-2019.php

59 of 111

“programmer”

Image from http://image-net.org/update-sep-17-2019.php

60 of 111

https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world/

“Science progresses through trial and error, through understanding the limitations and flaws of past results,”

“We believe that ImageNet, as an influential research dataset, deserves to be critically examined, in order for the research community to design better collection methods and build better datasets.”

Announced to scrub more than half of the 1.2 million pictures in the dataset’s “people” category.

“There is no easy technical ‘fix’ by shifting demographics, deleting offensive terms, or seeking equal representation by skin tone,”

“The whole endeavor of collecting images, categorizing them, and labeling them is itself a form of politics, filled with questions about who gets to decide what images mean and what kinds of social and political work those representations perform.”

61 of 111

Toward more sound AI

  • Fairness
    • Hiring algorithm without racial, gender, regional bias
  • Accountability
    • Self-driving car killing pedastrians
    • Facebook’s “X years ago today” reminding users of sad incidents
  • Transparency
    • Why does AI recommend jailing this person?

​

62 of 111

Algorithmic auditing & AI-free products?

https://twitter.com/andresmh/status/994983030530883585/photo/1

63 of 111

Trend 3: Humans vs AI

64 of 111

65 of 111

Ever advancing AI, where do they take us?

STAN HONDA/AFP/Getty Images

66 of 111

https://www.ted.com/talks/shyam_sankar_the_rise_of_human_computer_cooperation

67 of 111

Advanced chess

https://en.chessbase.com/post/first-female-advanced-che-match-drawn-050413

68 of 111

AI: Imitating humans

photo credit: KAIST Hubo Lab

69 of 111

Human-AI collaboration

Image courtesy of Christine Daniloff at MIT

70 of 111

GauGAN: Mixed-Initiative Drawing

71 of 111

GauGAN: Mixed-Initiative Drawing

72 of 111

Technology Forces Us To �Do Things We’re Bad At? (Don Norman)

  • 90% of car accidents are caused by human mistakes

​

  • “The real cause of most accidents is an engineering mentality that favors automating whatever we can and leaving people to fill in the gaps. This forces people to behave according to the machine's needs and on its terms: Things people are bad at. And when people are asked to do things they are bad at, they do them badly, which leads to accidents.”

https://www.fastcodesign.com/3067411/technology-forces-us-to-do-things-were-bad-at-time-to-change-how-design-is-done

73 of 111

Lessons from working on human-AI interaction

  • Just having technology is not enough.
  • Full automation is often not the answer.
    • Full automation - Human-in-the-loop - Machine-in-the-loop - Fully manual
  • UX of AI should be considered.
    • Motivation, Difficulty, Risk, Trust
    • Feedback, Transparency, Fairness, Accountability
  • Adapt AI to people and teams, not the other way around.

Lubars, Brian, and Chenhao Tan. "Ask Not What AI Can Do, But What AI Should Do: �Towards a Framework of Task Delegability." arXiv preprint arXiv:1902.03245 (2019).

74 of 111

http://gimgane.co.kr/img/main/ver2/mv01_img01.png

75 of 111

Are you sold this is an important and exciting topic?

76 of 111

Now to class logistics

77 of 111

Your instructors

Prof. Jean Young Song

Prof. Juho Kim

77

78 of 111

Your TA

Hyungyu Shin

  • 3nd year Ph.D. student at KIXLAB
  • Interested in HAI design
  • hyungyu.com

​

  • M.S., KIXLAB@KAIST
  • B.S., POSTECH

78

79 of 111

Who are we? Professor Jean Young Song

  • 2nd year at KAIST, Research Assistant Professor
  • Research Interests: HCI, Crowdsourcing, Human-AI Interaction, Computer Vision
  • kixlab.org
  • jyskwon.github.io
  • Ph.D., University of Michigan, Ann Arbor
  • M.S., Yonsei University
  • B.S., Yonsei University

79

80 of 111

Who are we? Professor Juho Kim

  • 5th year at KAIST, Associate Professor
  • Research Interests: HCI, social computing, interaction at scale
  • KAIST’s Grand Prize for Creative Teaching (2020), Excellent in Teaching Award (2019)
  • kixlab.org
  • juhokim.com
  • Ph.D., MIT
  • M.S., Stanford University
  • B.S., Seoul National University

​

80

81 of 111

Why you should listen to us?

  • We collectively have many years of experience
    • designing interactive sytems with AI.
    • designing crowdsourcing techniques & workflows to power AI.
    • consulting & collaborating with companies, startups, and government that want to built AI products.
      • Hint: They often fail in the beginning when only thinking about using AI. User-centered HAI is key to success.

81

82 of 111

82

83 of 111

Interactive systems powered by large-scale data from users

83

ToolScape

LectureScape

Crowdy

BudgetMap

Confer

Factful

84 of 111

Courseware

  • Course website
    • human-ai.kixlab.org
    • All course updates & assignments will be posted here.
  • KLMS: assignment submission & grading
  • Campuswire: Q&A, announcements, discussion
    • Will mostly replace email.

84

85 of 111

No exams

No lectures

86 of 111

86

Deslauriers, Louis, Ellen Schelew, and Carl Wieman.

"Improved learning in a large-enrollment physics class." science 332.6031 (2011): 862-864.

Experienced, highly-rated instructor with lectures

Trained but inexperienced instructor with active learning

87 of 111

Active Learning

  • Improves student attention, engagement, & learning
  • Clicker responses
  • Small group tasks
  • Student discussions
  • Factual knowledge transfer outside of class
  • Frequent feedback

87

Deslauriers, Louis, Ellen Schelew, and Carl Wieman.

"Improved learning in a large-enrollment physics class." science 332.6031 (2011): 862-864.

88 of 111

In this course, you will

  • READ, CRITIQUE

​

  • ANALYZE, REFLECT

​

  • DESIGN, BUILD, TEST

​

  • DISCUSS, SHARE

​

​

88

89 of 111

In this course, you will

  • READ, CRITIQUE
    • Reading Response (each class)

​

  • ANALYZE, REFLECT
    • Assignments (two times a semester)

​

  • DESIGN, BUILD, TEST
    • Design Project (2nd half of the semester)

​

  • DISCUSS, SHARE
    • In-class, asynchronous discussion (anytime)

​

​

89

90 of 111

In this course, you will

  • READ, CRITIQUE
    • Reading Response (each class)

​

  • ANALYZE, REFLECT
    • Assignments (two times a semester)

​

  • DESIGN, BUILD, TEST
    • Design Project (2nd half of the semester)

​

  • DISCUSS, SHARE
    • In-class, asynchronous discussion (anytime)

​

​

90

30%

30%

30%

10%

91 of 111

Reading Response

  • Each class has pre-class material for you to read or watch.
    • Research paper, news article, video clip, etc.
    • You’ll alternate roles as a writer and a reader: writer writes a response, reader reads and comments.
  • Critique
    • Summarize main ideas and discuss why they matter.
    • What have you learned? What did you like about the paper?
    • Methodological / logical / technical concerns? How would you improve it?

91

92 of 111

Assignments

  • Hands-on design & implementation activities
    • Improve fairness in the model and data for content moderation in an online community.
    • Improve explainability & design user-centered explanations for an image classification model.
    • Visualize AI output to convey a clear message.

92

93 of 111

Final Project

  • Design, build, and test your own human-AI interaction system.
    • A full platform is discouraged (e.g., login, followers, …)
    • Focus on the novel, core human-AI interaction
  • Team of 3: HCI + AI teams are strongly encouraged.
  • Connecting to your own research is encouraged.

93

94 of 111

Participation

  • In-class
    • Please speak!
    • Contribute your own (incomplete, half-baked) perspective.
    • Don’t worry about English.
  • At home
    • Share cool examples, ask and answer questions
    • Discussion forum

94

95 of 111

Prerequisites and skills you’ll need

  • Background in HCI or AI (at least intro-level coursework)
  • Programming skills: no need to have them all
    • HTML/CSS/JavaScript
      • Basic UI implementation, basic visualization
    • Python
      • Handling data I/O, analyzing data, running (pre-trained) models

95

96 of 111

Changes from 2020

  • Added tutorials for both HCI & AI sides.
    • For prepping you to do more hands-on implementation in both.
  • Revised assignments

​

96

97 of 111

Takeaways from Today

  • This course is about human-AI interaction, and principles, techniques, & methods for realizing it.

​

  • We want you to succeed and learn.
    • It’s not really about evaluating where you are �at the end of the course.
    • But you have to do your part: active learning.
    • You have to speak up, otherwise you won’t learn.

​

97

98 of 111

TODO items for you

  • Contact course staff if you couldn’t access the “Student Instructions” document.

​

  • Complete the course sign-up form NOW
    • You’re not officially registered unless you fill this out. Due 3/9 (Tue).
    • bit.ly/cs492e-2021-signup

​

  • Visit the course website
    • human-ai.kixlab.org
    • Course updates and materials

​

  • Sign up for Campuswire
    • All announcements, Q&A, & discussions

​

99 of 111

Upcoming

  • 3/4: Tour of Human-AI Interaction

​

99

100 of 111

101 of 111

102 of 111

Why I chose this area of science

Please explain why you chose to investigate this particular aspect of science, computing, or engineering.

103 of 111

The problem or challenge

Please explain the question or problem that you investigated

104 of 111

The hypothesis (or prediction)

What do you think will happen?

105 of 111

Research

Explain all of the research you’ve done about this issue/challenge.

What was the goal of your research? Be sure to explain how you found it and anyone who might have helped you!

106 of 111

My testing method

Each scientist uses different methods of experimentation

What methods did you use in your experiment?

  • Lorem ipsum dolor sit amet, consectetur adipiscing elit
  • Incididunt ut labore et dolore
  • Consectetur adipiscing elit, sed do eiusmod tempor incididunt

107 of 111

Experiment

data

Record the information you get from your experiment

10min

20

5

15

20min

29

4

25

30min

39

4

35

40min

27

5

22

Item 1

Item 2

Include a table or graph to display what you see

108 of 111

Aha! �My discoveries

What did you learn after testing?

  1. Lorem ipsum dolor sit amet, consectetur adipiscing elit
  2. Incididunt ut labore et dolore
  3. Consectetur adipiscing elit, sed do eiusmod tempor incididunt

109 of 111

This is the most important takeaway that everyone has to remember.

110 of 111

Conclusion

What is the conclusion of your experiment? Did the results support your hypothesis or predicted outcome? How will your findings help the area of science you’ve researched?

111 of 111

What will I do next?

What will you do with your findings next? How will you further your research/findings?