1 of 28

Attacks and Threats

in Crowdsourcing

Lecturer: Kenneth Huang

txh710@psu.edu

​

Mar 31, 2020

2 of 28

Move the Final Project Presentation

From Live to Video Recording

  • Checkpoint Presentation (5 minutes, submit an URL)
    • Present the progress of your project
    • Get familiar with the recording software (https://screencast-o-matic.com/)
  • Final Presentation (10 minutes, submit an URL)�
  • Final Project Office Hour (Live)
    • 4/7 (Tue), 10:30am to 11:50am
    • 4/9 (Thu), 10:30am to 11:50am

​

​

​

2

3 of 28

3

4 of 28

Attacks and Threats

in Crowdsourcing

Lecturer: Kenneth Huang

txh710@psu.edu

​

Mar 31, 2020

5 of 28

We’re Going to Talk About…

  • Challenges
    • Inappropriate Workers
    • Inappropriate Users
    • Identifying the End of a Conversation
    • When Consensus Is Not Enough
  • Threats
    • Threat types
    • How to defend?

​

5

6 of 28

6

7 of 28

7

8 of 28

8

9 of 28

Spammer Workers

  • Known to exist in literature�
  • 3 Main Actions
    • Message
      • “how are you”, “yeah”, “yes (or no)”, “Sure you can”, or “It suits you best.”
    • Note
      • “user is dumb”
      • “like all the answers.”
    • Vote
      • Upvote on almost all messages

​

9

10 of 28

We’re Going to Talk About…

  • Challenges
    • Inappropriate Workers
    • Inappropriate Users
    • Identifying the End of a Conversation
    • When Consensus Is Not Enough
  • Threats
    • Threat types
    • How to defend?

​

10

11 of 28

Malicious Users

  • Abusive Languages
    • Sexual content
    • Profanity
    • Hate speech
    • Threats of criminal acts�
  • Solutions
    • Word detection

​

11

12 of 28

We’re Going to Talk About…

  • Challenges
    • Inappropriate Workers
    • Inappropriate Users
    • Identifying the End of a Conversation
    • When Consensus Is Not Enough
  • Threats
    • Threat types
    • How to defend?

​

12

13 of 28

13

14 of 28

14

15 of 28

15

16 of 28

16

17 of 28

We’re Going to Talk About…

  • Challenges
    • Inappropriate Workers
    • Inappropriate Users
    • Identifying the End of a Conversation
    • When Consensus Is Not Enough
  • Threats
    • Threat types
    • How to defend?

​

17

18 of 28

18

19 of 28

19

20 of 28

20

21 of 28

We’re Going to Talk About…

  • Challenges
    • Inappropriate Workers
    • Inappropriate Users
    • Identifying the End of a Conversation
    • When Consensus Is Not Enough
  • Threats
    • Threat types
    • How to defend?

​

21

22 of 28

Two Types of Attack

22

Lasecki, W. S., Teevan, J., & Kamar, E. (2014, February). Information extraction and manipulation threats in crowd-powered systems. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (pp. 248-256). ACM.

23 of 28

Information Extraction Attack

23

24 of 28

Information Manipulation Attack

24

25 of 28

Conditions

25

  • Baseline
  • “Innocent”
  • “Malicious”

26 of 28

Results

26

27 of 28

27

28 of 28

Attacks and Threats

in Crowdsourcing

Lecturer: Kenneth Huang

txh710@psu.edu

​

Mar 31, 2020