Course Description:

Crowdsourcing and human computation are useful in a number of real-world applications. Crowds generate large data sets useful for natural language process and computer vision; they work together to formulate intelligent responses far beyond what we can automate; and they power intelligent interactive systems currently impossible with automated approaches alone. In this course, students will learn to program the crowd. They will write programs that work with existing sources of crowds (Amazon Mechanical Turk, oDesk, Facebook …), apply usability principles for designing crowd tasks that elicit high-quality responses, use statistical methods to improve the quality of the work received, build systems that interface with crowd labor in real time, and conduct experiments to improve understanding of the differences between different sources of crowd work. Course work will consist of projects in each of these areas, and a final team project decided on by the students in each team.

Class time will be a mix of short lectures and interactive components

-- please bring your laptop!

Specific topics addressed in the course will include:

History of Crowdsourcing

Connections to Social Computing, Collective Intelligence, and Management Science

Usability Engineering for Crowdsourcing

Input-Agreement Mechanisms

Output-Agreement Mechanisms

Gold Standards

Consensus Algorithms

Incentive Mechanisms

Gamification

Real-Time Crowdsourcing

Queueing Theory for Predicting Worker Availability

Ensemble Learning

Crowd Agents

Crowdsourcing Subjective Tasks

Social Q&A

Expertise in Crowdsourcing

Local Crowdsourcing / Communitysourcing

CrowdFunding

Citizen Science

Crowds: Mechanical Turk

Crowds: oDesk

Crowds: Mobile Works

Crowds: Social Media

Applications: Computer Vision

Applications: NLP

Applications: Speech to Text

No specific prerequisites are required, but the course will including programming and basic statistics are assumed. Ask the instructor if you have questions about your preparation.