This page lists tools that we’ve found helpful for “programming the crowd,” and specifically those that we will use for Crowd Programming.
There are a bunch of SDKs available for developing with Mechanical Turk. They are generally all better than the commandline interface provided by Amazon, although often not a lot better. Jeff still often uses the commandline interface.
One popular SDK for accessing the Amazon Web Services API, which includes Amazon Mechanical Turk, is the Boto SDK for Python.
Our wrapper for Boto targets at what you need for using Amazon Mechanical Turk in the context of this course, and provides a couple of handy Python scripts for creating/deleating HITs.
You might also find our mturk.js useful. It prepares web pages that you’ve written for use within Amazon Mechanical Turk.
Finally, Turkit can be a useful tool for working with Mechanical Turk, especially on iterative tasks.
Weka isn’t the most powerful machine learning toolkit out there, but it’s pretty good and easy to use. We’ll use Weka to help make sense of the data that we get back from the crowd.
Etherpad is a simple, collaborative, real-time text editor. It’s surprisingly effective for prototyping experiments for the crowd. To try something out, you can just type in instructions for the crowd workers who come to your etherpad, and they can work right in the etherpad to get them done.