Consent & Pre-Study Survey for Field Deployment Study of the "Chorus" System
This field deployment study of the "Chorus" system is part of a research study conducted by Prof. Jeffrey P. Bigham at Carnegie Mellon University and is funded by Yahoo!.

The purpose of this study is to understand how people use crowd-powered conversational agents. Natural language dialog is an important and intuitive way for people to access information and services. However, current dialog systems are limited in scope, brittle to the richness of natural language, and expensive to produce. In response to this situation, we explore to use crowdsourcing to provide instant feedback or supports in an automated dialog system to make it better.

A crowd-powered dialog system contains active components that are run by a group of crowd workers. For instance, in Chorus, a group of crowd workers propose and select (by voting) the best response to answer users question in minutes; in Guardian, crowd workers extract information from a running dialog to assist web API perform a query; in InstructableCrowd, the crowd workers create executable rules based on users verbal instruction.

Humans will always be slower than machines; however, humans are more robust. Today's dialog systems often waste user’s time with errors or when a question is out of scope. We will evaluate our system by using traditional evaluation metrics in the field of dialog systems, e.g., task completion rate, task completion time, number of conversational turn, etc.

We are interested in how people use crowd-powered conversational agents to solve their own everyday-life tasks. The procedure for you, the participant, is as follows.

For the field deployment study, we will deploy our crowd-powered conversational agent as a web-based application that can be accessed via users' desktops, laptops, and mobile phones. Users will be asked to 1) install the agent on their own devices, 2) bring the agent around with them, and 3) use the agent to solve the real-world problems they encounter (e.g., find restaurants, check weathers, etc.) whenever they think the agent can help.

At the beginning of the study, we will first explain the purposes, requirements, and constraints of the study to you. A tutorial document will be provided to teach you how to use our system via your own devices. If you agreed to participate in this study, we will have you sign the on-line consent form. (You don’t need to physically come to our lab to sign the consent form.)

During the course of the study, one or more experimenter(s) will be available, in person or remotely, to answer your questions and collect data.

A pre-test and post-test survey might be performed to collect your opinions.

[Location] You can use the agent anywhere you wish. You don’t need to physically come to our lab to sign the consent form.

[Time] The recruitment along with survey and consent should take up to 30 minutes. You can use your own devices and interact with the agent as long or as short as you wish. This study will take up to one year. If you wish to remain in the study, they must not uninstall the agent for the first two weeks. The participants who uninstall the agent within the first two weeks will be excluded in our study.

Participant Requirements
We will randomly choose the participants to participate in the tasks, with no regard to sex, gender, race, ethnicity or sexual orientation. We will ask participants during the consent process to indicate that they are 18+.
The risks associated with participation in this study are breach of confidentiality. In our system, the crowd workers are not able to identify the true identities (including the Google Hangouts’ conversation IDs, user IDs, or user names) of any participants from the worker interface. However, our system can not prevent users from spontaneously describing their own personal information (such as names, address, emails, etc.) via conversation to the crowd.
There may be no personal benefit from your participation in the study but the knowledge received may be of value to humanity.
Compensation & Costs
You will not be compensated in the field deployment study.
There will be no cost to you if you participate in this study.
By participating in the study, you understand and agree that Carnegie Mellon may be required to disclose your consent form, data and other personally identifiable information as required by law, regulation, subpoena or court order. Otherwise, your confidentiality will be maintained in the following manner:

Your data and consent form will be kept separate. Your research data will be stored in a secure location on Carnegie Mellon property. Sharing of data with other researchers will only be done in such a manner that you will not be identified. By participating, you understand and agree that the data and information gathered during this study may be used by Carnegie Mellon and published and/or disclosed by Carnegie Mellon to others outside of Carnegie Mellon. However, your name, address, contact information and other direct personal identifiers will not be mentioned in any such publication or dissemination of the research data and/or results by Carnegie Mellon. Note that per regulation all research data must be kept for a minimum of 3 years.

The researchers will take the following steps to protect participants’ identities during this study: (1) Each participant will be assigned a number; (2) The researchers will record any data collected during the study by number, not by name; (3) Any original recordings or data files will be stored in a secured location accessed only by authorized researchers.

Right to Ask Questions & Contact Information
If you have any questions about this study, you should feel free to ask them now. If you have questions later, desire additional information, or wish to withdraw your participation please contact the Principal Investigator by mail, phone or e-mail in accordance with the contact information listed on the first page of this consent.

If you have questions pertaining to your rights as a research participant; or to report concerns to this study, you should contact the Office of Research Integrity and Compliance at Carnegie Mellon University. Email: . Phone: 412-268-1901 or 412-268-5460.

Voluntary Participation
Your participation in this research is voluntary. You may discontinue participation at any time during the research activity.
Consent Form Questions
Please answer the following three questions.
I am age 18 or older. *
You are not allowed to participate in this study if your answer is "No" to this question.
I have read and understand the information above. *
You are not allowed to participate in this study if your answer is "No" to this question.
I want to participate in this research and continue with the field deployment study of the "Chorus" system. *
You are not allowed to participate in this study if your answer is "No" to this question.
Pre-Study Survey
Please answer the following four survey questions.
Which type of smart phone do you use? *
How often do you converse via instant messaging client (such as Facebook Messenger, Line, or Google Hangouts) on your phone?
How often do you converse via instant messaging client on your desktop/laptop? *
Assume you had a perfect/excellent personal assistant (which you could contact via instant messaging), please list at least one scenario which you would use this personal assistant for. *
Your answer
Sign-Up Information
Please tell us some information about you.
Your First Name *
Your answer
Your Last Name *
Your answer
Your Age *
Your answer
Your Occupation *
Your answer
You identify your gender as… *
Your Google Hangouts Account (Email) *
Your answer
The Displayed Name of Your Google Hangouts Account *
Your answer
Please click the following "Submit" button to sign up!
Never submit passwords through Google Forms.
This content is neither created nor endorsed by Google. Report Abuse - Terms of Service - Additional Terms