Replication in political science

JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

***Note that as of 2/12/1014, we have written up our initial survey results in a post on the Monkey Cage (http://wapo.st/1dIs4v7). However, we welcome further survey participation, and further input will continue to be very valuable to the development of the initiative.***

We are developing an initiative aimed at promoting replications of quantitative work in political science. The initiative aims to create a site that will publish and organize replications done by graduate students in their courses. By “replications,” we refer to conducting re-analyses using the original data/code, as well as robustness checks.

We are now laying groundwork for the project, supported by a planning grant from the Sloan Foundation. In this phase, we have several goals, including 1. Gathering detailed information about replications performed in methods courses and other graduate work 2. Forming a network of professors who would be interested in encouraging their students to share their replications on the site.

Best,
Stephanie Wykstra (contact: stephanie.wykstra@gmail.com), Researcher (independent)
Nicole Janz, PhD candidate and Social Science Methods Lecturer at Cambridge University
Seth Werfel, Graduate student, Political Science, Stanford University

Please provide your name. *

One of our aims is to form a network of professors and graduate students who are doing replications. To this end, it will be helpful to know who is filling out the survey. Your personal information will not be shared except with project organizers. You may enter "anonymous" if you prefer.

Please provide your institution.

It will be helpful to us to know which institution you are affiliated with. This information will not be shared except with project organizers. You may leave blank if you prefer.

1. What is your professional status? *

Professor

Graduate student

Other:

2. Do you have students do replications (i.e., re-analyses/robustness checks) in any courses that you regularly teach? (If you are a student, have you done replications in courses?) *

If you neither have students doing replications, nor are a student doing replications, skip to question 7.

Yes

3. If you either teach or have taken a course in which you attempted a replication study, how would you describe the course?

First-year methods course

More advanced course

Both first-year and more advanced courses

Other:

Clear selection

4. If students are assigned replications in a course, how do they choose which study to replicate?

They are given a particular study to replicate

They choose from a list of options

They choose a study themselves

It varies from course to course

Other:

Clear selection

5. What are the problems that students most often encounter when attempting to exactly replicate the reported results?

(Please check all that apply, including those that arise with some frequency and excluding those that very rarely arise).

The data weren't available online and the author did not respond to a request to provide them.

The code wasn't made available online and the author did not respond to a request to provide it.

Variables and their transformations were not sufficiently described.

Model specifications were not sufficiently described.

Other:

6. What is the review process for replications done in the course?

Please check each option that applies.

The instructor reviews a draft of the replication, which is then revised for a final assignment.

At least one other student replicates each student's replication

There are course Teaching Assistants who assist in the replications

Other:

7. Do you supervise students who do replication outside of courses (e.g. as an independent project, for their thesis, etc)? (If you are a student, have you done replications outside of courses?) *

Yes

Very rarely

8. Which statement describes the extent to which students now share their replications?

Please answer N/A if you neither assign replications or supervise students doing replication.

Students never (or almost never) share work outside of the class

The students are asked to post their replications and associated materials in a repository or course website, etc.

N/A

Other:

Clear selection

9. Do you think it would be valuable to create a site for disseminating replications? *

Again, where "replications" refers to re-analyses/robustness checks. The site would allow for sharing replication papers and other research materials (code, data), and would also allow original authors and others to comment on the replications.

Yes

Maybe: I'd be interested in the site and would want to learn more.

10. Please expand on your answer to question 9.

11. Would you encourage students to share their replications on such a site? (If you do replications, would you be interested in sharing work on a replication site?) *

Yes

Depends

Probably not

N/A (I neither have students doing replications nor am I doing them myself).

12. What best describes your view about quality control on a replication site? *

The site would have to have referees carefully rejecting/accepting student submissions based on their quality

The site could allow un-refereed submissions and still be valuable; other mechanisms of quality control would be useful, such as noting when students had replicated each others' work, allowing open discussion on the site, inviting responses from original authors, and so on.

Other:

Please provide your email address.

We are forming an email list for the purpose of sending updates on the project. We will not share your personal information. (If you'd prefer not to share your email, you may leave blank.)

Further comments or suggestions?

Submit

Clear form

Never submit passwords through Google Forms.

This content is neither created nor endorsed by Google. - Terms of Service - Privacy Policy

Does this form look suspicious? Report

Forms