Some Questions
For a future blog post, it would be really cool if you could answer the questions below.
Sign in to Google to save your progress. Learn more
You have performed three studies, all testing the same idea in a similar manner, of which two turn out to be significant, and one is not significant. Before this line of studies, you judged it was equally likely that the null-hypothesis is true, as that it is false (a uniform prior). The significance level in all studies was 0.05 (analyses were pre-registered to guarantee the Type 1 error rate). You designed the study to have 80% power if there is a true effect (assume you succeeded perfectly). Based just on this pattern of significant and not-significant results, how likely do you think it is you observed these significant and non-significant outcomes when there  is a real effect?
Answer a percentage (from 0 to 100%)
You are planning a new study (in a different research line than in the question above). Beforehand, you again judge it is equally likely that the null-hypothesis is true, as that it is false (a uniform prior). You set the significance level at 0.05 (and pre-register this single confirmatory test to guarantee the Type 1 error rate). You design the study to have 80% power if there is a true effect (assume you succeed perfectly). What do you expect is the most likely outcome of this single study?
Clear selection
How likely do you think it is that you will observe the outcome of this single study that you have chosen above as most likely?
Answer a percentage (from 0 to 100%)
Assume you performed the single study described above, and have observed a statistical difference (p < .05, but you don’t have any further details about effect sizes, exact p-values, or the sample size). Simply based on the fact that the study is statistically significant, how likely do you think it is you observed a significant difference because you were examining a true effect?
Answer a percentage (from 0 to 100%)
Optional demographic questions and comments
Answer them if you want, but feel free to skip them and just hit 'send' at the bottom of the page.
What's your age?
What your gender?
Clear selection
How many years have you worked in science?
if you don't work in science, answer 0
Did you think any information required to answer the questions above was lacking (meaning the questions were badly formulated)? If so, which information did you need, and which additional assumptions did you make while answering the questions above? Do you have any other comments?
Submit
Clear form
Never submit passwords through Google Forms.
This content is neither created nor endorsed by Google. - Terms of Service - Privacy Policy

Does this form look suspicious? Report

Google Forms