Some Questions
For a future blog post, it would be really cool if you could answer the questions below.
You have performed three studies, all testing the same idea in a similar manner, of which two turn out to be significant, and one is not significant. Before this line of studies, you judged it was equally likely that the null-hypothesis is true, as that it is false (a uniform prior). The significance level in all studies was 0.05 (analyses were pre-registered to guarantee the Type 1 error rate). You designed the study to have 80% power if there is a true effect (assume you succeeded perfectly). Based just on this pattern of significant and not-significant results, how likely do you think it is you observed these significant and non-significant outcomes when there is a real effect?
Answer a percentage (from 0 to 100%)
Your answer
You are planning a new study (in a different research line than in the question above). Beforehand, you again judge it is equally likely that the null-hypothesis is true, as that it is false (a uniform prior). You set the significance level at 0.05 (and pre-register this single confirmatory test to guarantee the Type 1 error rate). You design the study to have 80% power if there is a true effect (assume you succeed perfectly). What do you expect is the most likely outcome of this single study?
How likely do you think it is that you will observe the outcome of this single study that you have chosen above as most likely?
Answer a percentage (from 0 to 100%)
Your answer
Assume you performed the single study described above, and have observed a statistical difference (p < .05, but you don’t have any further details about effect sizes, exact p-values, or the sample size). Simply based on the fact that the study is statistically significant, how likely do you think it is you observed a significant difference because you were examining a true effect?
Answer a percentage (from 0 to 100%)
Your answer
Optional demographic questions and comments
Answer them if you want, but feel free to skip them and just hit 'send' at the bottom of the page.
What's your age?
Your answer
What your gender?
How many years have you worked in science?
if you don't work in science, answer 0
Your answer
Did you think any information required to answer the questions above was lacking (meaning the questions were badly formulated)? If so, which information did you need, and which additional assumptions did you make while answering the questions above? Do you have any other comments?
Your answer
Submit
Never submit passwords through Google Forms.
This content is neither created nor endorsed by Google. Report Abuse - Terms of Service - Additional Terms