Some Questions
For a future blog post, it would be really cool if you could answer the questions below.
You have performed three studies, all testing the same idea in a similar manner, of which two turn out to be significant, and one is not significant. Before this line of studies, you judged it was equally likely that the nullhypothesis is true, as that it is false (a uniform prior). The significance level in all studies was 0.05 (analyses were preregistered to guarantee the Type 1 error rate). You designed the study to have 80% power if there is a true effect (assume you succeeded perfectly). Based just on this pattern of significant and notsignificant results, how likely do you think it is you observed these significant and nonsignificant outcomes when there is a real effect?
Answer a percentage (from 0 to 100%)
Your answer
You are planning a new study (in a different research line than in the question above). Beforehand, you again judge it is equally likely that the nullhypothesis is true, as that it is false (a uniform prior). You set the significance level at 0.05 (and preregister this single confirmatory test to guarantee the Type 1 error rate). You design the study to have 80% power if there is a true effect (assume you succeed perfectly). What do you expect is the most likely outcome of this single study?
1) It is most likely that you will observe a true positive (i.e., there is an effect, and the observed difference is significant).
2) It is most likely that you will observe a true negative (i.e., there is no effect, and the observed difference is not significant)
3) It is most likely that you will observe a false positive (i.e., there is no effect, but the observed difference is significant).
4) It is most likely that you will observe a false negative (i.e., there is an effect, but the observed difference is not significant).
How likely do you think it is that you will observe the outcome of this single study that you have chosen above as most likely?
Answer a percentage (from 0 to 100%)
Your answer
Assume you performed the single study described above, and have observed a statistical difference (p < .05, but you don’t have any further details about effect sizes, exact pvalues, or the sample size). Simply based on the fact that the study is statistically significant, how likely do you think it is you observed a significant difference because you were examining a true effect?
Answer a percentage (from 0 to 100%)
Your answer
Optional demographic questions and comments
Answer them if you want, but feel free to skip them and just hit 'send' at the bottom of the page.
What's your age?
Your answer
What your gender?
male
female
Other:
How many years have you worked in science?
if you don't work in science, answer 0
Your answer
Did you think any information required to answer the questions above was lacking (meaning the questions were badly formulated)? If so, which information did you need, and which additional assumptions did you make while answering the questions above? Do you have any other comments?
Your answer
Submit
Never submit passwords through Google Forms.
This content is neither created nor endorsed by Google.
Report Abuse

Terms of Service

Additional Terms
Forms