ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACAD
1
ClusterTitleDescriptionLearning outcomesMediumDeliveryTimeSupporting resourcesReference (if applicable)@dropdown
34
False PositiveFalse positivesSimulation of false positives (understanding the simulation in Simmons et al)Raise awareness on false positivesSIPS Workshop (possible to use for taching?)
36
Open Data and materialsInformation on PsychoPy/Converting it to Pavlovia/JS codeCrib sheet on how to convert your psychoPy experiment to Pavlovia/JS, common errors and how to solve them.Increase understanding of both PsychoPy and Pavlovia and how to convert from python to javascript. This is probably only relevant for students who are programming an experiment in PsychoPy and who want to conduct that experiment as an online studyWebsiteSeminars/Workshopsasynchronoushttps://www.youtube.com/user/peircej https://docs.google.com/document/d/183xmwDgSbnJZHMGf3yWpieV9Bx8y7fOCm3QKkMOOXFQ/edit
37
Open data and materialsOpen your courseThis repository contains two 20 minute presentations (and corresponding 10 minute hands on). One shows how you can introduce and illustrate open data analyses in one of your lectures. The other shows how you can open the data analyses of your bachelor thesis.How to open the data analyses of your bachelor thesisLectureLecture30 minuteshttps://osf.io/rz24g/
38
Open data and materialsCreative Common LicencesWhenever you use data/code/materials of others, it is important to know what you are allowed to do with it. This is specified in a licence. A class of popular licences in science are the Creative Common license. You can discuss these six Creative Common license with students, to make sure they understand what they mean when they encounter them. Here’s a description of the licences, as provided by Creative CommonsUnderstanding creative commons licensesPowerpointVerbal discussion10 minuteshttps://docs.google.com/presentation/d/1KbysZCt7C-RDE7v8Sa_aIB14ZQGTwhe5k2TZrJOHCM0/edit#slide=id.g6031556d35_0_152
39
Open data and materialsOpen accessIf you want to dive deeper into the topic of Open Access, you can introduce students to the various formats of Open Access publishing: e.g. full open access journals (gold), open access repositories (green), (see https://www.openaccess.nl/nl/wat-is-open-access).Raise awareness of open access papersPowerpointLecture30 minutesPart of: https://figshare.com/articles/presentation/Lecture_Open_Science_for_masterstudents/9963161
40
Open data and materialsA FAIRy taleSharing data, materials or code is only useful is if it is shared such that it is Findable, Accessible, Interoperable and Reusable. These are the FAIR principles, which are further specified in The FAIR Guiding Principles for scientific data management and stewardship. A nice format to introduce the FAIR principles is by using the brochure ‘A FAIRy tale’, in which the FAIR principles are introduced in fairytale story.

A possible teaching format is:
Split the group in four, each group has one topic:
Findable, Accessible, Interoperable, Reusable
Students read the sections of the FAIRytale regarding their topic (30-45 minutes)
Students present to fellow students (4x 5 minutes)
Importantly, students should strive to implement the FAIR principles when sharing data, materials or code. As a supervisor, you can use the FAIR principles as a check-list, to evaluate whether students comply with the FAIR principles.
Learning about FAIR dataBrochureSeminar30-120 minuteshttps://forskningsdata.dk/fairytale/index.html
41
Open data and materials; Academic life and cultureHow to be FAIR with your data: A teaching and training handbook for higher education institutionsA handbook which provides guidance, competence profiles and lesson plans on how to make research data FAIR. The main target groups are teachers and trainers at HE institutions.e.g. general principles and concepts of data management, data management planning, metadata, persistent identifiers, backup, data publishing - for more details consult the book, chapter 3.3.HandbookEngelhardt, C. et al. (2022). How to be FAIR with your data. Universitätsverlag Göttingen. https://doi.org/10.17875 /gup2022-1915
46
Reproducibility & replicability KnowledsgeLink to RIOT Science ClubLink to the RIOT Science club, links to videos that demonstrate concepts in an easy way.Understand concepts such as simulation, interpreting replications, and tons of others WebsiteSeminars/extra materialAround two hours per seminar/workshophttp://riotscience.co.uk/
47
Reproducibility & replicability KnowledsgeReproducibiliteaLink to Reproducibilitea, links to Reproducibilitea that discusses reproducibility from grassroot movementsIntro to open scienceWebsiteJournal club1-2 hours per reproducibilitea sessionhttps://reproducibilitea.org/
48
Reproducibility and replicability knowledge"Error Tight"Set of exercises for lab groups to identify places in their research workflow where errors may occur and pinpoint ways to address them. It is intended to be completed during a lab meeting.Avoiding errors in research, for example for students' thesesDiscussionLab meeting1 hourhttps://www.carleton.edu/perception-lab/open-science/error-tight/Strand, J. (2021). Error Tight: Exercises for Lab Groups to Prevent Research Mistakes. Retrieved from: ErrorTight.com. doi:10.31234/osf.io/rsn5y
49
Reproducibility and replicability knowledgeReappraising classic studiesDemonstrate importance of replicability as a means of reappraising robustness of 'classic' social psychology studies, e.g., MilgramGain understanding of why replicating studies, even classic studies, is importantPaper-based discussionShare supporting resource and discuss, with particular attention to the section on "From Replication to Reinterpretation"30-60 minshttps://journals.sagepub.com/doi/full/10.1177/0098628316677644#_i1Griggs, R. A. (2017). Milgram’s obedience study: A contentious classic reinterpreted. Teaching of Psychology, 44(1), 32-37.
50
Reproducibility and replicability knowledgeUK Consortium for undergraduate reproducible research projects Multi-site collaboration between universities for running undergraduate research projects in a reproducible manner. Students learn about open research practices, issues with the literature in psychology, methodological rigour etc and they participate in the design of the study which is preregistered on OSF before data collection. Student contributions are credited and they become co-authors on any final research outputs.Gain hands-on experience on how to make research reproducibleThesis projectsThesis projectsDuration of final year research projects (institution-dependent)Button, K. S., Chambers, C. D., Lawrence, N., & Munafò, M. R. Grassroots Training for Reproducible Science: A Consortium-Based Approach to Empirical Dissertation. Psychology Learning & Teaching. https://doi.org/10.1177/1475725719857659
51
Reproducibility and replicability knowledge"Prediction Markets"After teaching students about the Replication Crisis, I use the following activity within a seminar to reinforce learning. This is an exercise where students are shown a classic study (e.g., Srull & Wyer, 1979) and the associated findings, and then are given some preliminary detail about an associated Registered Replication Report (RRR; e.g., McCarthy et al., 2017) and are asked a series of questions such as "Do you think this will replicate or not?", "Why do you think it will replicate or not?", "are there any differences between when the study was performed and now, which might explain differences in findings? (e.g., historical, contextual)?"- To understand replication within the scientific method. - To distinguish direct/exact and conceptual replications. - To understand contemporarty issues in psychology, i.e., the replication crisis. - To critically assess original research findings and replication attempts. PowerPoint Slides summarising findings and outlining questions and/or original papers provided to groups of 3-6.Group activity/discussion30 minutesSee references to use for this activity >Develop by Charlotte Pennington using the following original studies and their RRRs: Srull, T. K., & Wyer, R. S. (1979). The role of category accessibility
in the interpretation of information about persons: Some determinants and implications. Journal of Personality and
Social Psychology, 37, 1660–1672. doi:10.1037/0022-3514.37.10.1660 McCarthy, R. J., Skowronski, J. J., Verschuere, B., Meijer, E. H., Jim, A., Hoogesteyn, K., ... & Yıldız, E. (2018). Registered replication report on Srull and Wyer (1979). Advances in Methods and Practices in Psychological Science, 1(3), 321-336. Verschuere, B., Meijer, E. H., Jim, A., Hoogesteyn, K., Orthey, R., McCarthy, R. J., ... & Yıldız, E. (2018). Registered replication report on Mazar, Amir, and Ariely (2008). Advances in Methods and Practices in Psychological Science, 1(3), 299-317. Experiment 1 from Mazar, N., Amir, O., & Ariely, D. (2008). The dishonesty of honest
people: A theory of self-concept maintenance. Journal of Marketing Research, 45, 633–644. doi:10.1509/jmkr.45.6.633
52
reproducibility and replicability knowledgeThe Turing WayThe Turing Way is an open source community-driven guide to reproducible, ethical, inclusive and collaborative data science.General overview/intro to open and reproducible scienceWebsiteReadingWeeks to read and discuss chaptershttps://the-turing-way.netlify.app/welcome.html
53
Reproducibility and replicability knowledgePublication bias - harms of file drawing of null effectsBen Goldacre's TedX talk on publication bias can serve as a really nice springboard for discussions about the consequences of publication bias, ethics of publishing, pre-registration, and so onUnderstanding academic publishing and publication biasVideo followed by discussionSeminar30-45 minshttps://www.youtube.com/watch?v=RKmxL8VYy0M
54
Reproducibility and replicability knowledgeMyth-Busting With Infographicsa creative and engaging myth refutation assignment that uses an infographic format to promote the refutation of psychological misconceptionsImprove students' abilities to spot and assess myths in psychologyIndividual or group assignmentAssignment15 hourshttps://journals.sagepub.com/na101/home/literatum/publisher/sage/journals/content/topa/2021/topa_48_2/0098628320977269/20210306/images/large/10.1177_0098628320977269-table3.jpegCavazos, J. T., Stern, W., Stephenson, E., & Heddy, B. (2021). Myth-Busting With Infographics: Do Creative Assignments Help Students Learn? Teaching of Psychology, 48(2), 117–123. https://doi.org/10.1177/0098628320977269
55
Reproducibility and replicability knowledgeIntro and History of Open SciencePaper offering a link to an introductory lecture on the reproducibility crisis in psychology (https://osf.io/mh9pe/). Based on this paper, teachers can create their own lectureIncrease awareness and understanding of the reproducibility crisisLectureLecturea few hours to create the lecture, approximately 2 hours for the actual lecturehttps://journals.sagepub.com/doi/10.1177/0098628318762900Chopik WJ, Bremner RH, Defever AM, Keller VN. How (and Whether) to Teach Undergraduates About the Replication Crisis in Psychological Science. Teaching of Psychology. 2018;45(2):158-163. doi:10.1177/0098628318762900
56
Reproducibility and replicability knowledge"Open Lunch" series at UoLRecordings and discussion from regular series of talks at the University of Leeds exploring the culture and practice of open and reproducible research in different disciplinesGeneral (preprints, preregistration, open data, barriers to open research)Blog/videoReading/viewing1 hour per lunch sessionhttps://leedsunilibrary.wordpress.com/tag/open-lunch/
57
reproducibility and replicability knowledgeLecture on if to teach about the replication crisis or not, how to avoid turning students away from the areaLcture session covering replciation crisis but emphasis on being a positiveGetting familiar with the replication crisis and main terms of open science, as well as initiatives to improve psychological science. Students also learn how to do good research themselvesLectureLecture1 hourhttps://osf.io/mh9pe/ https://journals.sagepub.com/doi/abs/10.1177/0098628318762900
58
Reproducibility and replicability knowledgeCREP projectProject that was e.g. conducted by Bachelor students as part of a intro to stats course. Contains information on how to do a full replication, with which students got practical experience with replicationsGetting hands-on experience with replicationAssignmentPracticalsWeekshttps://docs.google.com/document/d/1Wtv514fz-_xDgH7UdOQcxUWvB8RtdKU1SCm2G04YMEs/edit, https://osf.io/stv7w/, https://docs.google.com/document/d/1iB6ANRrIwi8ObfB2DBgrR6jyzgSXsedt2Sb8XZpc9Ic/edit, https://osf.io/3vaem/ https://online.ucpress.edu/collabra/article/5/1/5/112984/A-Demonstration-of-the-Collaborative-Replication
59
Reproducibility and replicability knowledgeTeaching the how and why of replication studiesModule 1: The credibility crisis. Module 2: Replications as a way of highlighting what is known. Module 3: Resource planning and piloting. Module 4: PreregistrationModule 1 Learning goals: The credibility crisis revealed weaknesses in research; these weaknesses are caused by hidden processes that don’t show up in published reports. Module 2 Learning goals: Learning goals: Close replications test the credibility of a finding. They work best with good documentation & structured ways of choosing replication targets. Module 3 Learning goals: Learning goals: Resources are critical for making your replication precise; you should think through the sample your resources allow & use those to calculate your power. Module 4 Learning goals: To preregister something, create an OSF project & put the replication recipe in the registry. There’s evidence this helps make research more credibleLectures and seminarsLectures/seminars4 weeks?https://forrt.org/educators-corner/004-teaching-why-how-replication/Credit to Patrick Forscher: https://forrt.org/educators-corner/004-teaching-why-how-replication/
60
Reproducibility and replicability knowledgeIs there a reproducibility crisis in science? - Matt AnticoleVery brief video introduction to reproducibility crisis in science. Talks about incentive structure in academia. Very broad, approachable and short. Could be useful for UG students.Raise awareness of reproducibility issues in ScienceYoutube videoAnimation5 minshttps://www.youtube.com/watch?v=FpCrY7x5nEE
61
reproducibility and replicability knowledge/conceptual and statistical knowledgeEscape Room Review ActivityAn exam review activity in which students answered questions and solved puzzles in order to escape the review session. In a research methods in psychology (300-level) course, students solved a series of puzzles and review questions.Can be used to cover a range of learning outcomes depending on content used. Particularly helpful for gaining a better understanding of basic research methods, key ideas in reproducibility, and statistical concepts.WorkshopOnline/in class1-3 hours (flexible)https://journals.sagepub.com/doi/full/10.1177/0098628320901383 LaPaglia, J. A. (2020). Escape the Evil Professor! Escape Room Review Activity. Teaching of Psychology, 47(2), 141–146. https://doi.org/10.1177/0098628320901383
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138