| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | AC | AD | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Cluster | Title | Description | Learning outcomes | Medium | Delivery | Time | Supporting resources | Reference (if applicable) | |||||||||||||||||||||
2 | Preregistration | Fake preregistration | Students use an existing study to construct a hypothetical pre-registration for the same study using the information provided in the existing study | Gain an understanding of preregistration | Paper-based discussion | Group activity/take home exercise | 30 mins? | preregistration templates: https://osf.io/zab38/wiki/home/ | ||||||||||||||||||||||
3 | Conceptual and statistical knowledge | p-hacking | Demonstrate an example of how a researcher might p-hack using a shiny app | Get a feeling for how p-hacking works | Shiny app | In class demo? | 10 minutes | https://shinyapps.org/apps/p-hacker/, https://projects.fivethirtyeight.com/p-hacking/ | ||||||||||||||||||||||
4 | Reproducibility and replicability knowledge | "Error Tight" | Set of exercises for lab groups to identify places in their research workflow where errors may occur and pinpoint ways to address them. It is intended to be completed during a lab meeting. | Avoiding errors in research, for example for students' theses | Discussion | Lab meeting | 1 hour | https://www.carleton.edu/perception-lab/open-science/error-tight/ | Strand, J. (2021). Error Tight: Exercises for Lab Groups to Prevent Research Mistakes. Retrieved from: ErrorTight.com. doi:10.31234/osf.io/rsn5y | |||||||||||||||||||||
5 | Preregistration | Analytic flexibility | Provide student groups with the same dataset and ask them all to test the same hypotheses and cross-examine results from different groups to demonstrate how analytic flexibility changes results, and the importance of pre-registration, etc | Get a feeling for analytic flexibility and how to prevent it | Paper-based discussion | Group activity/take home exercise | 30 minutes | https://journals.sagepub.com/doi/10.1177/2515245917747646 | Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 337–356. https://doi.org/10.1177/2515245917747646 | |||||||||||||||||||||
6 | Reproducibility and replicability knowledge | Reappraising classic studies | Demonstrate importance of replicability as a means of reappraising robustness of 'classic' social psychology studies, e.g., Milgram | Gain understanding of why replicating studies, even classic studies, is important | Paper-based discussion | Share supporting resource and discuss, with particular attention to the section on "From Replication to Reinterpretation" | 30-60 mins | https://journals.sagepub.com/doi/full/10.1177/0098628316677644#_i1 | Griggs, R. A. (2017). Milgram’s obedience study: A contentious classic reinterpreted. Teaching of Psychology, 44(1), 32-37. | |||||||||||||||||||||
7 | Conceptual and statistical knowledge | Correlation is not causation | A website which demonstrates how two variables can be closely correlated, but are clearly unrelated. Helps students to understand how correlation does not equal causation, using 'real world' data | Gain a clear overview of how correlation is not causation | Website | Resource to share with students, can include in lecture slides | 5 mins | http://www.tylervigen.com/spurious-correlations | ||||||||||||||||||||||
8 | Reproducibility and replicability knowledge | UK Consortium for undergraduate reproducible research projects | Multi-site collaboration between universities for running undergraduate research projects in a reproducible manner. Students learn about open research practices, issues with the literature in psychology, methodological rigour etc and they participate in the design of the study which is preregistered on OSF before data collection. Student contributions are credited and they become co-authors on any final research outputs. | Gain hands-on experience on how to make research reproducible | Thesis projects | Thesis projects | Duration of final year research projects (institution-dependent) | Button, K. S., Chambers, C. D., Lawrence, N., & Munafò, M. R. Grassroots Training for Reproducible Science: A Consortium-Based Approach to Empirical Dissertation. Psychology Learning & Teaching. https://doi.org/10.1177/1475725719857659 | ||||||||||||||||||||||
9 | Reproducibility and replicability knowledge | "Prediction Markets" | After teaching students about the Replication Crisis, I use the following activity within a seminar to reinforce learning. This is an exercise where students are shown a classic study (e.g., Srull & Wyer, 1979) and the associated findings, and then are given some preliminary detail about an associated Registered Replication Report (RRR; e.g., McCarthy et al., 2017) and are asked a series of questions such as "Do you think this will replicate or not?", "Why do you think it will replicate or not?", "are there any differences between when the study was performed and now, which might explain differences in findings? (e.g., historical, contextual)?" | - To understand replication within the scientific method. - To distinguish direct/exact and conceptual replications. - To understand contemporarty issues in psychology, i.e., the replication crisis. - To critically assess original research findings and replication attempts. | PowerPoint Slides summarising findings and outlining questions and/or original papers provided to groups of 3-6. | Group activity/discussion | 30 minutes | See references to use for this activity > | Develop by Charlotte Pennington using the following original studies and their RRRs: Srull, T. K., & Wyer, R. S. (1979). The role of category accessibility in the interpretation of information about persons: Some determinants and implications. Journal of Personality and Social Psychology, 37, 1660–1672. doi:10.1037/0022-3514.37.10.1660 McCarthy, R. J., Skowronski, J. J., Verschuere, B., Meijer, E. H., Jim, A., Hoogesteyn, K., ... & Yıldız, E. (2018). Registered replication report on Srull and Wyer (1979). Advances in Methods and Practices in Psychological Science, 1(3), 321-336. Verschuere, B., Meijer, E. H., Jim, A., Hoogesteyn, K., Orthey, R., McCarthy, R. J., ... & Yıldız, E. (2018). Registered replication report on Mazar, Amir, and Ariely (2008). Advances in Methods and Practices in Psychological Science, 1(3), 299-317. Experiment 1 from Mazar, N., Amir, O., & Ariely, D. (2008). The dishonesty of honest people: A theory of self-concept maintenance. Journal of Marketing Research, 45, 633–644. doi:10.1509/jmkr.45.6.633 | |||||||||||||||||||||
10 | academic life and culture | Hackathon on Hackathons | Hackathons are a way to collaborate and problem solve that may be new to many students. This resource introduces the concept of hackathons and how to lead one. | Know how to effectively plan and lead a hackathon | Google Doc | Hackathon | anything from 2 hours to days | https://docs.google.com/document/d/1aC5MdVQhmZ_SQzvrv7fxWF1Asf5sVhYD2nPv2eE0gRE/edit | ||||||||||||||||||||||
11 | academic life and culture | Psychology Job Wiki | This crowdsourced site is intended to provide information to applicants about the status of academic job searches occurring in the academic year in all areas of psychology. The wikis for previous years are still available. | Available jobs in psychology (tenure track and other) | Job wiki | Sending link to students | 5 minutes | http://psychjobsearch.wikidot.com/ | ||||||||||||||||||||||
12 | reproducibility and replicability knowledge/conceptual and statistical knowledge | Escape Room Review Activity | An exam review activity in which students answered questions and solved puzzles in order to escape the review session. In a research methods in psychology (300-level) course, students solved a series of puzzles and review questions. | Can be used to cover a range of learning outcomes depending on content used. Particularly helpful for gaining a better understanding of basic research methods, key ideas in reproducibility, and statistical concepts. | Workshop | Online/in class | 1-3 hours (flexible) | https://journals.sagepub.com/doi/full/10.1177/0098628320901383 | LaPaglia, J. A. (2020). Escape the Evil Professor! Escape Room Review Activity. Teaching of Psychology, 47(2), 141–146. https://doi.org/10.1177/0098628320901383 | |||||||||||||||||||||
13 | Conceptual and statistical knowledge | Use simulation to understand False Positives | use simulation to recreate the conditions in Simmons et al 2011 to help students understand the extent to which certain research practics inflate the Type I error rate. | understand QRP such as p hacking; write simple R codes for simulation; | R | flexible, group/individual/homework | 1hour | https://journals.sagepub.com/doi/full/10.1177/0956797611417632; https://osf.io/43zfq/ | ||||||||||||||||||||||
14 | academic life and culture | Whose open science are we talking about? Research paradigm and open science | have a structured discussion on how open science is conceptualised and identify challenges associated with different types of research (paradigms). Questions for discussion: what comes to your mind when you think about open science? what are the controversies around open science? what are your concerns for adopting open science? how would you define open science in a discipline-specific manner? | develop a more holistic and critical understanding of open science; address potential concerns and/or misconceptions about open science; promote epistemological pluralism | discussion | group activity | 1hour | https://bpspsychub.onlinelibrary.wiley.com/hub/journal/20448309/homepage/callforpapersopenscience | ||||||||||||||||||||||
15 | Conceptual and statistical knowledge | Hypothesis testing with proverbs | The current exercise consisted of developing a hypothesis to test a proverb as if it were a theory, designing and conducting a relevant study, and producing a poster to present the findings. The exercise is presented as an alternative or complement to writing laboratory reports in undergraduate psychology research methods classes. A standard poster presentation was used as the assessment for this task | (a) be familiar with the basic principles of research design and sampling, (b) be able to consider ethical issues relevant to research in psychology, (c) be able to write and format psychological laboratory reports according to American Psychological Association guidelines, and (d) be able to plan and conduct a psychology experiment. | Research methods class | flexible, group/individual/homework | 1 week+ | https://journals.sagepub.com/doi/full/10.1177/0098628319853937 | Gamble, T., Turner-Cobb, J. M., Walker, I., & Holt, N. (2019). Putting Proverbs to the Test: An Engaging Approach for Developing Students’ Hypothesis Testing Skills. Teaching of Psychology, 46(3), 230–235. https://doi.org/10.1177/0098628319853937 | |||||||||||||||||||||
16 | Conceptual and statistical knowledge | Dodgy research papers | An exercise where students read some "dodgy"research papers (a good example would be the examination of the hot-crazy-matrix, Sumich, A. (2020). Testing the Hot-Crazy Matrix: Borderline personality traits in attractive women and wealthy unattractive men are relatively favoured by the opposite sex. Personality and Individual Differences. And identify flaws in the research study: did not ask for gender identity, did not ask for sexual identity,... | Identify flaws in research studies. Identify methodological and analytical issues. Consequences for reliability and validity. | Workshop | Group activity (read paper at home beforehand) | 40 minutes. | Sumich, A. (2020). Testing the Hot-Crazy Matrix: Borderline personality traits in attractive women and wealthy unattractive men are relatively favoured by the opposite sex. Personality and Individual Differences. | ||||||||||||||||||||||
17 | Conceptual and statistical knowledge | Effect size (Cohen's d) | Demonstrate an example of how a researcher might check Cohen's d using a shiny app | Understand how Cohen's D and effect sizes work | Shiny app | In class demo? | 5 minutes | https://rpsychologist.com/cohend/ | ||||||||||||||||||||||
18 | Conceptual and statistical knowledge | Visual interpretation of correlation (coefficients) | Demonstrate how to interpret strength of correlation in a scatterplot | Understand how to interpret the strengh of a correlation in a scatterplot | Website | In-class demo/ Virtual learning environment resources | 10 mins | http://guessthecorrelation.com/ | N/A | |||||||||||||||||||||
19 | Conceptual and statistical knowledge | Using parapsychology as a way to communicate about false positives | Wiseman et al (2019) compared two types of journal articles published in the European Journal of Parapsychology- a type of Registered Report, and typical (non-preregistered) articles. They showed that the non-preregistered articles were much more likely to report a positive result than the pre-registered articles. If you can get students to accept that people are probably not psychic, and that therefore, positive effects are likely to be false positive errors, | -Understand false positive errors; understand how publication bias allows false positives to proliferate; understand how pre-registration reduces the false positive error rate | Podcast and article-based lecture | Seminar | 2 hours | https://peerj.com/articles/6232/; https://www.theskepticsguide.org/podcasts/episode-798 | ||||||||||||||||||||||
20 | Open Data and materials | Information on PsychoPy/Converting it to Pavlovia/JS code | Crib sheet on how to convert your psychoPy experiment to Pavlovia/JS, common errors and how to solve them. | Increase understanding of both PsychoPy and Pavlovia and how to convert from python to javascript. This is probably only relevant for students who are programming an experiment in PsychoPy and who want to conduct that experiment as an online study | Website | Seminars/Workshops | asynchronous | https://www.youtube.com/user/peircej https://docs.google.com/document/d/183xmwDgSbnJZHMGf3yWpieV9Bx8y7fOCm3QKkMOOXFQ/edit | ||||||||||||||||||||||
21 | Academic Life and Culture | Syllabi to locate other gender, ethnicity and race | Demonstrate to others that the syllabi includes people of varying race, ethinicity, gender and that science can be done by everyone | Increase understanding of diversity and how to include everyone into science | Syllabi | Reading/discussion | 30 minutes for a discussion | Skitka, L. J., Melton, Z. J., Mueller, A. B., & Wei, K. Y. (2021). The gender gap: Who is (and is not) included on graduate-level syllabi in social/personality psychology. Personality and Social Psychology Bulletin, 47(6), 863-872. | ||||||||||||||||||||||
22 | Reproducibility & replicability Knowledsge | Link to RIOT Science Club | Link to the RIOT Science club, links to videos that demonstrate concepts in an easy way. | Understand concepts such as simulation, interpreting replications, and tons of others | Website | Seminars/extra material | Around two hours per seminar/workshop | http://riotscience.co.uk/ | ||||||||||||||||||||||
23 | reproducibility and replicability knowledge | The Turing Way | The Turing Way is an open source community-driven guide to reproducible, ethical, inclusive and collaborative data science. | General overview/intro to open and reproducible science | Website | Reading | Weeks to read and discuss chapters | https://the-turing-way.netlify.app/welcome.html | ||||||||||||||||||||||
24 | Reproducibility & replicability Knowledsge | Reproducibilitea | Link to Reproducibilitea, links to Reproducibilitea that discusses reproducibility from grassroot movements | Intro to open science | Website | Journal club | 1-2 hours per reproducibilitea session | https://reproducibilitea.org/ | ||||||||||||||||||||||
25 | Conceptual and statistical knowledge | Methodological Thinking Drills | A novel technique for improving introductory psychology students’ ability to identify and evaluate methodological concepts in real-life media by prompting them to analyze secondary source news and blog articles. | Improving critical thinking about research concepts and methods | Seminar/workshop | Individual/group activity | 60 mins? | https://journals.sagepub.com/doi/full/10.1177/0098628319888093 | Cascio, T. V. (2020). Methodological Thinking Drills: A Novel Technique for Boosting Real-World Methodological Thinking Ability. Teaching of Psychology, 47(1), 63–67. https://doi.org/10.1177/0098628319888093
| |||||||||||||||||||||
26 | Conceptual and statistical knowledge | Teaching Integrity in Empirical Research: A Protocol for Documenting Data Management and Analysis | paper showing a workflow for teaching students how to document their data and analyses | Increasing understanding of how to document their data and analysis | Journal article | Reading | 60 minutes? | https://www.bitss.org/wp-content/uploads/2014/06/teaching-integrity-in-empirical-research-required1.pdf | Ball, R., & Medeiros, N. (2012). Teaching integrity in empirical research: A protocol for documenting data management and analysis. The Journal of Economic Education, 43(2), 182-189. | |||||||||||||||||||||
27 | Conceptual and statistical knowledge | Teaching of Type I and Type II error | A paper that shows a good practice for teaching of Type I and Type 2 error. It makes intuitive parallels between null hypothesis significance testing and the juidicial process. | Understanding of Type I/ II errors | Journal article | Reading | 15 mins | https://www.bitss.org/wp-content/uploads/2014/06/teaching-integrity-in-empirical-research-required1.pdf | Feinberg, W. E. (1971). Teaching the type I and type II errors: The judicial process. The American Statistician, 25(3), 30-32. | |||||||||||||||||||||
28 | Reproducibility and replicability knowledge | Publication bias - harms of file drawing of null effects | Ben Goldacre's TedX talk on publication bias can serve as a really nice springboard for discussions about the consequences of publication bias, ethics of publishing, pre-registration, and so on | Understanding academic publishing and publication bias | Video followed by discussion | Seminar | 30-45 mins | https://www.youtube.com/watch?v=RKmxL8VYy0M | ||||||||||||||||||||||
29 | Reproducibility and replicability knowledge | Myth-Busting With Infographics | a creative and engaging myth refutation assignment that uses an infographic format to promote the refutation of psychological misconceptions | Improve students' abilities to spot and assess myths in psychology | Individual or group assignment | Assignment | 15 hours | https://journals.sagepub.com/na101/home/literatum/publisher/sage/journals/content/topa/2021/topa_48_2/0098628320977269/20210306/images/large/10.1177_0098628320977269-table3.jpeg | Cavazos, J. T., Stern, W., Stephenson, E., & Heddy, B. (2021). Myth-Busting With Infographics: Do Creative Assignments Help Students Learn? Teaching of Psychology, 48(2), 117–123. https://doi.org/10.1177/0098628320977269
| |||||||||||||||||||||
30 | Reproducibility and replicability knowledge | Intro and History of Open Science | Paper offering a link to an introductory lecture on the reproducibility crisis in psychology (https://osf.io/mh9pe/). Based on this paper, teachers can create their own lecture | Increase awareness and understanding of the reproducibility crisis | Lecture | Lecture | a few hours to create the lecture, approximately 2 hours for the actual lecture | https://journals.sagepub.com/doi/10.1177/0098628318762900 | Chopik WJ, Bremner RH, Defever AM, Keller VN. How (and Whether) to Teach Undergraduates About the Replication Crisis in Psychological Science. Teaching of Psychology. 2018;45(2):158-163. doi:10.1177/0098628318762900 | |||||||||||||||||||||
31 | All potentially | Hagen Culumlative Science Project | Steps to use replication as a target outcome for final year projects emphasis on using original data first then developing to preregistration and so on with aim for publication outcome | Lots really, open science princ, replicaiton, higher level analysis methods as access to larger data sets to start with with guided analysis in class | Lecture, seminar, one-to-one support | As needed, would work with online and in person | Long, final year stratergy | https://journals.sagepub.com/doi/pdf/10.1177/1475725719868149 | ||||||||||||||||||||||
32 | Conceptual and statistical knowledge | Interpreting Confidence Intervals | Teach students about various ways to represent variance in data, such as SD, SE, or confidence intervals. Work out together what method might be best. | Explain how to interpret CI and how to think about a CI | Workshop, interactive. First presentation, later workshop. | Seminar. | 30-40 minutes. | https://rpsychologist.com/d3/ci/ | ||||||||||||||||||||||
33 | Reproducibility and replicability knowledge | "Open Lunch" series at UoL | Recordings and discussion from regular series of talks at the University of Leeds exploring the culture and practice of open and reproducible research in different disciplines | General (preprints, preregistration, open data, barriers to open research) | Blog/video | Reading/viewing | 1 hour per lunch session | https://leedsunilibrary.wordpress.com/tag/open-lunch/ | ||||||||||||||||||||||
34 | Conceptual and statistical knowledge | Multiple choice questions at the end of the paper | You read a series of questions and you answer questions based on the paper | Knowledge learning | multiple choice questions | In-class discussion | 30 minutes per set of questions | Journal of Fluency Disorders | ||||||||||||||||||||||
35 | Preregistration | Preregistration for Dissertation Projects | Document for students about to start their dissertation getting them to preregister their dissertation ideas ahead of meeting with the supervisor | Learning how to preregister dissertation ideas ahead of meeting the supervisor | Out of class/Within final research methods classes | At-home assignment: preregistering your thesis | a week? | https://osf.io/preprints/metaarxiv/epgjd/ | ||||||||||||||||||||||
36 | FORRT | Gilad Feldman | Course materials, course and workshop videos, and workshop materials of courses Gilad Feldman teaches | Depends per course | Lectures/full courses | Syllabus, prerecorded videos | weeks | https://mgto.org/teaching-courses/ | ||||||||||||||||||||||
37 | reproducibility and replicability knowledge | Lecture on if to teach about the replication crisis or not, how to avoid turning students away from the area | Lcture session covering replciation crisis but emphasis on being a positive | Getting familiar with the replication crisis and main terms of open science, as well as initiatives to improve psychological science. Students also learn how to do good research themselves | Lecture | Lecture | 1 hour | https://osf.io/mh9pe/ https://journals.sagepub.com/doi/abs/10.1177/0098628318762900 | ||||||||||||||||||||||
38 | Conceptual and statistical knowlegde | Bem's "Feeling the Future" - the importance of prior plausibility | Students are introduced to Bem's feeling the future studies as an example of a hypothesis with very low prior plausibility. Use this as a springboard for discussing what can and cannot be inferred from p values | Interpretation of p values; understanding the role of plausibility in scientific inference | Seminar | Seminar/pre-seminar reading | 1 hour | https://psycnet.apa.org/record/2011-01894-001 | ||||||||||||||||||||||
39 | Conceptual and Statistical knowledge | Debunking Handbook | summarises the current state of the science of misinformation and its debunking. It was written by a team of 22 prominent scholars of misinformation and its debunking, and it represents the current consensus on the science of debunking for engaged citizens, policymakers, journalists, and other practitioners. | Learn to identify misinformation; effectively prevent and debunk misinformation | Website | Reading/seminar | 1 hour | https://www.climatechangecommunication.org/debunking-handbook-2020/ | ||||||||||||||||||||||
40 | Reproducibility and replicability knowledge | CREP project | Project that was e.g. conducted by Bachelor students as part of a intro to stats course. Contains information on how to do a full replication, with which students got practical experience with replications | Getting hands-on experience with replication | Assignment | Practicals | Weeks | https://docs.google.com/document/d/1Wtv514fz-_xDgH7UdOQcxUWvB8RtdKU1SCm2G04YMEs/edit, https://osf.io/stv7w/, https://docs.google.com/document/d/1iB6ANRrIwi8ObfB2DBgrR6jyzgSXsedt2Sb8XZpc9Ic/edit, https://osf.io/3vaem/ | https://online.ucpress.edu/collabra/article/5/1/5/112984/A-Demonstration-of-the-Collaborative-Replication | |||||||||||||||||||||
41 | Conceptual and statistical knowledge | Unequal variances in a t-test | Through a Shiny app and accompanying worksheet, students observe the effect of unequal variances on the distribution of p values and the Type 1/Type 2 error rate | Understanding how violations of the assumption of homogeneity of variance impacts on the false positive rate when using the Student's t test | Shiny app | Workshop/asynchronous learning materials | 20 minutes | https://tomfaulkenberry.shinyapps.io/unequalVariances | ||||||||||||||||||||||
42 | Ressource | Student initiative for open science workshops | Workshops and recorded lectures targeting psychology students from psychology students | Depends per lecture/workshop | Lecture/workshop | Classes | 2 hours per session | https://studentinitiativeopenscience.com/workshops/ | ||||||||||||||||||||||
43 | Conceptual and statistical knowledge | Statistical rethinking | A blog on statistical rethinking | Reading and seeing R code about statistical rethinking. | Blog post | Reading | 15 minutes | https://david-salazar.github.io/2020/04/19/statistical-rethinking-week-1/ | ||||||||||||||||||||||
44 | Reproducibility and replicability knowledge | Teaching the how and why of replication studies | Module 1: The credibility crisis. Module 2: Replications as a way of highlighting what is known. Module 3: Resource planning and piloting. Module 4: Preregistration | Module 1 Learning goals: The credibility crisis revealed weaknesses in research; these weaknesses are caused by hidden processes that don’t show up in published reports. Module 2 Learning goals: Learning goals: Close replications test the credibility of a finding. They work best with good documentation & structured ways of choosing replication targets. Module 3 Learning goals: Learning goals: Resources are critical for making your replication precise; you should think through the sample your resources allow & use those to calculate your power. Module 4 Learning goals: To preregister something, create an OSF project & put the replication recipe in the registry. There’s evidence this helps make research more credible | Lectures and seminars | Lectures/seminars | 4 weeks? | https://forrt.org/educators-corner/004-teaching-why-how-replication/ | Credit to Patrick Forscher: https://forrt.org/educators-corner/004-teaching-why-how-replication/ | |||||||||||||||||||||
45 | FAIR | Changing the Culture of Data Management and Sharing: A Workshop | A series of videos about data management | Teaches about and discusses data maangement | Video followed by discussion | Video | approximately 10 minutes per video + discussion time | https://www.youtube.com/playlist?list=PLGTMA6QkejfgsxahcRn0HwFuF8m8lOnOQ | ||||||||||||||||||||||
46 | Conceptual and Statistical Knowledge | Identify and Evaluate Sources: Evaluate Science Literature > Question the Science | Library Guide from Nazarbayev University Library with example questions related to identifying bias and other methodological concerns in scientific literature. Contains "Reading Scholarly Papers" infographic. Suitable for introductory contexts. | Getting tools to critically evaluate scientific papers | Library Guide (Website) | Seminar | 2 hours discussion | https://nu.kz.libguides.com/c.php?g=683093&p=4875096 | ||||||||||||||||||||||
47 | Conceptual and Statistical Knowledge | Understanding Bayesian parameter estimation | A practicial example of how Bayesian parameter works by using a case study of estimating a prisoner's IQ. Useful for introductory courses to (Bayesian) statistics | Develop an understanding of how Bayesina parameter estimation works | Paper/ Shiny app | Seminar | 15 mins | https://mvasilev.shinyapps.io/shiny/ | https://journals.sagepub.com/doi/abs/10.1177/0963721416643289 | |||||||||||||||||||||
48 | Conceptual and Statistical Knowledge | Understanding Bayesian parameter estimation | RIOTS lecture by E.-J. Wagenmakers about Bayesian estimation | Understand Bayes Theorem | Video | Lecture | 1 hour | https://www.youtube.com/watch?v=VCxXkMFhu5o&ab_channel=RIOTScienceClub | ||||||||||||||||||||||
49 | Conceptual and Statistical Knowledge | Preprints | As some point in the curricula, students are introduced to the system of peer-review. While peer-review is one of the hallmarks of science, it also slows down the process dissemination of knowledge. Preprints are versions of manuscripts that are not peer-reviewed (yet). They are often uploaded to preprint servers at the same time that they are submitted to a journal. | Introduce students to preprints | Powerpoint | Seminar | 10 minutes | https://docs.google.com/presentation/d/1KbysZCt7C-RDE7v8Sa_aIB14ZQGTwhe5k2TZrJOHCM0/edit#slide=id.g6031556d35_0_24 | ||||||||||||||||||||||
50 | Open data and materials | Open your course | This repository contains two 20 minute presentations (and corresponding 10 minute hands on). One shows how you can introduce and illustrate open data analyses in one of your lectures. The other shows how you can open the data analyses of your bachelor thesis. | How to open the data analyses of your bachelor thesis | Lecture | Lecture | 30 minutes | https://osf.io/rz24g/ | ||||||||||||||||||||||
51 | Academic Life and Culture | Open science can enhance research quality and collaboration Interview with Dr Ineke Wessel, associate professor in Experimental Psychopathology at the Faculty of Behavioural and Social Sciences | Interview with University of Groningen researcher. Comments on the process of "convert"ing to open science. She recommends "a lecture by Zoltan Dienes, based on Gelman and Loken’s famous paper that compared data analysis to a garden of forking paths," and "The Seven Deadly Sins of Psychology by Chris Chambers." She notes being "a direct witness of the Stapel affair." She describes why she is a supporter of pre-registrations. She notes obstacles such as P&T incentives. She describes teaching on the topic of open science. She shares an experience (at a prior SIPS!) where she felt vulnerable as someone new to R in a session. Overall, there are a lot of helpful personal stories that may be useful for persona-development or simply sharing when interacting in conversations on this topic. | Quickly and informally getting an introduction to basic open science terms | Blog Post (Website) | Discussion group session or portion of a broader lesson | 10 minutes | https://www.rug.nl/library/open-access/blog/open-science-can-enhance-research-quality-and-collaboration-10-02-2020?lang=en | ||||||||||||||||||||||
52 | Open data and materials | Creative Common Licences | Whenever you use data/code/materials of others, it is important to know what you are allowed to do with it. This is specified in a licence. A class of popular licences in science are the Creative Common license. You can discuss these six Creative Common license with students, to make sure they understand what they mean when they encounter them. Here’s a description of the licences, as provided by Creative Commons | Understanding creative commons licenses | Powerpoint | Verbal discussion | 10 minutes | https://docs.google.com/presentation/d/1KbysZCt7C-RDE7v8Sa_aIB14ZQGTwhe5k2TZrJOHCM0/edit#slide=id.g6031556d35_0_152 | ||||||||||||||||||||||
53 | Open data and materials | Open access | If you want to dive deeper into the topic of Open Access, you can introduce students to the various formats of Open Access publishing: e.g. full open access journals (gold), open access repositories (green), (see https://www.openaccess.nl/nl/wat-is-open-access). | Raise awareness of open access papers | Powerpoint | Lecture | 30 minutes | Part of: https://figshare.com/articles/presentation/Lecture_Open_Science_for_masterstudents/9963161 | ||||||||||||||||||||||
54 | Reproducibility and replicability knowledge | Is there a reproducibility crisis in science? - Matt Anticole | Very brief video introduction to reproducibility crisis in science. Talks about incentive structure in academia. Very broad, approachable and short. Could be useful for UG students. | Raise awareness of reproducibility issues in Science | Youtube video | Animation | 5 mins | https://www.youtube.com/watch?v=FpCrY7x5nEE | ||||||||||||||||||||||
55 | False Positive | False positives | Simulation of false positives (understanding the simulation in Simmons et al) | Raise awareness on false positives | SIPS Workshop (possible to use for taching?) | |||||||||||||||||||||||||
56 | Open data and materials | A FAIRy tale | Sharing data, materials or code is only useful is if it is shared such that it is Findable, Accessible, Interoperable and Reusable. These are the FAIR principles, which are further specified in The FAIR Guiding Principles for scientific data management and stewardship. A nice format to introduce the FAIR principles is by using the brochure ‘A FAIRy tale’, in which the FAIR principles are introduced in fairytale story. A possible teaching format is: Split the group in four, each group has one topic: Findable, Accessible, Interoperable, Reusable Students read the sections of the FAIRytale regarding their topic (30-45 minutes) Students present to fellow students (4x 5 minutes) Importantly, students should strive to implement the FAIR principles when sharing data, materials or code. As a supervisor, you can use the FAIR principles as a check-list, to evaluate whether students comply with the FAIR principles. | Learning about FAIR data | Brochure | Seminar | 30-120 minutes | https://vidensportal.deic.dk/sites/default/files/uploads/A%20FAIRy%20tale%20book%20digital_ny.pdf | ||||||||||||||||||||||
57 | Academic Life and Culture | 50 Ways to Jumpstart Academic Integrity Discussions in Your Class | This is a starting point for a discussion activity that can be targeted to specific integrity issue conversations in a class. Here is an activity plan currently being drafted based on this: https://docs.google.com/document/d/1e-VuvePAbU893QA9YGy0-oDywxBDPSU-ib8IkmmyPaQ/edit | Broadening students' horizons about academic integrity. Understanding the concept, and being able to have solid discussions about the topic. | Paper and lesson plan | Seminar | 1 hour | https://drive.google.com/file/d/1UqNNmW7Rb3Qr3RplxAzBDMS6gJhzm9pH/view?usp=sharing | Developed by Renée Gravois Lee, Sam Houston State University and Lisa M. Burns, Quinnipiac University | |||||||||||||||||||||
58 | Academic Life and Culture | Optimism about the future of Psychological Science | Paper about "psychology's renaissance": the period following the replication crisis where the focus was on methodological reflection and improvement | - Getting an understanding of the replication crisis and the period that followed afterwards. - Getting an overview of what has improved, and what can still be improved in psychological science | Paper | Discussion | 30 minutes | https://urisohn.com/sohn_files/wp/wordpress/wp-content/uploads/2019/01/Psychologys-Renaissance-2018.pdf | ||||||||||||||||||||||
59 | Pre-registration | Registered Reports work | Scheel et al (2021) showed that Registered Reports report support for their hypothesis about 40% of the time, compared to about 95% of standard articles | -We have the solutions to address many of these issues, and they work! | Paper | Discussion | 30 minutes | https://journals.sagepub.com/doi/pdf/10.1177/25152459211007467 | ||||||||||||||||||||||
60 | Academic Life and Culture | Feminist voices in psychology | Psychology's Feminist Voices (PFV) is a web resource which houses oral histories of leading feminist figures in psychology. This may be used in the context of open/reproducible science to demonstrate how history is often told through a particular lens, and there are other diverse voices that have been neglected from mainstram histories. This could spark a discussion about inclusion and diversity in open science | - To familiarise oneself with the diverse history of psychology and to understand how the (re)telling of history in psychology is often done through a biased lens | Web based resource | Discussion/independent research | https://feministvoices.com/ | |||||||||||||||||||||||
61 | Conceptual and statistical knowledge | Practice approaching research questions from different epistemologies | The goal of this activity is to teach students to consider different ways of knowing and how these shape our research questions and hypotheses. A detailed plan is linked in the Supporting Resources column | Students will be able to identify epistemological biases that have influenced other researchers, understand that that isn't a bad thing, and approach their own question and hypothesis development situated within an epistemology | Lesson plan | Lecture + reading on epistemology, in-class activity | Flexible, 1 hr | https://docs.google.com/document/d/1naQ5QVR0KrKAVzXrGFXqLDFrj4Qp4D3vwhW9YqQbYi8/edit?usp=sharing | PoPS 16(4): https://journals.sagepub.com/toc/pps/16/4 | |||||||||||||||||||||
62 | Open data and materials; Academic life and culture | How to be FAIR with your data: A teaching and training handbook for higher education institutions | A handbook which provides guidance, competence profiles and lesson plans on how to make research data FAIR. The main target groups are teachers and trainers at HE institutions. | e.g. general principles and concepts of data management, data management planning, metadata, persistent identifiers, backup, data publishing - for more details consult the book, chapter 3.3. | Handbook | Engelhardt, C. et al. (2022). How to be FAIR with your data. Universitätsverlag Göttingen. https://doi.org/10.17875 /gup2022-1915 | ||||||||||||||||||||||||
63 | ||||||||||||||||||||||||||||||
64 | ||||||||||||||||||||||||||||||
65 | ||||||||||||||||||||||||||||||
66 | ||||||||||||||||||||||||||||||
67 | ||||||||||||||||||||||||||||||
68 | ||||||||||||||||||||||||||||||
69 | ||||||||||||||||||||||||||||||
70 | ||||||||||||||||||||||||||||||
71 | ||||||||||||||||||||||||||||||
72 | ||||||||||||||||||||||||||||||
73 | ||||||||||||||||||||||||||||||
74 | ||||||||||||||||||||||||||||||
75 | ||||||||||||||||||||||||||||||
76 | ||||||||||||||||||||||||||||||
77 | ||||||||||||||||||||||||||||||
78 | ||||||||||||||||||||||||||||||
79 | ||||||||||||||||||||||||||||||
80 | ||||||||||||||||||||||||||||||
81 | ||||||||||||||||||||||||||||||
82 | ||||||||||||||||||||||||||||||
83 | ||||||||||||||||||||||||||||||
84 | ||||||||||||||||||||||||||||||
85 | ||||||||||||||||||||||||||||||
86 | ||||||||||||||||||||||||||||||
87 | ||||||||||||||||||||||||||||||
88 | ||||||||||||||||||||||||||||||
89 | ||||||||||||||||||||||||||||||
90 | ||||||||||||||||||||||||||||||
91 | ||||||||||||||||||||||||||||||
92 | ||||||||||||||||||||||||||||||
93 | ||||||||||||||||||||||||||||||
94 | ||||||||||||||||||||||||||||||
95 | ||||||||||||||||||||||||||||||
96 | ||||||||||||||||||||||||||||||
97 | ||||||||||||||||||||||||||||||
98 | ||||||||||||||||||||||||||||||
99 | ||||||||||||||||||||||||||||||
100 |