ABCDEFGHIJKLMNOPQRSTUV
1
Be Unconference on Open Scholarship Practices in Education Research
2
ABC
3
2/24/2022 (all times listed reflect the Eastern Time Zone of the United States)
4
19:00 AMA1: Welcome and Orientation with Brian Nosek - Logistics Overview and Introductions
5
10:00 AMOpen Time
6
211:00 AMA2: Deep Dive on Open Practices: Understanding Data Sharing with Sara Hart - As sharing data openly becomes more and more the norm, and not just because of mandates for federal funding, more researchers may become more interested in sharing data. Benefits of data sharing for educational research include increased collaboration, acceleration of knowledge through novel and creative research questions, and an increase in equitable opportunities for early career researchers and faculty at under-resourced institutions. In this session, I will cover the benefits of data sharing as well as the “how to” of how to prepare data for sharing. Participants will leave with information about data sharing and resources to support their own data sharing.B2: Promoting Open Science, Open Access, and Replication through Instructional Materials with Avery Closser & Stacy Shaw (unconference session) - One avenue to promote open science through teaching is by carefully selecting instructional materials that 1) are available through open-access sources, 2) provide current depictions of views in the field, and 3) emphasize the importance of replication in research. In this session, we will discuss pathways and compile resources to promote open science through careful selection of instructional materials. This two-hour session will include discussions among participants, brief presentations and demos, and crowdsourcing open science materials and resources for attendees to use in the future. We hope that attendees walk away with more information about the value in using open instructional materials as an avenue for promoting open science as well as resources to reconstruct their own syllabi and course materials in the future.C2: This is not a moment; it’s the movement: Open science revolution in special education research with William J. Therrien, Bryan G. Cook, Sarah Emily Wilson, Jesse Fleming & Alan McLucas (unconference session) - In this unconference session, we will work with conference attendees to develop a comprehensive plan to increase the adoption of open science practices across the special research enterprise, from individual study implementation to the creation of open access journals, use of crowdsourcing and changing our community norms regarding what it means to be a successful scholar in our field.
7
312:00 PMA3: Deep Dive on Open Practices: Understanding Replication in Education Research with Matt Makel - In this deep dive session, we will introduce the purpose of replication, different conceptions of replication, and some models for implementation in education. Relevant terms, methods, publication possibilities, and existing funding mechanisms will be reviewed. Frequently asked questions and potential answers will be shared. Attendees will be connected with existing resources, tools, and examples. Finally, the session will leave time at the end for questions and discussion.
8
1:00 PMOpen Time
9
42:00 PMA4: Deep Dive on Open Practices: Understanding Preprints and Open Access with Bryan Cook and Stacy Shaw - In this deep dive session, we will provide an introduction to preprints with a focus on their contributions and limitations in the context of current models of scholarly publishing. We will discuss various "levels" of open access publishing (Gold, Bronze, and Green), and how the use of preprints can supplement some of the limitations of these common open access publishing models. Research on the use of preprints will also be highlighted, before discussing how preprints/postprints may be leveraged to promote open access.B4: Qualitative Differences: Continuing the Conversation on Open Science Practices for Qualitative Research with Rachel Renbarger, Sondra Stegenga, Crystal Steltenpohl & Thomas Lösch (unconference session) - Education research, similar to other fields, has seen an increase in the attention paid to open science practices thanks to issues with replicability and quality of the research. However, most of the attention has been focused on open science practices for quantitative research, such as open analytical syntax and a priori power analyses, that often do not apply to qualitative research or look very different in qualitative contexts. Our objectives for this session are to a) bring researchers familiar with qualitative research, education research, and open science practices together; b) discuss new trends regarding open science in qualitative research; and c) critically review what has been done to move this work forward. At the end of the session, we will conclude by asking participants what they hope would be next in enabling qualitative researchers to engage in open science work.
C4: Creating a Get Started Guide: How to Design an Open Education Science Study with Jessica Logan & Sara Hart (hackathon) - With this hackathon session, we propose to create a “get started guide” specific to education research. The finished product will walk education researchers through the process of setting up a project with Open Science practices at the center. Join us to help collate the get started guide under headings of Open Data, Open Materials, Open Code/Analysis, Preregistration, Registered Reports, and Open Access. Within each heading section, the participants will create a bullet list of what to do, including links to resources and good websites to use, at each stage of the education research project. Specifically, planning, collecting data, post data collection, post data analysis and post publication. After the session, the hackathon leads will finalize and post the co-developed “get started guide”.
10
53:00 PMA5: Lightning Talks 1) Open Science at JREE with Sean Grant 2) Current Issues in Education, an open-access journal produced by doctoral students at Arizona State University with Ivonne Lujano Vilchis, 3) A Community-Sourced Glossary of Open Scholarship Terms with Flávio Azevedo, 4) Data Management and Open Science: Practices of Education Researchers with A. Busra Ceviren & Jessica Logan, 5) Open qualitative research with Markus Konkol, 6) Raising awareness of Neurodiversity in Academia: Navigating Open scholarship for neurodiverse researchers with Mahmoud Elsherif, Amélie Gourdon-Kanhukamwe & Samantha Tyler
11
64:00 PMA6: Day 1 Reflection and Planning
12
ABC
13
2/25/2022 (all times listed reflect the Eastern Time Zone of the United States)
14
15
79:30 AMA7: What's Ahead
16
810:00 AMA8: Deep Dive on Open Practices: Understanding Preregistration with Scott Peters and Karen Rambo-Hernandez - In this deep dive session, we introduce the basics of pre-registration: a method for creating a permanent record of a research plan prior to conducting data collection and/or data analysis. We discuss the conceptual similarities and practical differences between pre-registration and registered reports and traditional approaches to educational research. We provide some practical advice from our own experiences using this practice in our own research and resources available for researchers interested in pre-registering their work. Finally, we end with questions and discussion about adopting pre-registration practices and unique considerations for implementing pre-registration in education research. B8: Discovering open access journals for educational research using the DOAJ with Ivonne Lujano (workshop) - This workshop will present an overview of the Directory of Open Access Journals (DOAJ), highlighting its relevance in the international Open Access landscape. We will discuss the Principles of Transparency and Best Practice in Scholarly Publishing with an emphasis on its applicability in education research journals. Finally, participants will use the DOAJ in a hands-on activity to search education journals of their interest, compare and assess them as potential venues to publish their research, and/or use them as sources of information.C8: Towards a culture of open scholarship: The role of pedagogical communities with Flávio
Azevedo, Meng Liu & Charlotte R. Pennington - We argue that there is one important component of research integrity that is often absent from discussion: the pedagogical consequences of how we teach, mentor, and supervise students through open scholarship. We justify the need to integrate open scholarship principles into research training within higher education and argue that pedagogical communities play a key role in fostering an inclusive culture of open scholarship. We illustrate these benefits by presenting the Framework for Open and Reproducible Research Training (FORRT), an international grassroots community whose goal is to provide support, resources, visibility, and advocacy for the adoption of principled, open teaching and mentoring practices, whilst generating conversations about the ethics and social impact of higher-education pedagogy.
17
911:00 AMA9: - Lightning Talks 1) Linking Open Scholarship to Reducing Frustration and Increasing Motivation with Angela Ford 2) Transparency- and reproducibility-related research practices in special education: A replication of Hardwicke et al. (2020) with Bryan Cook & Jesse Fleming, 3) Open Science and Doctoral Students: Findings from the Open Scholarship Survey with Jesse Fleming & Sarah Emily Wilson, 4) Looking into the Black Box: Conceptualizing Open Science Practices in Education Policy with Jessica Goldstein Holmes, Dr. Jonathan Wai & Emily Coady, 5) Reengineering the Scientific Method for the 21st Century with Dave Lucas, 6) Building Community through pedagogical reform with Flávio Azevedo.B9: Engaging the Curriculum to Develop an Inclusive and Equitable Model for OER Creation with Becca Neel & Kelly Sparks (unconference session) - Two of the most common obstacles to OER use among faculty are the lack time and incentives to adapt or create OER. In this session, we will discuss methods for transforming these barriers into pedagogical opportunities by using internship structures and course objectives to include teacher education students in the review and creation of OER. These pedagogical partnerships expose current and future educators to OER as mechanisms for developing culturally-relevant and community-based educational content that traditional textbooks cannot offer. We will provide examples from the literature and work at our current institution to frame this discussion, but hope to engage attendees in a larger conversation regarding the merits and logistics of promoting and building collaborative OER development into education (and other academic) programs.C9: Open Science Approaches to Secondary Data Analyses with Graham Rifenbark, Allison Lombardi & Tyler Hicks (unconference session) - In this session, we will present our process to generate a successful pre-registered secondary data analysis study using the National Longitudinal Transition Study 2012 data. Specifically, we will highlight how we (a) satisfied the power analysis requirement for the Stage-1 report without data snooping, (b) leveraged Bayesian methods to provide deeper interpretation of null findings, and (c) discuss key elements such as accounting for the complex sampling design and how software packages may differ in this respect. We will share a preliminary checklist of best practices to designing and submitting a Stage-1 registered report that aligns with Open Science guidelines specific to secondary data analyses.
18
12:00 PMOpen Time
19
101:00 PMA10: Deep Dive on Open Practices: Understanding Registered Reports in Education Research with Amanda Montoya and Betsy McCoach - Registered reports are a new publication mechanism where peer review and the decision to publish the results of a study occur prior to data collection and/or analysis. Registered reports share many characteristics with preregistration, but are distinct by involving the journal prior to completing the study. Journals in the field of education are increasingly offering opportunities to publish registered reports. Registered reports offer a variety of benefits to both the researcher and to the research field. In this workshop, we will discuss the basics of registered reports, benefits and limitations of registered reports, and which journals in education accept registered reports. We provide some practical advice on deciding which projects are appropriate for registered reports, implementing registered reports, and time management throughout the process. We discuss how special cases can be implemented as registered reports, such as secondary data analysis, replications, meta-analyses, and longitudinal studies. B10: Be afraid, be very afraid: Data Management Horror Stories and Lessons Learned with With Crystal Lewis, Jessica Logan, and Busra Ceviren (unconference session) - Managing research data is a key part of any research project, but is prone to sometimes catastrophic errors. Come to this session ready to share your most dreadful (or any!) data management mishaps and commiserate with others about these shared experiences. We will also talk about how these mistakes have shaped our data management practices, and end by brainstorming ideas for how to capture this knowledge to share with others. C10: Risk of Bias and Open Practices in Systematic Reviews with Thomas Nordström, Rickard Carlsson, Lucija Batinović, André Kalmendal, Ida Henriksson (unconference session) - As part of a larger meta-review project that aims to assess the quality of systematic reviews of educational intervention, this study provides a snapshot of reviews published between 2019 and 2021. Main goal was to assess the current state of literature and have the best studies published as CAMAs. We looked for reviews that investigated educational interventions’ effectiveness for the k-12 population using experimental designs (RCT, QED, SCD). We searched for systematic reviews in the ERIC database and four journals which publish educational reviews. Studies that were included in the full-text screening were assessed using the ROBIS (risk of bias in systematic reviews) tool, first by assessing if the PICOS fit ours, then moved on to first stage ROBIS screening, which was conducted for all articles included in the full-text reading phase. Preliminary results of the first stage ROBIS screening indicate the lack of preregistration and data sharing practice, no standardized approach in conducting searches and reporting results, and often absent quality check of studies included in the reviews.
20
2:00 PMOpen Time
21
113:00 PMA11: On the fly B11: On the fly C11: On the fly
22
124:00 PMA12: Closing Plenary
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100