| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | AB | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Focus | Type of publication | Year | Publisher | Citation | Abstract / Description | Link | |||||||||||||||||||||
2 | Comprehensive | Report | 2002 | National Academy Press | Towne, Lisa, and Richard J. Shavelson. Scientific research in education. National Academy Press Publications Sales Office., 2002. | Researchers, historians, and philosophers of science have debated the nature of scientific research in education for more than 100 years. Recent enthusiasm for "evidence-based" policy and practice in education—now codified in the federal law that authorizes the bulk of elementary and secondary education programs—have brought a new sense of urgency to understanding the ways in which the basic tenets of science manifest in the study of teaching, learning, and schooling. Scientific Research in Education describes the similarities and differences between scientific inquiry in education and scientific inquiry in other fields and disciplines and provides a number of examples to illustrate these ideas. Its main argument is that all scientific endeavors share a common set of principles, and that each field—including education research—develops a specialization that accounts for the particulars of what is being studied. The book also provides suggestions for how the federal government can best support high-quality scientific research in education. | https://nap.nationalacademies.org/catalog/10236/scientific-research-in-education | |||||||||||||||||||||
3 | Research design,Statistical approaches | Article | 2005 | Journal of Chemical Education | Lewis, S. E., & Lewis, J. E. (2005). The same or not the same: Equivalence as an issue in educational research. Journal of Chemical Education, 82(9), 1408. | In educational research, particularly in the sciences, a common research design calls for the establishment of a control and experimental group to determine the effectiveness of an intervention. As part of this design, it is often desirable to illustrate that the two groups were equivalent at the start of the intervention, based on measures such as standardized cognitive tests or student grades in prior courses. In this article we use SAT and ACT scores to illustrate a more robust way of testing equivalence. The method incorporates two one-sided t tests evaluating two null hypotheses, providing a stronger claim for equivalence than the standard method, which often does not address the possible problem of low statistical power. The two null hypotheses are based on the construction of an equivalence interval particular to the data, so the article also provides a rationale for and illustration of a procedure for constructing equivalence intervals. Our consideration of equivalence using this method also underscores the need to include sample sizes, standard deviations, and group means in published quantitative studies. | https://pubs.acs.org/doi/abs/10.1021/ed082p1408 | |||||||||||||||||||||
4 | Theoretical framework | Book | 2007 | Prentice Hall | Bodner, G. M., & Orgill, M. (2007). Theoretical frameworks for research in chemistry/science education. Prentice Hall | Part of the Prentice Hall Series in Educational Innovation, this concise new volume is the first book devoted entirely to describing and critiquing the various theoretical frameworks used in chemistry education/science education research with explicit examples of related studies. The book provides a broad spectrum of theoretical perspectives upon which readers can base educational research. A useful guide for practicing chemists, chemistry instructors, and chemistry educators for learning how to do basic educational research within the context of their own instructional laboratories and classrooms. | https://static1.squarespace.com/static/56816ce84bf118911f0712a8/t/5a53a8d5f9619a6ffcb5112e/1515432170768/1+TheorFrame+TOC.pdf | |||||||||||||||||||||
5 | Comprehensive | Book | 2008 | American Chemical Society | Bunce, D. M., & Cole, R. S. (Eds.). (2008). Nuts and bolts of chemical education research. American Chemical Society. | Nuts and Bolts of Chemical Education Research is an informative and engaging book that would be useful for the chemist who is writing the educational outreach or evaluation component of a grant or planning his own chemical education research project. This book brings to the surface the key elements that are common to both. These key elements include establishing clear goals and research questions for your efforts: placing your outreach or research on a firm theoretical foundation so that the results of your work expand the current state of knowledge; developing an outreach or research design that address the goals and questions asked; locating, developing and testing the validity-reliability of the tools used in the study; selecting appropriate data analyses from quantitative, qualitative or mixed design disciplines to address the questions asked; writing conclusions based upon the data presented; and describing the implications of the outreach or research effort for chemistry practitioners. This book will address these key issues from a pragmatic point of view in an effort to assist those who are engaged or considering becoming engaged in this type of scholarly activity. | https://pubs.acs.org/isbn/9780841269583 | |||||||||||||||||||||
6 | Instrument development and use | Book chapter | 2011 | American Chemical Society | Barbera, J., & VandenPlas, J. R. (2011). All assessment materials are not created equal: The myths about instrument development, validity, and reliability. In Investigating classroom myths through research on teaching and learning (pp. 177-193). American Chemical Society. | Educators and educational researchers strive to evaluate the impact of teaching on learning. To perform these assessments, we rely on instruments (e.g., surveys, questionnaires, tests, inventories) constructed of various items. But how do we know that the instrument itself produces valid and reliable results? Incorrect student responses on any assessment can stem from a number of sources. To truly uncover what information students understand, educators and researchers need to not only know whether students answers are correct or incorrect, they must uncover why they are correct or incorrect. Carefully constructed and validated assessment instruments can provide a useful tool for this purpose. This chapter will explore several myths regarding the development, validation, and reliability of assessment instruments. The goal is to both inform instrument users on what to look for when seeking out instruments as well as to inform instrument developers on various validation techniques. | https://pubs.acs.org/doi/pdf/10.1021/bk-2011-1074.ch011 | |||||||||||||||||||||
7 | Data trustworthiness | Article | 2013 | Journal of Chemical Education | Arjoon, J. A., Xu, X., & Lewis, J. E. (2013). Understanding the state of the art for measurement in chemistry education research: Examining the psychometric evidence. Journal of Chemical Education, 90(5), 536-545. | Many of the instruments developed for research use by the chemistry education community are relatively new. Because psychometric evidence dictates the validity of interpretations made from test scores, gathering and reporting validity and reliability evidence is of utmost importance. Therefore, the purpose of this study was to investigate what “counts” as psychometric evidence within this community. Using a methodology based on concepts described and operationalized in the Standards for Educational and Psychological Testing, instruments first published in the Journal between 2002 and 2011, and follow-up publications reporting the use of these instruments, were examined. Specifically, we investigated the availability of evidence based on test content, response processes, internal structure, relations to other variables, temporal stability, and internal consistency. Findings suggest that our peer review and reporting practices value some types of evidence while neglecting others. Results of this study serve as an indication of the need for the chemistry education research community to view gathering psychometric evidence as a collective activity and to report such evidence at the level of detail appropriate to inform future research. | https://pubs.acs.org/doi/abs/10.1021/ed3002013 | |||||||||||||||||||||
8 | Statistical approaches | Article | 2013 | CBE—Life Sciences Education | Maher, J. M., Markey, J. C., & Ebert-May, D. (2013). The other half of the story: effect size analysis in quantitative research. CBE—Life Sciences Education, 12(3), 345-351. | Statistical significance testing is the cornerstone of quantitative research, but studies that fail to report measures of effect size are potentially missing a robust part of the analysis. We provide a rationale for why effect size measures should be included in quantitative discipline-based education research. Examples from both biological and educational research demonstrate the utility of effect size for evaluating practical significance. We also provide details about some effect size indices that are paired with common statistical significance tests used in educational research and offer general suggestions for interpreting effect size measures. Finally, we discuss some inherent limitations of effect size measures and provide further recommendations about reporting confidence intervals. | https://www.lifescied.org/doi/full/10.1187/cbe.13-04-0082 | |||||||||||||||||||||
9 | Instrument development and use,Statistical approaches | Article | 2013 | CBE—Life Sciences Education | Lovelace, M., & Brickman, P. (2013). Best practices for measuring students’ attitudes toward learning science. CBE—Life Sciences Education, 12(4), 606-617. | Science educators often characterize the degree to which tests measure different facets of college students’ learning, such as knowing, applying, and problem solving. A casual survey of scholarship of teaching and learning research studies reveals that many educators also measure how students’ attitudes influence their learning. Students’ science attitudes refer to their positive or negative feelings and predispositions to learn science. Science educators use attitude measures, in conjunction with learning measures, to inform the conclusions they draw about the efficacy of their instructional interventions. The measurement of students’ attitudes poses similar but distinct challenges as compared with measurement of learning, such as determining validity and reliability of instruments and selecting appropriate methods for conducting statistical analyses. In this review, we will describe techniques commonly used to quantify students’ attitudes toward science. We will also discuss best practices for the analysis and interpretation of attitude data. | https://www.lifescied.org/doi/10.1187/cbe.12-11-0197 | |||||||||||||||||||||
10 | Comprehensive | Book | 2014 | American Chemical Society | Bunce, D. M., & Cole, R. S. (Eds.). (2014). Tools of chemistry education research. American Chemical Society. | Tools of Chemistry Education Research meets the current need for information on more in-depth resources for those interested in doing chemistry education research. Renowned chemists Diane M. Bunce and Renée S. Cole present this volume as a continuation of the dialogue started in their previous work, Nuts and Bolts of Chemical Education Research. With both volumes, new and experienced researchers will now have a place to start as they consider new research projects in chemistry education. Tools of Chemistry Education Research brings together a group of talented researchers to share their insights and expertise with the broader community. The volume features the contributions of both early career and more established chemistry education researchers, so as to promote the growth and expansion of chemistry education. Drawing on the expertise and insights of junior faculty and more experienced researchers, each author offers unique insights that promise to benefit other practitioners in chemistry education research. | https://pubs.acs.org/doi/book/10.1021/bk-2014-1166 | |||||||||||||||||||||
11 | Research design,Statistical approaches | Article | 2014 | CBE—Life Sciences Education | Theobald, R., & Freeman, S. (2014). Is it the intervention or the students? Using linear regression to control for student characteristics in undergraduate STEM education research. CBE—Life Sciences Education, 13(1), 41-48. | Although researchers in undergraduate science, technology, engineering, and mathematics education are currently using several methods to analyze learning gains from pre- and posttest data, the most commonly used approaches have significant shortcomings. Chief among these is the inability to distinguish whether differences in learning gains are due to the effect of an instructional intervention or to differences in student characteristics when students cannot be assigned to control and treatment groups at random. Using pre- and posttest scores from an introductory biology course, we illustrate how the methods currently in wide use can lead to erroneous conclusions, and how multiple linear regression offers an effective framework for distinguishing the impact of an instructional intervention from the impact of student characteristics on test score gains. In general, we recommend that researchers always use student-level regression models that control for possible differences in student ability and preparation to estimate the effect of any nonrandomized instructional intervention on student performance. | https://doi.org/10.1187/cbe-13-07-0136 | |||||||||||||||||||||
12 | Statistical approaches | Article | 2014 | CBE—Life Sciences Education | Grunspan, D. Z., Wiggins, B. L., & Goodreau, S. M. (2014). Understanding classrooms through social network analysis: A primer for social network analysis in education research. CBE—Life Sciences Education, 13(2), 167-178. | Social interactions between students are a major and underexplored part of undergraduate education. Understanding how learning relationships form in undergraduate classrooms, as well as the impacts these relationships have on learning outcomes, can inform educators in unique ways and improve educational reform. Social network analysis (SNA) provides the necessary tool kit for investigating questions involving relational data. We introduce basic concepts in SNA, along with methods for data collection, data processing, and data analysis, using a previously collected example study on an undergraduate biology classroom as a tutorial. We conduct descriptive analyses of the structure of the network of costudying relationships. We explore generative processes that create observed study networks between students and also test for an association between network position and success on exams. We also cover practical issues, such as the unique aspects of human subjects review for network studies. Our aims are to convince readers that using SNA in classroom environments allows rich and informative analyses to take place and to provide some initial tools for doing so, in the process inspiring future educational studies incorporating relational data. | https://www.lifescied.org/doi/10.1187/cbe.13-08-0162 | |||||||||||||||||||||
13 | Theoretical framework | Book | 2016 | Sage Publications | Ravitch, S. M., & Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research. Sage Publications. | Designed for novice as well as more experienced researchers, Reason & Rigor presents conceptual frameworks as a mechanism for aligning literature review, research design, and methodology. The book explores the conceptual framework—defined as both a process and a product—that helps to direct and ground researchers as they work through common research challenges. Focusing on published studies on a range of topics and employing both quantitative and qualitative methods, the updated Second Edition features two new chapters and clearly communicates the processes of developing and defining conceptual frameworks. | https://us.sagepub.com/en-us/nam/reason-rigor/book241777 | |||||||||||||||||||||
14 | Data trustworthiness | Article | 2016 | CBE—Life Sciences Education | Reeves, T. D., & Marbach-Ad, G. (2016). Contemporary test validity in theory and practice: A primer for discipline-based education researchers. CBE—Life Sciences Education, 15(1), rm1. | Most discipline-based education researchers (DBERs) were formally trained in the methods of scientific disciplines such as biology, chemistry, and physics, rather than social science disciplines such as psychology and education. As a result, DBERs may have never taken specific courses in the social science research methodology—either quantitative or qualitative—on which their scholarship often relies so heavily. One particular aspect of (quantitative) social science research that differs markedly from disciplines such as biology and chemistry is the instrumentation used to quantify phenomena. In response, this Research Methods essay offers a contemporary social science perspective on test validity and the validation process. The instructional piece explores the concepts of test validity, the validation process, validity evidence, and key threats to validity. The essay also includes an in-depth example of a validity argument and validation approach for a test of student argument analysis. In addition to DBERs, this essay should benefit practitioners (e.g., lab directors, faculty members) in the development, evaluation, and/or selection of instruments for their work assessing students or evaluating pedagogical innovations. | https://www.lifescied.org/doi/10.1187/cbe.15-08-0183 | |||||||||||||||||||||
15 | Instrument development and use,Data trustworthiness | Article | 2016 | CBE—Life Sciences Education | Bass, K. M., Drits-Esser, D., & Stark, L. A. (2016). A primer for developing measures of science content knowledge for small-scale research and instructional use. CBE—Life Sciences Education, 15(2), rm2. | The credibility of conclusions made about the effectiveness of educational interventions depends greatly on the quality of the assessments used to measure learning gains. This essay, intended for faculty involved in small-scale projects, courses, or educational research, provides a step-by-step guide to the process of developing, scoring, and validating high-quality content knowledge assessments. We illustrate our discussion with examples from our assessments of high school students’ understanding of concepts in cell biology and epigenetics. Throughout, we emphasize the iterative nature of the development process, the importance of creating instruments aligned to the learning goals of an intervention or curricula, and the importance of collaborating with other content and measurement specialists along the way. | https://www.lifescied.org/doi/10.1187/cbe.15-07-0142 | |||||||||||||||||||||
16 | Instrument development and use,Statistical approaches,Diversity, equity, and inclusion | Article | 2017 | CBE—Life Sciences Education | Martinková, P., Drabinová, A., Liaw, Y. L., Sanders, E. A., McFarland, J. L., & Price, R. M. (2017). Checking equity: Why differential item functioning analysis should be a routine part of developing conceptual assessments. CBE—Life Sciences Education, 16(2), rm2. | We provide a tutorial on differential item functioning (DIF) analysis, an analytic method useful for identifying potentially biased items in assessments. After explaining a number of methodological approaches, we test for gender bias in two scenarios that demonstrate why DIF analysis is crucial for developing assessments, particularly because simply comparing two groups’ total scores can lead to incorrect conclusions about test fairness. First, a significant difference between groups on total scores can exist even when items are not biased, as we illustrate with data collected during the validation of the Homeostasis Concept Inventory. Second, item bias can exist even when the two groups have exactly the same distribution of total scores, as we illustrate with a simulated data set. We also present a brief overview of how DIF analysis has been used in the biology education literature to illustrate the way DIF items need to be reevaluated by content experts to determine whether they should be revised or removed from the assessment. Finally, we conclude by arguing that DIF analysis should be used routinely to evaluate items in developing conceptual assessments. These steps will ensure more equitable—and therefore more valid—scores from conceptual assessments. | https://www.lifescied.org/doi/10.1187/cbe.16-10-0307 | |||||||||||||||||||||
17 | Research design,Statistical approaches | Article | 2018 | CBE—Life Sciences Education | Theobald, E. (2018). Students are rarely independent: When, why, and how to use random effects in discipline-based education research. CBE—Life Sciences Education, 17(3), rm2. | Discipline-based education researchers have a natural laboratory—classrooms, programs, colleges, and universities. Studies that administer treatments to multiple sections, in multiple years, or at multiple institutions are particularly compelling for two reasons: first, the sample sizes increase, and second, the implementation of the treatments can be intentionally designed and carefully monitored, potentially negating the need for additional control variables. However, when studies are implemented in this way, the observations on students are not completely independent; rather, students are clustered in sections, terms, years, or other factors. Here, I demonstrate why this clustering can be problematic in regression analysis. Fortunately, nonindependence of sampling can often be accounted for with random effects in multilevel regression models. Using several examples, including an extended example with R code, this paper illustrates why and how to implement random effects in multilevel modeling. It also provides resources to promote implementation of analyses that control for the nonindependence inherent in many quasi-random sampling designs. | https://www.lifescied.org/doi/10.1187/cbe.17-12-0280 | |||||||||||||||||||||
18 | Data trustworthiness,Instrument development and use | Article | 2018 | Chemistry Education Research and Practice | Komperda, R., Hosbein, K. N., & Barbera, J. (2018). Evaluation of the influence of wording changes and course type on motivation instrument functioning in chemistry. Chemistry Education Research and Practice, 19(1), 184-198. | Increased understanding of the importance of the affective domain in chemistry education research has led to the development and adaptation of instruments to measure chemistry-specific affective traits, including motivation. Many of these instruments are adapted from other fields by using the word ‘chemistry’ in place of other disciplines or more general ‘science’ wording. Psychometric evidence is then provided for the functioning of the new adapted instrument. When an instrument is adapted from general language to specific (e.g. replacing ‘science’ with ‘chemistry’), an opportunity exists to compare the functioning of the original instrument in the same context as the adapted instrument. This information is important for understanding which types of modifications may have small or large impacts on instrument functioning and in which contexts these modifications may have more or less influence. In this study, data were collected from the online administration of scales from two science motivation instruments in chemistry courses for science majors and for non-science majors. Participants in each course were randomly assigned to view either the science version or chemistry version of the items. Response patterns indicated that students respond differently to different wordings of the items, with generally more favorable response to the science wording of items. Confirmatory factor analysis was used to investigate the internal structure of each instrument, however acceptable data-model fit was not obtained under any administration conditions. Additionally, no discernable pattern could be detected regarding the conditions showing better data-model fit. These results suggest that even seemingly small changes to item wording and administration context can affect instrument functioning, especially if the change in wording affects the construct measured by the instrument. This research further supports the need to provide psychometric evidence of instrument functioning each time an instrument is used and before any comparisons are made of responses to different versions of the instrument. | https://pubs.rsc.org/en/content/articlelanding/2018/RP/C7RP00181A | |||||||||||||||||||||
19 | Statistical approaches,Data trustworthiness | Article | 2018 | Journal of Chemical Education | Komperda, R., Pentecost, T. C., & Barbera, J. (2018). Moving beyond alpha: A primer on alternative sources of single-administration reliability evidence for quantitative chemistry education research. Journal of Chemical Education, 95(9), 1477-1491. | This methodological paper examines current conceptions of reliability in chemistry education research (CER) and provides recommendations for moving beyond the current reliance on reporting coefficient alpha (α) as reliability evidence without regard to its appropriateness for the research context. To help foster a better understanding of reliability and the assumptions that underlie reliability coefficients, reliability is first described from a conceptual framework, drawing on examples from measurement in the physical sciences; then classical test theory is used to frame a discussion of how reliability evidence for psychometric measurements is commonly examined in CER, primarily in the form of single-administration reliability coefficients. Following this more conceptual introduction to reliability, the paper transitions to a more mathematical treatment of reliability using a factor analysis framework with emphasis on the assumptions underlying coefficient alpha and other single-administration reliability coefficients, such as omega (ω) and coefficient H, which are recommended as successors to alpha in CER due to their more broad applicability to a variety of factor models. The factor analysis-based reliability discussion is accompanied by R code that demonstrates the mathematical relations underlying single-administration reliability coefficients and provides interested readers the opportunity to compute coefficients beyond alpha for their own data. | https://pubs.acs.org/doi/abs/10.1021/acs.jchemed.8b00220 | |||||||||||||||||||||
20 | Research design,Statistical approaches | Article | 2019 | Journal of Chemical Education | Mack, M. R., Hensen, C., & Barbera, J. (2019). Metrics and methods used to compare student performance data in chemistry education research articles. Journal of Chemical Education, 96(3), 401-413. | Quasi-experiments are common in studies that estimate the effect of instructional interventions on student performance outcomes. In this type of research, the nature of the experimental design, the choice in assessment, the selection of comparison groups, and the statistical methods used to analyze the comparison data dictate the validity of causal inferences. Therefore, gathering and reporting validity evidence in causal studies is of utmost importance, especially when conclusions have real policy implications for students and faculty, among other stakeholders. This review examines 24 articles that reported quantitative investigations of the effect of instructional interventions on performance-based outcomes conducted within undergraduate chemistry courses. Specifically, we examined four aspects of conducting such evaluations, including: (1) the type of quasi-experimental design used to study the relationship between interventions, students, outcomes, and settings; (2) the metrics used to measure performance outcomes; (3) the type of groups used to contrast outcomes across experimental conditions; and (4) the statistical methods used to analyze the comparison data. Through the examination of these four aspects of causal studies, together with a validity typology for quasi-experimental designs, we catalogued the metrics and methods used to compare student performance outcomes across varied instructional contexts. Recommendations for researchers and practitioners planning quasi-experimental investigations and for interpreting results from causal studies in chemistry education are provided. | https://pubs.acs.org/doi/abs/10.1021/acs.jchemed.8b00713 | |||||||||||||||||||||
21 | Data trustworthiness,Instrument development and use,Statistical approaches | Article | 2019 | CBE—Life Sciences Education | Knekta, E., Runyon, C., & Eddy, S. (2019). One size doesn’t fit all: Using factor analysis to gather validity evidence when using surveys in your research. CBE—Life Sciences Education, 18(1), rm1. | Across all sciences, the quality of measurements is important. Survey measurements are only appropriate for use when researchers have validity evidence within their particular context. Yet, this step is frequently skipped or is not reported in educational research. This article briefly reviews the aspects of validity that researchers should consider when using surveys. It then focuses on factor analysis, a statistical method that can be used to collect an important type of validity evidence. Factor analysis helps researchers explore or confirm the relationships between survey items and identify the total number of dimensions represented on the survey. The essential steps to conduct and interpret a factor analysis are described. This use of factor analysis is illustrated throughout by a validation of Diekman and colleagues’ goal endorsement instrument for use with first-year undergraduate science, technology, engineering, and mathematics students. We provide example data, annotated code, and output for analyses in R, an open-source programming language and software environment for statistical computing. For education researchers using surveys, understanding the theoretical and statistical underpinnings of survey validity is fundamental for implementing rigorous education research. | https://www.lifescied.org/doi/10.1187/cbe.18-04-0064 | |||||||||||||||||||||
22 | Theoretical framework | Article | 2020 | Academic Medicine | Varpio, L., Paradis, E., Uijtdehaage, S., & Young, M. (2020). The distinctions between theory, theoretical framework, and conceptual framework. Academic Medicine, 95(7), 989-994. | Health professions education (HPE) researchers are regularly asked to articulate their use of theory, theoretical frameworks, and conceptual frameworks in their research. However, all too often, these words are used interchangeably or without a clear understanding of the differences between these concepts. Further problematizing this situation is the fact that theory, theoretical framework, and conceptual framework are terms that are used in different ways in different research approaches. In this article, the authors set out to clarify the meaning of these terms and to describe how they are used in 2 approaches to research commonly used in HPE: the objectivist deductive approach (from theory to data) and the subjectivist inductive approach (from data to theory). In addition to this, given that within subjectivist inductive research theory, theoretical framework, and conceptual framework can be used in different ways, they describe 3 uses that HPE researchers frequently rely on: fully inductive theory development, fully theory-informed inductive, and theory-informing inductive data analysis. | https://journals.lww.com/academicmedicine/Fulltext/2020/07000/The_Distinctions_Between_Theory,_Theoretical.21.aspx?context=FeaturedArticles&collectionId=8 | |||||||||||||||||||||
23 | Research design,Statistical approaches,Diversity, equity, and inclusion | Article | 2020 | Chemistry Education Research and Practice | Rocabado, G. A., Komperda, R., Lewis, J. E., & Barbera, J. (2020). Addressing diversity and inclusion through group comparisons: a primer on measurement invariance testing. Chemistry Education Research and Practice, 21(3), 969-988. | As the field of chemistry education moves toward greater inclusion and increased participation by underrepresented minorities, standards for investigating the differential impacts and outcomes of learning environments have to be considered. While quantitative methods may not be capable of generating the in-depth nuances of qualitative methods, they can provide meaningful insights when applied at the group level. Thus, when we conduct quantitative studies in which we aim to learn about the similarities or differences of groups within the same learning environment, we must raise our standards of measurement and safeguard against threats to the validity of inferences that might favor one group over another. One way to provide evidence that group comparisons are supported in a quantitative study is by conducting measurement invariance testing. In this manuscript, we explain the basic concepts of measurement invariance testing within a confirmatory factor analysis framework with examples and a step-by-step tutorial. Each of these steps is an opportunity to safeguard against interpretation of group differences that may be artifacts of the assessment instrument functioning rather than true differences between groups. Reflecting on and safeguarding against threats to the validity of the inferences we can draw from group comparisons will aid in providing more accurate information that can be used to transform our chemistry classrooms into more socially inclusive environments. To catalyze this effort, we provide code in the ESI for two different software packages (R and Mplus) so that interested readers can learn to use these methods with the simulated data provided and then apply the methods to their own data. Finally, we present implications and a summary table for researchers, practitioners, journal editors, and reviewers as a reference when conducting, reading, or reviewing quantitative studies in which group comparisons are performed. | https://pubs.rsc.org/en/content/articlelanding/2020/RP/D0RP00025F | |||||||||||||||||||||
24 | Statistical approaches,Data trustworthiness | Article | 2020 | Journal of Chemical Education | Barbera, J., Naibert, N., Komperda, R., & Pentecost, T. C. (2020). Clarity on Cronbach’s Alpha Use. Journal of Chemical Education, 98(2), 257-258. | The Cronbach’s alpha (α) statistic is regularly reported in science education studies. However, recent reviews have noted that it is not well-understood. Therefore, this commentary provides additional clarity regarding the language used when describing and interpreting alpha and other estimates of reliability. | https://pubs.acs.org/doi/abs/10.1021/acs.jchemed.0c00183 | |||||||||||||||||||||
25 | Data trustworthiness,Instrument development and use | Article | 2021 | Journal of Chemical Education | Deng, J. M., Streja, N., & Flynn, A. B. (2021). Response process validity evidence in chemistry education research. Journal of Chemical Education, 98(12), 3656-3666. | Response process validity evidence can provide researchers with insight into how and why participants interpret items on instruments such as tests and questionnaires. In chemistry education research literature and the social sciences more broadly, response process validity evidence has been used and reported in a variety of ways. This paper’s objective is to support researchers in developing purposeful, theory-driven protocols to investigate response processes. We highlight key considerations for researchers who are interested in using cognitive interviews in their research, including the following: the theoretical basis for response process investigations, collection of response process validity evidence through cognitive interviews, and use of that evidence to inform instrument modifications. | https://pubs.acs.org/doi/abs/10.1021/acs.jchemed.1c00749 | |||||||||||||||||||||
26 | Data trustworthiness | Article | 2021 | Chemistry Education Research and Practice | Watts, F. M., & Finkenstaedt-Quinn, S. A. (2021). The current state of methods for establishing reliability in qualitative chemistry education research articles. Chemistry Education Research and Practice, 22(3), 565-578. | The tradition of qualitative research drives much of chemistry education research activity. When performing qualitative studies, researchers must demonstrate the trustworthiness of their analysis so researchers and practitioners consuming their work can understand if and how the presented research claims and conclusions might be transferable to their unique educational settings. There are a number of steps researchers can take to demonstrate the trustworthiness of their work, one of which is demonstrating and reporting evidence of reliability. The purpose of this methodological review is to investigate the methods researchers use to establish and report reliability for chemistry education research articles including a qualitative research component. Drawing from the literature on qualitative research methodology and content analysis, we describe the approaches for establishing the reliability of qualitative data analysis using various measures of inter-rater reliability and processes including negotiated agreement. We used this background literature to guide our review of research articles containing a qualitative component and published in Chemistry Education Research and Practice and the Journal of Chemical Education from the years 2010 through 2019 for whether they report evidence of reliability. We followed this by a more in-depth analysis of how articles from the years 2017 through 2019 discuss reliability. Our analysis indicates that, overall, researchers are presenting evidence of reliability in chemistry education research (CER) articles by reporting reliability measures, describing a process of negotiated agreement, or mentioning reliability and the steps taken to demonstrate it. However, there is a reliance on reporting only percent agreement, which is not considered an acceptable measure of reliability when used on its own. In addition, the descriptions of how reliability was established were not always clear, which may make it difficult for readers to evaluate the veracity of research findings. Our findings indicate that, as a field, CER researchers should be more cognizant of the appropriateness of how we establish reliability for qualitative analysis and should more clearly present the processes by which reliability was established in CER manuscripts. | https://pubs.rsc.org/en/content/articlelanding/2021/rp/d1rp00007a | |||||||||||||||||||||
27 | Theoretical framework, Diversity, equity, and inclusion | Article | 2022 | CBE—Life Sciences Education | Shukla, S. Y., Theobald, E. J., Abraham, J. K., & Price, R. M. (2022). Reframing Educational Outcomes: Moving beyond Achievement Gaps. CBE—Life Sciences Education, 21(2), es2. | The term “achievement gap” has a negative and racialized history, and using the term reinforces a deficit mindset that is ingrained in U.S. educational systems. In this essay, we review the literature that demonstrates why “achievement gap” reflects deficit thinking. We explain why biology education researchers should avoid using the phrase and also caution that changing vocabulary alone will not suffice. Instead, we suggest that researchers explicitly apply frameworks that are supportive, name racially systemic inequities and embrace student identity. We review four such frameworks—opportunity gaps, educational debt, community cultural wealth, and ethics of care—and reinterpret salient examples from biology education research as an example of each framework. Although not exhaustive, these descriptions form a starting place for biology education researchers to explicitly name systems-level and asset-based frameworks as they work to end educational inequities. | https://www.lifescied.org/doi/10.1187/cbe.21-05-0130 | |||||||||||||||||||||
28 | Statistical approaches | Article | 2021 | Journal of Microbiology & Education | Ballen, C. J., & Salehi, S. (2021). Mediation analysis in discipline-based education research using structural equation modeling: beyond “what works” to understand how it works, and for whom. Journal of microbiology & biology education, 22(2), e00108-21. | Advancing the field of discipline-based education research (DBER) requires developing theories based on outcomes that integrate across multiple methodologies. Here, we describe mediation analysis with structural equation modeling as one statistical tool that allows us to further examine mechanisms underlying well-documented trends in higher education. The use of mediation analysis in educational settings is particularly powerful, as learning outcomes result from complex relationships among many variables. We illustrate how mediation analysis can enhance education research, addressing questions that cannot be easily reached otherwise. We walk through critical steps to guide decision-making in mediation analysis and apply them to questions using real data to examine performance gaps in large introductory courses in biology. Through the use of mediation analysis with structural equation modeling, we add to a growing body of research that shows diverse quantitative approaches support evidence-based teaching in higher education. | https://journals.asm.org/doi/pdf/10.1128/jmbe.00108-21 | |||||||||||||||||||||
29 | Theoretical framework | Article | 2022 | CBE—Life Sciences Education | Luft, J. A., Jeong, S., Idsardi, R., & Gardner, G. (2022). Literature Reviews, Theoretical Frameworks, and Conceptual Frameworks: An Introduction for New Biology Education Researchers. CBE—Life Sciences Education, 21(3), rm33. | To frame their work, biology education researchers need to consider the role of literature reviews, theoretical frameworks, and conceptual frameworks as critical elements of the research and writing process. However, these elements can be confusing for scholars new to education research. This Research Methods article is designed to provide an overview of each of these elements and delineate the purpose of each in the educational research process. We describe what biology education researchers should consider as they conduct literature reviews, identify theoretical frameworks, and construct conceptual frameworks. Clarifying these different components of educational research studies can be helpful to new biology education researchers and the biology education research community at large in situating their work in the broader scholarly literature | https://www.lifescied.org/doi/10.1187/cbe.21-05-0134 | |||||||||||||||||||||
30 | Data trustworthiness | Article | 2022 | Chemistry Education Research and Practice | Lewis, S. E. (2022). Considerations on validity for studies using quantitative data in chemistry education research and practice. Chemistry Education Research and Practice. | An explicit account of validity considerations within a published paper allows readers to evaluate the evidence that supports the interpretation and use of the data collected within a project. This editorial is meant to provide considerations on how validity has been presented and reviewed among papers submitted to Chemistry Education Research and Practice (CERP) that analyze quantitative data. Authors submitting to CERP are encouraged to make an explicit case for validity and this editorial describes the varying sources of evidence that can be used to organize the evidence presented for validity. | https://pubs.rsc.org/en/Content/ArticleLanding/2022/RP/D2RP90009B?s=09 | |||||||||||||||||||||
31 | ||||||||||||||||||||||||||||
32 | ||||||||||||||||||||||||||||
33 | ||||||||||||||||||||||||||||
34 | ||||||||||||||||||||||||||||
35 | ||||||||||||||||||||||||||||
36 | ||||||||||||||||||||||||||||
37 | ||||||||||||||||||||||||||||
38 | ||||||||||||||||||||||||||||
39 | ||||||||||||||||||||||||||||
40 | ||||||||||||||||||||||||||||
41 | ||||||||||||||||||||||||||||
42 | ||||||||||||||||||||||||||||
43 | ||||||||||||||||||||||||||||
44 | ||||||||||||||||||||||||||||
45 | ||||||||||||||||||||||||||||
46 | ||||||||||||||||||||||||||||
47 | ||||||||||||||||||||||||||||
48 | ||||||||||||||||||||||||||||
49 | ||||||||||||||||||||||||||||
50 | ||||||||||||||||||||||||||||
51 | ||||||||||||||||||||||||||||
52 | ||||||||||||||||||||||||||||
53 | ||||||||||||||||||||||||||||
54 | ||||||||||||||||||||||||||||
55 | ||||||||||||||||||||||||||||
56 | ||||||||||||||||||||||||||||
57 | ||||||||||||||||||||||||||||
58 | ||||||||||||||||||||||||||||
59 | ||||||||||||||||||||||||||||
60 | ||||||||||||||||||||||||||||
61 | ||||||||||||||||||||||||||||
62 | ||||||||||||||||||||||||||||
63 | ||||||||||||||||||||||||||||
64 | ||||||||||||||||||||||||||||
65 | ||||||||||||||||||||||||||||
66 | ||||||||||||||||||||||||||||
67 | ||||||||||||||||||||||||||||
68 | ||||||||||||||||||||||||||||
69 | ||||||||||||||||||||||||||||
70 | ||||||||||||||||||||||||||||
71 | ||||||||||||||||||||||||||||
72 | ||||||||||||||||||||||||||||
73 | ||||||||||||||||||||||||||||
74 | ||||||||||||||||||||||||||||
75 | ||||||||||||||||||||||||||||
76 | ||||||||||||||||||||||||||||
77 | ||||||||||||||||||||||||||||
78 | ||||||||||||||||||||||||||||
79 | ||||||||||||||||||||||||||||
80 | ||||||||||||||||||||||||||||
81 | ||||||||||||||||||||||||||||
82 | ||||||||||||||||||||||||||||
83 | ||||||||||||||||||||||||||||
84 | ||||||||||||||||||||||||||||
85 | ||||||||||||||||||||||||||||
86 | ||||||||||||||||||||||||||||
87 | ||||||||||||||||||||||||||||
88 | ||||||||||||||||||||||||||||
89 | ||||||||||||||||||||||||||||
90 | ||||||||||||||||||||||||||||
91 | ||||||||||||||||||||||||||||
92 | ||||||||||||||||||||||||||||
93 | ||||||||||||||||||||||||||||
94 | ||||||||||||||||||||||||||||
95 | ||||||||||||||||||||||||||||
96 | ||||||||||||||||||||||||||||
97 | ||||||||||||||||||||||||||||
98 | ||||||||||||||||||||||||||||
99 | ||||||||||||||||||||||||||||
100 |