SOTL Research on Alternative Grading (part 2)
Resources, schedule, Zoom: thegradingconference.com �Slack community: tinyurl.com/join-alt-grading
Recording and posting
We are recording this session.
We plan to post recordings and slides so you can review them later.
We retain the right to rebroadcast and distribute the recording.
By participating, you are agreeing that your contributions become part of the recording. This includes video, audio, chat, and Q&A.
The Grading Conference is funded this year by
National Science Foundation, DUE #2304776
Today: Getting started with?
Now Parallel session
13A - Tools for Alternative Grading (part 2)
13B - SOTL research on Alternative Grading (part 2)
13C - Implementations of Ungrading
4:45 - 5:00 Tea break (offline)
5:00 - 6:00 Town Hall discussion and Wrap-up
SOTL Research on Alternative Grading, part 2 - Presenter
Adria Updike
she/hers
Associate Professor of Physics @
Roger Williams University (Bristol, RI)
aupdike@rwu.edu
Please add a pic here
What happens when students have unlimited retake opportunities?
Standards-Based Grading in Introductory Physics
Adria C. Updike
Roger Williams University, Bristol, RI
(We are hiring!)
Standards-Based Grading (SBG)
Benefits to SBG
Drawbacks to SBG
Instructor Time Requirements
More re-testing allowed = more time required
What is the optimal number of re-assessments to allow?
Lit. review: most articles recommend 1-2 re-assessment opportunities
Limited number of re-assessments:
Large number of re-assessments:
How do students respond to unlimited re-testing opportunities?
4 sections of introductory calculus-based physics
Example LO: Solve a word problem involving unbalanced forces.
How do students respond to unlimited re-testing opportunities?
More than 7,000 LOs attempted and graded
Data collected: number of attempts and amount of time taken by each student to pass each LO
How many attempts did it take them to master an LO?
40% passed on the first attempt
20% passed on the second attempt
13% passed on the third attempt
3% needed 4 attempts
2.6% needed 5 attempts
1.2% needed six attempts
0.5% needed seven or more attempts
21% did not pass the LO
How many attempts did it take them to master an LO?
61% of students fell into the “4 or more attempts” category at least once during the semester
How do students respond to unlimited re-testing opportunities?
If the students didn’t pass the LO on the first attempt, how long did it take them to eventually pass it?
Fall 2021: 13.3 ± 7.3 days
Spring 2022: 20.8 ± 15.3 days
How do students respond to unlimited re-testing opportunities?
If the students didn’t pass the LO on the first attempt, how long did it take them to eventually pass it?
Re-testing opportunities given during class time
Fall 2021: 13.3 ± 7.3 days
Re-testing during office hours or by appointment
Spring 2022: 20.8 ± 15.3 days
How do students respond to unlimited re-testing opportunities?
Are students more likely to achieve LOs covered early in the semester?
r(68) = 0.386, p < .001: weak yes
Does putting off LO re-testing longer have a negative impact on the number of LOs a student achieves (directly related to course grade)?
r(68) = 0.539, p < .001: moderate yes
How do students respond to unlimited re-testing opportunities?
Weak to no correlation findings:
How do students respond to unlimited re-testing opportunities?
Continuation of study:
Spring 2023, two sections of calculus-based electromagnetism
44 enrolled students, 23 LOs
Advertised limits: 2 re-tests on first exam material, 1 re-test on the rest of material
Actual limits: students making progress towards achieving the objective are allowed an additional opportunity to re-test
Regular re-testing times scheduled, students told they need to pass an LO before the next test.
Recommendations
SOTL Research on Alternative Grading, part 2 - Presenter
Jacob Adler
He/Him
Assistant Professor of Practice
Purdue University
Twitter: @thejacobadler
Student perceptions of alternative grading in the biology classroom
National call to change how instructors in the sciences grade to help encourage student learning.
Jeff Schinske and Kimberly Tanner (2014) Teaching More by Grading Less (or Differently). CBE-Life Sciences Education.
Concept adapted from Rachel Weir (2022) The Grading Conference. Image adapted from cliparting.com
These strategies in the biology classroom are beneficial to student learning and motivation.
Shoshana Katzman et al. (2021) The effect of specifications grading on students’ learning and attitudes in an undergraduate-level cell biology course. Journal of Microbiology and Biology Education.
Alternative Grading Course Structure
IRB approval at Brescia University
(IRB Protocol #5_Spring2022 and IRB Protocol #2_Fall2022)
was completed prior to data collection on participants and was determined to be IRB exempt.
Student’s Qualitative Feedback
Method: Respondents (N=52/57) elaborated on their reflection about the types of evaluations in the course and meeting with their instructor.
Qualitative content analysis on responses was performed. Data-driven inductive categories were coded based on themes in the responses. Data was coded using cross-indexing.
Totals will not sum to 52 because respondents indicated multiple codes.
Student’s Qualitative Feedback
“By meeting with my instructor, I was able to demonstrate in words rather than writing that I understand the material. Sometimes I cannot put certain topics into words, but being able to explain that in the meeting helped me to demonstrate that I understood the material. The reflection meetings are very helpful for reviewing learning goals. By having these meetings, I am not stuck with the grade that I received on the exam. Sometimes it is easier to speak one on one rather than typing out explanations for the exam. This also allows me to prove that I have mastered the topics even if it did not seem like I did during the exam.”
Student’s Qualitative Feedback
“This is a great way of conducting a semester. It takes so much pressure off of us students who on a weekly basis are stressed out of their minds. This is great as the low pressure allows for a much better chance to learn the material and understand it.”
Student’s Qualitative Feedback
“Meeting with the instructor is a great opportunity to show growth and prove my understanding on concepts that I may have missed on the exam. I think meeting with the instructor is very helpful because I am given the opportunity to grow and prove my knowledge in areas where I may have struggled with before. When I meet with my instructor I am also reminded that this class is all about growth, which encourages me to improve.”
Qualitative Feedback Takeaways
Great starting point for things we can investigate further in terms of impact that these strategies are having.
Two things that immediately I wanted to investigate were the impact on perception of stress reduction and increased growth opportunities.
Test Anxiety and Standards-Based Grading
Recent work by Drew Lewis:
Demonstrated in Math Standards-Based Grading courses that students reported reduced test anxiety compared to their other classes.
Drew Lewis (2022) Impacts of Standards-Based Grading on Students’ Mindset and Test Anxiety. Journal of the Scholarship of Teaching and Learning.
Test Anxiety
Method: Respondents (N=50/55) completed the Test Anxiety Inventory.
Reflecting once on the alternative graded course and again for their other science courses.
Quantitative analysis was performed on the Likert-type scale (1=almost never, 4=almost always) responses and scores on individual statements were summed.
Rate your agreement with each of the following statements:
During tests I feel very tense.
I wish examinations did not bother me so much.
I seem to defeat myself while working on important tests.
I feel very panicky when I take an important test.
During examinations I get so nervous that I forget facts I really know.
Likert-type scale: almost never, sometimes, often, almost always
Joanne Taylor and Frank Deane (2002) Development of a Short Form of the Test Anxiety Inventory (TAI). The Journal of General Psychology.
Students overall report sometimes to often having test anxiety in their courses regardless of format.
S.D. = Standard Deviation
Cronbach's (1951) alpha was measured to ensure the reliability or internal consistency of the Test Anxiety Instrument test items. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951). https://doi.org/10.1007/BF02310555
A repeated-measures paired samples t-test was performed to compare student’s perceptions of test anxiety for the alternative graded course and other science courses. Lower level (100-level) and upper level (300-400-level) courses were examined.
Context:
Lowest Possible Anxiety Score: 5
Highest Possible Anxiety Score: 20
Divided Respondents into 2 Groups
Lower Level
(100-level)
Upper Level
(300-level and 400-level)
In lower level alternative grading biology courses students report having significantly lower test anxiety
S.D. = Standard Deviation
Cronbach's (1951) alpha was measured to ensure the reliability or internal consistency of the Test Anxiety Instrument test items. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951). https://doi.org/10.1007/BF02310555
A repeated-measures paired samples t-test was performed to compare student’s perceptions of test anxiety for the alternative graded course and other science courses. Lower level (100-level) and upper level (300-400-level) courses were examined.
S.D. = Standard Deviation
Cronbach's (1951) alpha was measured to ensure the reliability or internal consistency of the Test Anxiety Instrument test items. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951). https://doi.org/10.1007/BF02310555
A repeated-measures paired samples t-test was performed to compare student’s perceptions of test anxiety for the alternative graded course and other science courses. Lower level (100-level) and upper level (300-400-level) courses were examined.
In upper level courses, students report having high test anxiety regardless of course format.
In upper level courses, students report having significantly higher test anxiety regardless of format of course.
S.D. = Standard Deviation
Cronbach's (1951) alpha was measured to ensure the reliability or internal consistency of the Test Anxiety Instrument test items. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951). https://doi.org/10.1007/BF02310555
An independent unpaired t-test was performed to compare student’s perceptions of test anxiety for the alternative graded courses at the lower level (100-level) and upper level (300-400-level). This was also performed for comparisons for other science courses.
Test Anxiety Study Takeaway
Highlights the importance of using alternative grading practices early on in a student’s academic progress to help with test anxiety perceptions.
Growth Mindset
Method: Respondents (N=45/55) completed the Growth Mindset Survey.
Reflecting once on the alternative graded course and again for their other science courses.
Quantitative analysis was performed on the average (mean) scores on mindset for biology intelligence and general intelligence. Mindset scale was a 7-point Likert-type scale with the option to not to respond.
Rate your agreement with each of the following statements:
You have a certain amount of ____ intelligence and you can’t really do much to change it.
Your ____ intelligence is something about you that you can’t change very much.
No matter who you are, you can significantly change your ____ intelligence level.
To be honest, you can’t really change how intelligent you are in ____.
You can always substantially change how intelligent you are in ____.
This is a control question. Please select “____” as your response to this item.
You can learn new things, but you can’t really change your basic ____ intelligence.
No matter how much ____ intelligence you have, you can always change it quite a bit.
You can change even your basic ____ intelligence level considerably.
Lisa Limeri et al. (2020) Growing a growth mindset: characterizing how and why undergraduate students’ mindsets change. IJ STEM Ed.
No changes to growth mindset scores between alternative graded courses and other science courses
S.D. = Standard Deviation
Cronbach's (1951) alpha was measured to ensure the reliability or internal consistency of the Growth Mindset Survey items. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika
A repeated-measures t-test, paired samples t-test was performed to compare student’s perceptions of intelligence in terms of fixed and growth mindset scores. Biology intelligence and general intelligence perceptions were scored for both the alternative graded course and other science courses.
2.5-4.0
4.0-5.5
Lisa Limeri et al. (2020)
Lisa Limeri et al. (2020)
“I wasn't good at biology in high school so I had the mindset that I wasn't good at biology and that my brain didn't work well with biology, but after taking this class that has changed dramatically. I'm starting to really enjoy biology and I'm not that bad at it.”
Growth Mindset Study Takeaways
So, if student’s mindset is not changing via the courses, what is causing them to have such high growth mindsets?
Perhaps it is something that needs a unique instrument to delineate the differences.
For example, about 60% of students are student athletes. Maybe this is playing a role in their high growth mindset scores.
Student perceptions of alternative grading in the biology classroom
SOTL Research on Alternative Grading, part 2 - Presenter
Jayme Dyer, PhD she/her
Adjunct Instructor of Biology Durham Technical Community College, Durham, NC
Freelance Science Communicator
Biology Education YouTuber https://youtube.com/YouTooBio
Multiple grading schemes especially benefits students of color.
What’s next?
Now until 4:45 Tea break (offline)
Next
5:00 - 6:00 Town Hall discussion and Wrap-up