Rethinking Coursework Assessment
in the Era of ChatGPT
Peter Merrick Ph.D
Rethinking Coursework Assessment in the Era of ChatGPT
If the student uses ChatGPT to do all the work - the result is of no value
ChatGPT demands coursework and homework change
ChatGPT demands a fundamental shift in coursework assessment
Judging work on the final submission won’t work because ChatGPT makes everything sound good
The entire learner journey needs to be assessed
Rethinking Coursework Assessment in the Era of ChatGPT
Designing an effective alternative process requires striking a balance between simplicity and efficacy.
The goal is to create an assessment framework that is not only straightforward but also capable of facilitating a comprehensive understanding of students' engagement with the learning material.
Communicating this new approach should be seamless, ensuring that both educators and students can readily embrace and implement the process in their coursework preparation and completion.
Rethinking Coursework Assessment in the Era of ChatGPT
The process is divided into three parts:
* Teacher modelling needs only be done once per cohort.
Teacher modelling
Select Tasks
Identify a model task which will be used to illustrate the process for undertaking the assessed task.
Define the assessed task - the work the student(s) will do on their own individually or in groups
Teacher modelling
Define Assessment Criteria
Establish clear guidelines for assessment, outlining the criteria for earning credit and explicitly stating what actions will not be credited.
For instance:
Teacher modelling
Define Assessment Criteria
There are two types of assessment
ChatGPT internal assessment - used to give immediate feedback during the student learning journey. Undertaken against a given grading rubric
Teacher assessment - used to give a formal grade upon student final submission.
Teacher modelling
Define Assessment Criteria
ChatGPT will give feedback against the following example criteria
Teacher modelling
Define Assessment Criteria
ChatGPT can give an indication of the grade the answer might receive.
**Grade 1 (Poor - Fail):**
There is minimal or no evidence of critical thinking, analysis, or argumentation.
**Grade 2 (Basic - Pass):**
There is limited evidence of critical thinking or analysis.
**Grade 3 (Satisfactory - Average):**
There is some evidence of critical thinking and analysis, although it may not be consistently applied.
**Grade 4 (Good - Above Average):**
It provides accurate and comprehensive coverage of the topic.
**Grade 5 (Excellent - Outstanding):**
The structure and organization of the answer are highly effective, facilitating a logical flow.
Teacher modelling
Define Assessment Criteria
Submitting an answer with an enquiry trail is mandatory.
An enquiry trail is the set of relevant edited prompt/responses generated in the course of the student enquiry
Submissions will be penalized if facts are left unverified by external sources, such as Wikipedia, especially where a student answer relies on 'facts' which are subsequently proven to be untrue.
In every case credit will be given for catching inaccuracies in information provided by ChatGPT.
��
Teacher modelling
The teacher demonstrates the defined process with respect to the model task
The answer step is iterative (i.e. it is done multiple times whereby the student answer becomes progressively better based on the feedback given by ChatGPT according to the supplied assessment criteria��
Student process
Deconstruct / Clarify
Students enquire regarding the background and key issues relevant and alluded to by the task. This step is crucial in gaining a comprehensive understanding, laying the foundation for a reasoned / critical response.
Students refrain from asking ChatGPT to directly answer the question although this may be done at the end of the process for the purpose of comparison. The student submission should aim to be superior to that which ChatGPT can produce.
��
Student process
Explore
This is the creative engagement phase of the process whereby students explore any and all aspects of the responses given by ChatGPT whether they ultimately become part of the student’s submission or not.
Students may find themselves investigating what turn out to be ‘dead ends’ (aka ‘rabbit holes’) however this is an important phase as no student can know in advance how valuable a particular line of enquiry will be.
Students may be unable to obtain a satisfactory response from ChatGPT. They should not be deterred. Credit is given for accessing alternative sources. It should be remembered that ChatGPT knows only so much - it does not know everything!
��
Student process
Verify
Students ensure the accuracy of the information provided by ChatGPT by cross-referencing with credible internet resources, such as Wikipedia or references from peer-reviewed journals.
Verifying information is an essential skill that adds depth and reliability to the student's responses.
Verification should be correctly referenced.
Relying on information provided by ChatGPT that is untrue may severely compromise the student submission. It is well-known that ChatGPT may ‘hallucinate’. There is no way to know ��
Student process
Answer, Grade, Refine
Students engage in an iterative process with ChatGPT.
���
Student process
Collate and Submit
Students conclude the process by deciding what to include in their final submission.
This encompasses relevant prompts, edited responses, the refined answer, and any pertinent comments explaining the process leading to the final answer.
Finally, the student includes a link to their actual ChatGPT session where the entire process of their learning journey can be reviewed should it be necessary.
This holistic approach not only demonstrates the journey of learning but also adds depth to the final submission.
���
Assessment
Discussion
Following the submission deadline, students will convene in a class dedicated to discussing their work in an open and collaborative forum.
During this session, the degree of participation and the quality of each student's contribution will be actively observed by the instructor.
Those students who have delved deeply into the subject material are encouraged to share their findings and advocate for their perspectives.
The class discussion, coupled with the submitted work, forms the basis for final grading.
���
Assessment
Individual assessment (if required)
For students absent from the class discussion or those whose participation is limited, a one-on-one interview is arranged.
This interview serves as an opportunity to individually assess the student’s understanding of the submitted work.
In cases where there is doubt about the level of engagement or comprehension, this personalized assessment ensures a fair evaluation and provides students with a platform to articulate their perspective.
This approach not only encourages active participation and knowledge sharing but also allows for a tailored assessment that considers individual circumstances, fostering a holistic and equitable grading process.
���
Is this where we want to demonstrate the process? Should we craft a series of slides based on a potential task? If so, I’d like to work on this collaboratively.