Generative AI and �Education Research
Roehampton doctoral students’ conference�Prof Miles Berry
These slides: bit.ly/genaier
23 March 2024
Foundations
Applications
Implications
How does this work
How can it be used
What this all means
Foundations
Foundations
Input
Model
Output
ML Algorithm
Training data
A new common language?
Google’s researchers think their system achieves this breakthrough by finding a common ground whereby sentences with the same meaning are represented in similar ways regardless of language – which they say is an example of an “interlingua”. In a sense, that means it has created a new common language, albeit one that’s specific to the task of translation and not readable or usable for humans.
Open AI?
GPT-4 is a Transformer-style model [39] pre-trained to predict the next token in a document, using both publicly available data (such as internet data) and data licensed from third-party providers. The model was then fine-tuned using Reinforcement Learning from Human Feedback (RLHF) [40]. Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.
Open AI, 2023
“It uses bits of what it’s heard and stitches them into something new … that’s exactly what we do.”
Ockleford, 2024
Applications
How can it help?
It’s very well read
It writes very well
It programs well too
It tries to be helpful
Other things that it can help with
Checking for readability / SPAG / argument
Reducing word count
What have I missed?
Transcribing interviews
Translations
Role play
Prompting well
Completion
Primary content
Examples
Cue
Supporting content
Have a conversation!
Be clear and precise
Break the task down
Chain of thought
Persona
System messages
Fine tuning
There are limits
It doesn’t really understand
It doesn’t really think - problem solving is a problem
It’s over-confident
It does make things up
It sometimes pays attention to the wrong thing
It’s kind rather than critical
GPTn are not up to date
Reliability costs
�Implications
Is it cheating if ChatGPT…
Explains something to you?
Gives you ideas for your paper?
Suggests how to improve your paper?
Writes your paper for you?
Academic integrity
Passing off the work of a generative AI tool as the student’s own is to be treated as any other form of plagiarism and colleagues will follow the university’s usual disciplinary processes. Similarly, using data from a generative AI tool in place of experiment, interview or survey would be considered falsification and treated as such.
Citing the AIs - persistent URL
In-text citation
The AI-generated flower (Shutterstock AI, 2023)
Reference list
Shutterstock AI (2023) Photo of pond with lotus flower[Digital art]. Available at: https://www.shutterstock.com/image-generated/photo-pond-lotus-flower-2252080005 (Accessed: 31 March 2023).
Citing the AIs - transient response
In-text citation
When prompted by the author, ChatGPT responded with a ‘definition of academic integrity’ (OpenAI ChatGPT, 2023). A copy of this response is in Appendix 1.
Reference list
OpenAI ChatGPT (2023) ChatGPT response to John Stephens, 2 April.
Publishers’ policies
AI use must be declared and clearly explained in publications such as research papers, just as we expect scholars to do with other software, tools and methodologies.
AI does not meet the Cambridge requirements for authorship, given the need for accountability. AI and LLM tools may not be listed as an author on any scholarly work published by Cambridge
Authors are accountable for the accuracy, integrity and originality of their research papers, including for any use of AI.
Any use of AI must not breach Cambridge’s plagiarism policy. Scholarly works must be the author’s own, and not present others’ ideas, data, words or other material without adequate citation and transparent referencing.
Data protection and IP
There are risks to privacy and intellectual property associated with the information that students and/or staff may enter. It is important to consider GDPR, data protection and whose intellectual property may be infringed when using generative AI.
Terms and �conditions apply
What stakeholders expect
Publishers, research funders and the public have a reasonable expectation that Roehampton's research is the original work of academics and research students and not the output of generative AI, unless the contribution of generative AI is clearly acknowledged. In any circumstance, academics must abide by all relevant ethical and legal frameworks and any contractual obligations for funded or published research.
Bias…
Mind and society
The child begins to perceive the world not only through his [or her] eyes but also through his [or her] speech
Vygotsky, 1978
Talk as the currency of learning
Talk is … the currency of learning — how we develop and shape our ideas, deepen our thinking, explore subject matter and share our thoughts and feelings.
Should learning be hard?
Learning is at its best, human beings are at their best, when they are challenged and overcome those challenges. AI will make life easy and strip away learning and teaching — unless we get ahead of it.
Are you thinking
As AI performance improves, human overseers face greater incentives to delegate. If the AI appears too high quality, workers are at risk of “falling asleep at the wheel" and mindlessly following its recommendations without deliberation. In such settings, maximizing combined human/AI performance requires trading off the quality of AI against the potential adverse impact on human effort.
And here's how you should think about memory: it's the residue of thought, meaning that the more you think about something, the more likely it is that you'll remember it later.
Any questions?
These slides: bit.ly/genaier
m.berry@roehampton.ac.uk
0208 392 3241