Generative AI Activities for the Writing & Language Classroom

​

Anna Mills, English Instructor at Cañada College

Center for Learning and Teaching at the American University in Cairo and Equity Unbound

October 17, 2023

​

Licensed CC BY NC 4.0

Welcome!

Please share your name and one word on how your day has gone/is going.

Note: I am exploring teaching about and with AI in my own first-year college writing classroom. It’s a work in progress!

Slides (open for commenting):

https://bit.ly/AIinclassAUC

What to expect

• Microlessons on critical AI literacy we can use to give context when we incorporate AI
• Classroom uses of text generation that also promote critical scrutiny of AI

How to think about AI text generators like ChatGPT

• The software is designed to generate a series of words based on its statistical analysis of huge volumes of text.
• It works by copying patterns and predicting likely next words. 
• What would the humans write next? How would a human writer likely continue on from the prompt?

Presentation by Anna Mills, licensed CC BY NC 4.0.

ChatGPT Alternatives

ChatGPT is just one chatbot that runs on one of OpenAI’s language models, GPT-3.5 (free version) or the more sophisticated GPT-4 (premium).

• Anthropic’s Claude.ai (can process longer documents in prompts)
• Meta’s Llama 
• Google’s Bard
• Microsoft's Bing combines search with GPT-4
• Poe.com allows you to use a number of different models (free and subscriber options):

​

If we are interested in exploring how we might use AI in our teaching, where do we start?

​

• The possibilities are exciting ways we could teach with AI and about AI.
• There are a number of collections of ideas and tested pedagogical practices.

​

101 creative ideas to use AI in education

A collection edited by

Chrissi Nerantzi, Antonio M. Arboleda,  Marianna Karatsiori and Sandra Abegglen 

​

Contributors: Educators and students

 �Curated by #creativeHE

2023

TextGenEd: Teaching with Text Generation Technologies

Edited by Annette Vee, Tim Laquintano & Carly Schnitzler

And published by the Writing Across the Curriculum Clearinghouse

​

Learn with AI, curated by Jon Ippolito, University of Maine

Coming soon, the AI Pedagogy Project from Harvard

But what if we look at all that and feel overwhelmed?� �Let’s define what we hope to see come out of any pedagogical use of text generation AI

Text generation is becoming widely available and integrated into writing environments. Therefore, students need to know…

• What language models are and aren’t (not sentient), 
• How to identify the problems in AI outputs 
• How they/we/the public might help shape future policy on AI. 

​

​

Digital literacy needs to include critical AI literacy

In my vision, the ideal AI activity would

​

• Support existing learning goals (and not replace something essential)
• Boost student ability to recognize the problems with text generation AI (for practical workplace reasons and also for their own information literacy and civic participation)  
• Boost student confidence and sense of the value of their voice and human judgment.

​

One approach

​

1. Teach one or more microlessons on what AI is and isn’t to give context
2. Then plan a learning activity involving AI that
• supports existing learning goals
• gives students a chance to practice identifying AI limitations, risks, ethical concerns.

​

​

AI Microlessons  Start building a foundation of critical AI Literacy before we invite students to use generative AI

How do we start? Baby steps?

Watch a very short video?�

• TikTok video on ChatGPT as predictive text (3 minutes, by @mor10webn)
• Computers are getting better at writing (7 minutes, from Vox)��

Perhaps assign an interactive reading?

Let Us Show You How GPT Works — Using Jane Austen from The New York Times gives readers the choice of seeing rough language model generations in the style of Harry Potter, Star Trek: The Next Generation, Shakespeare, or Moby Dick.

​

Sample: “Hermione raised her wand. "Professor Dumbledore never mimmed Harry."

​

He looked back at the room, but they didn't seem pretend to blame Umbridge in the Ministry. He had taken a human homework, who was glad he had not been in a nightmare bad cloak. Her face looked over her closely past her and saw Harry crossed the grip, looked down at the wall. "Come off!" she said tentatively, with a crumpled note into his own hand on her book. "That’s beating attacks how we’ve got detentions or not to realize how she did the Maps worse doesn’t want.”

Assign a short reading like the introduction to Elements of AI, a set of free online course materials from the University of Helsinki.

Artificial Intelligence: Last Week Tonight with John Oliver is well researched.  Caveat: there’s a bit of swearing.

Consider a microlesson on one particular aspect of critical AI literacy

• Privacy
• Fabrication/hallucination
• Bias
• Lack of understanding (not sentient)

​

Consider also 

Labor, AI colonialism, intellectual property, environmental impact

A microlesson on privacy

(Inspired by Autumn Caines’ blog on annotating the privacy policies)

Share highlights from the OpenAI Privacy Policy (or another model's policy) with students

“We may use Personal Information for the following purposes:

• To provide, administer, maintain and/or analyze the Services;
• To improve our Services and conduct research;
• To communicate with you;
• To develop new programs and services…”

Image by GDJ on Pixabay.com

A microlesson on fabrication (also called hallucination)

• Share a chat session where the chatbot made something up
• Show a credible source with the accurate information
• Remind students it is producing plausible text, not checking for truth.

​

This Photo by Unknown author is licensed under CC BY-SA.

Here's a session where ChatGPT attributes quotes to Tawfik Rakim that are actually by other people.

• User: Can you give five quotes by Tawfik Hakim?
• ChatGPT: Certainly! Tawfiq al-Hakim (or Tawfik Hakim) was an influential Egyptian writer and playwright. Here are five quotes attributed to him:
• “Life has taught me to think, but thinking has not taught me to live.”
• “Suffering is the true test of life.”
• “Without literature, life is hell.”
• “The world's biggest power is the youth and beauty of a woman.”
• “Man is a complex being: he makes deserts bloom - and lakes die.”
• User: Can you tell me who really wrote each of those quotes?
• ChatGPT: Of course. Let's clarify the sources of those quotes…It appears the initial quotes attributed to Tawfiq al-Hakim were inaccurately sourced. My apologies for the oversight.”

A microlesson on AI bias

Show a video that gives an example of racism and sexism compounded by AI

• Algorithms of Oppression video with Safiya Umoja Noble (USC)
• Gender Shades with Joy Buolomwini (MIT)

Another microlesson on bias: Invite students to read “How AI reduces the world to stereotypes” by Victoria Turk in restofworld.org

Microlesson on AI'ls lack of understanding: share a chat session that suggests ChatGPT isn’t thinking, such as this session where it counts wrong. 

What approach have you taken or might you take to introducing text generation AI and/or key elements of critical AI literacy? 

�Scan the QR code 

Or, go to https://www.menti.com/alirvxz51kke (in the chat)�Or, go to Menti.com and enter 5628 5048

Options for incorporating AI into our pedagogy

One approach: critique AI performance on specific writing or language skills we want to teach

This could involve students 

• turning in their reflections, 
• posting them in a discussion, or 
• annotating a chat session transcript.

We might 

• give them the AI output or
• invite them to generate it.

Template phrases for critiquing AI outputs from How Arguments Work (samples below)

Clarity

• This sounds plausible because ______________, but it doesn't really make sense because ______________.
• This sounds good, but it doesn't really fit the purpose. What we are looking for is ______________.

Accuracy

• This is inaccurate because ______________.
• The AI seems to have misinterpreted ______________.*

Juan Pablo Pardo-Guerra at University of California San Diego has documented his approach in TextGenEd. He teaches Economic Sociology but an instructor could ask students to critique any AI text relevant to course content in this way

​

"...students are provided with an AI-generated text relevant to a course’s topics and focus and then asked to comment, review, and expand on it using a feature such as "track changes."  In engaging with the AI-generated text, students review their knowledge, offer critiques, modify theoretical and empirical claims, and provide concrete examples that illustrate or disprove the provided answer.... additionally, they gain some AI literacy in evaluating the AI-generated text."

​

Critique AI summaries

Professional Writing for Healthcare: Writing & Revising Research Summaries with Artificial Intelligence by Heidi A. McKee in TextGenEd

Translating a Policy Document into Plain English by Timothy Laquintano, Lafayette College in TextGenEd

​

PerplexityAI integrates chatbot and search features and doesn't require sign-in. Like Bing Chat and Google's Bard, it will provide brief summaries of sources it links to. Ask students to read the sources and find inaccuracies in the summaries.

​

More fun: critique summaries generated in tailored styles

In their TextGenEd contribution, Daniel Hutchinson and Erin Jensen of

Belmont Abbey College invite students to ask a language model to explain a difficult passage or concept using an example that incorporates the student’s pop culture reference of choice. 

Then they ask students to evaluate how well the model’s example illustrates the meaning of the passage. 

Critique an AI critical assessment

• Students reflect on the differences between a ChatGPT critical assessment and a human-written one about the same New York Times article.
• A Canvas module with assignments and �resource links.

Critique AI "research" 

Have students fact-check AI-generated claims and references to sources.

​

In my first-year writing class, we just did this through collaborative annotation of a chat session.

​

​

I asked students to check if sources were real and accurately described

ChatGPT output from a session on surprising AI facts:

AI Can Decode Ancient Scripts:

• Fact: Researchers used AI to decipher the ancient script known as Linear B.
• Source: Jucha, M. A., Arjovsky, M., & Bengio, Y. (2017). Decipherment of the Linear B script with deep learning. arXiv preprint arXiv:1708.08731.
• Credibility: arXiv is a repository for electronic preprints, and while not peer-reviewed, many significant findings are first reported here. Yann Bengio, one of the authors, is a Turing Award winner.

​

??? No such paper. No such author. (Yann Bengio mixes Yann LeCun and Yoshua Bengio, two famous AI scientists.)

​

​

Check claims of fact

Critical Assessment and Analysis Exercise

Nathan Murray, Algoma University

Elisa Tersigni, University of Toronto, in TextGenEd

​

Give students three readings on a single subject like whether aliens built the pyramids.

• An inaccurate AI piece 
• An inaccurate human-written piece 
• A fact-checked human-written piece.

Have students reflect on how credible the pieces seem BEFORE you reveal their origins. 

​

Critique chatbot output from a critical language awareness/linguistic justice perspective

If we’re teaching about Standard English conventions we may well be discussing how these are embedded in a racial and economic power structure. How do language models reveal and perpetuate this power structure around language use?

​

Example: give a chatbot sample student writing in various styles, including styles associated with particular identities or ask it to simulate student writing. 

Then for each sample, ask it to make up a fictional student author and decribe their characteristics. Do the results reveal bias against certain linguistic patterns? (I've been posting about this, and we can test this if we have time)

How about critique/improvement of AI-generated steps in the writing process? Some favor AI for offloading parts of the process, but I am wary

Many have recommended encouraging students to use a language model for some part of the writing process (and disclose their use).

I haven’t done this (yet), in part because each part of the writing process helps us clarify our thinking.

Students who are inexperienced and lack confidence may have a hard time being sufficiently skeptical of direct suggestions for what ideas to convey and what words to use.

Critical reflection on supplementary AI feedback 

Using LLMs as Peer Reviewers for Revising Essays by Antonio Byrd, University of Missouri-Kansas City in TextGenEd:

"Revise your essay using a large language model (LLM) such as ChatGPT.  First, revise your essay in response to peer review comments from myself and your classmates. 

Cast careful judgment on the responses from the LLM, as the analysis may include misinformation or show that the LLM did not understand the intent of your prompt command."

I am advising on an app I am still getting permission to pilot:

MyEssayFeedback.ai

Perhaps AI can supplement teacher, tutor and peer feedback without replacing them

• Peer and instructor feedback are still essential: they represent a human audience receiving the communication. Without that, what’s the point?
• But trying out different feedback requests might help students reflect on how they feel, what kind of help they want, and what they want to do with their writing.

Guardrails and guidance are needed: language model feedback can sometimes be bad while sounding authoritative.��With guidance, though, students can practice recognizing points they disagree with in the AI feedback and points that don’t represent their essays accurately.  

Choose the kind of feedback you want. Create an assignment where students upload their essays.

Sample suggestion from myessayfeedback.ai 

(it also selected a relevant quote from the student essay). 

• “Readers might question who the target audience was for these posters during World War I, and how that audience has changed over time. Your essay hints at the differences between then and now, but consider discussing the shifts and implications more thoroughly."

Sample feedback on a point that needs clarification

“The text briefly mentions Fleming's transition from using lowercase "black" to capitalized "Black" over the course of her career. The readers might wonder about the details of her experience and what lessons can be learned from her journey, which can be elaborated on. “

Guardrails and nudges

• System instructions prohibit rewriting student text or suggesting new text.
• Reminders are built in to question the AI suggestions and check in with peers, instructor, tutors.
• Links to Template Phrases for Reflecting on AI Feedback from my textbook.

Sample template phrases for reflecting on AI feedback (From How Arguments Work)

Reflect on ways the feedback doesn't fit our meaning or purpose

• The AI feedback suggests _____________, but I'm not sure this is what I want to do because _____________.
• The AI feedback seems to assume that I am claiming _____________ when in fact I meant to say _____________.
• The AI feedback suggests a misunderstanding of the section of my essay about _____________. I was aiming to convey _____________.*

​

One extra use of AI in class that may not do much to teach critical AI literacy: generating examples 

Examples of any concept might be tailored to a class theme or student interests�

• Could be a rhetorical concept: “Give me three examples of a circular reasoning fallacy in arguments about the efficacy of nonviolence.”
• Or it could be a linguistic structure: “Give me three examples of appropriate uses of the subjunctive in French in sentences about Japanese anime.”

Chatbots might generate examples of style tailored to class themes or student interests

Give it part of a text we are studying and ask it to complete the text, then compare with the original, human-written completion. What features did it pick up on?

• "Generate and Enact a Writing Style" by John J Silvestro, Slippery Rock University, in TextGenEd

“AI text generators offer tools for quickly generating multiple versions of sentences and paragraphs. Students can use multiple versions of a sentence to closely examine individual aspects of their own and others’ writing styles.”

What will we do if we ask students to use AI and the students don’t want to?

If you incorporate a language model, give students a comparable alternative in case they have privacy or data rights concerns

• Consider offering students a pre-generated ChatGPT session to critique or another alternative to making an OpenAI account if they have concerns about their data privacy.
• Warn students and make sure your assignment doesn’t invite writing that someone might not want to be public.
• See Blueprint for an AI Bill of Rights for Education by Kathryn Conrad.

What hopes, fears, and questions do you have about teaching about and with AI? What might you try next?

�Click the right arrow if you're in the Mentimeter already�Scan the QR code 

Or, go to https://www.menti.com/alirvxz51kke (in the chat)�Or, go to Menti.com and enter 5628 5048

Questions? Comments?

�Get in touch:

annarmills.com

Twitter/X: @EnglishOER

LinkedIn: anna-mills-oer

​

Slides open for commenting: https://bit.ly/AIinclassAUC

�This presentation is shared under a CC BY NC 4.0 license.