Preparing Students to Work with AI
Anna Mills, English Instructor at Cañada College
A presentation for Kentucky Community & Technical College System
October 26, 2023
Licensed CC BY NC 4.0
Welcome!
I like to approach this in a spirit of inquiry. I’ve had the chance to learn about the topic it for the last year, but I don’t claim to have all the answers. It feels best to explore the subject with educators and students.
Slides (open for commenting):
What to expect
What do we mean by “AI”?
“AI” or “Artificial Intelligence” in general is used these days to refer to computer systems that essentially uncover patterns in big data and follow those patterns to make predictions or recommendations. They use what’s known as “machine learning” to train until their predictions are more or less reliable.
When we say AI in higher ed these days, we are often talking about generative AI, systems that generate text, images, and video based on a user request.
Other kinds of AI: systems that make recommendations or take action based on patterns in past data (machine learning).
Such AI systems can be designed to (try to)
Some basics about AI text generators like ChatGPT
Presentation by Anna Mills, licensed CC BY NC 4.0.
ChatGPT Alternatives
ChatGPT is just one chatbot that runs on OpenAI’s large language models, GPT-3.5 (free version) or the more sophisticated GPT-4 (premium).
What does text generation look like? Say we need to write a memo on our organization’s new emergency plan. ChatGPT gives a version of such a memo (probably not copied from the internet), and we can refine it with our own specifics.
I’m going to bracket the questions about policy and academic integrity that have dominated the initial discourse in higher ed to give us a chance to look at how AI may change the kind of preparation students need.
I’ll just say that the prevailing strong recommendation is to make policies about AI explicit and detailed. For more on the questions below, see my resource list with the Writing Across the Curriculum Clearinghouse
Presentation by Anna Mills, licensed CC BY NC 4.0.
Will AI automate the jobs students are preparing for?
As I understand it, that is largely still to be determined, partly by the decisions we make as a society about regulation of AI.
Frank Pasquale makes a case for regulation in The New Laws of Robotics: Defending Human Expertise in the Age of AI, Harvard University Press, 2020:
“Robotic meat- cutters make sense; robotic day care gives us pause.”
“Too many CEOs tell a simple story about the future of work: if a machine can do what you do, your job will be automated. They envision everyone from doctors to soldiers rendered superfluous by ever-more-powerful AI. They offer stark alternatives: make robots or be replaced by them. Another story is possible. In virtually every walk of life, robotic systems can make labor more valuable, not less.”
What we know: professionals will be working with AI
“By 2030, activities that account for up to 30 percent of hours currently worked across the US economy could be automated—a trend accelerated by generative AI. However, we see generative AI enhancing the way STEM, creative, and business and legal professionals work rather than eliminating a significant number of jobs outright. Automation’s biggest effects are likely to hit other job categories. Office support, customer service, and food service employment could continue to decline.” –McKinsey Global Institute, “Generative AI and the future of work in America,” July 26, 2023
“Gen AI and other recent trends affecting the labor market point to the importance of upskilling, reskilling, and training to empower the global workforce.” - McKinsey Podcast, September 21, 2023
Chatbots may not be the future of text generation.
With the next generation of AI user interfaces, generative AI is present within familiar applications and might be used in myriad ways throughout a task.
Google now offers beta text generation within Google Docs
Nowhere in version history does it indicate which text is auto-generated and which is written by the user.
Let’s take a look at what it’s like to work with AI in Google Docs.
On the right is a screenshot of options for auto-revising the AI text.
Microsoft is releasing Copilot, “Your everyday AI companion”
“Microsoft 365 Chat combs across your entire universe of data at work, including emails, meetings, chats, documents and more, plus the web. Like an assistant, it has a deep understanding of you, your job, your priorities and your organization. It goes far beyond simple questions and answers to give you a head start on some of your most complex or tedious tasks — whether that’s writing a strategy document, booking a business trip, or catching up on emails.”
So how do we prepare students now to work with AI in future?
How do you work with generative AI?
How do you know how to change what it gives you?
Critical thinking, reading, and writing skills?
Subject-matter expertise?
Hmm, I think that’s what we do in higher ed…
So let’s keep teaching those things! They will be needed in new work contexts.
Should we be teaching “prompt engineering”--techniques for getting the most out of current systems like ChatGPT?
User interfaces are changing so quickly that any current technical aspects of prompting may go away.
“Despite the buzz surrounding it, the prominence of prompt engineering may be fleeting. A more enduring and adaptable skill will keep enabling us to harness the potential of generative AI? It is called problem formulation — the ability to identify, analyze, and delineate problems.” Aguz A. Acar, “AI Prompt Engineering Isn’t the Future,” Harvard Business Review, June 6, 2023
“Identify, analyze, and delineate problems”? We’re back to critical thinking, no?
“Prompt engineering” may not be essential in the long run.
But our guidance is still important
Students do need to understand some basic things about AI.
We don’t know through what interface we’ll interact with AI in a year. But we can be sure students need to know that they need to question what AI gives them to “add value.”
Digital literacy needs to include critical AI literacy
Goals around AI in higher ed
What (if anything) do you tell students about AI? Is there anything you think to yourself about AI that you don't tell them?
�Scan the QR code
Or, go to https://www.menti.com/al7nmcnexgqv (in the chat)�Or, go to Menti.com and enter 43 89 24 1
Critical AI Literacy Microlessons
Teach one or more microlessons on
AI Microlessons Start building a foundation of critical AI Literacy before we invite students to use generative AI
How do we start? Baby steps?
Watch a very short video?�
Perhaps assign an interactive reading?
Let Us Show You How GPT Works — Using Jane Austen from The New York Times gives readers the choice of seeing rough language model generations in the style of Harry Potter, Star Trek: The Next Generation, Shakespeare, or Moby Dick.
Sample: “Hermione raised her wand. "Professor Dumbledore never mimmed Harry."
He looked back at the room, but they didn't seem pretend to blame Umbridge in the Ministry. He had taken a human homework, who was glad he had not been in a nightmare bad cloak. Her face looked over her closely past her and saw Harry crossed the grip, looked down at the wall. "Come off!" she said tentatively, with a crumpled note into his own hand on her book. "That’s beating attacks how we’ve got detentions or not to realize how she did the Maps worse doesn’t want.”
Assign a short reading like the introduction to Elements of AI, a set of free online course materials from the University of Helsinki.
Artificial Intelligence: Last Week Tonight with John Oliver is well researched. Caveat: there’s a bit of swearing.
Consider a microlesson on one particular aspect of critical AI literacy
Consider also
Labor, AI colonialism, intellectual property, environmental impact
A microlesson on privacy
(Inspired by Autumn Caines’ blog on annotating the privacy policies)
Share highlights from the OpenAI Privacy Policy (or another model's policy) with students
“We may use Personal Information for the following purposes:
Image by GDJ on Pixabay.com
A microlesson on fabrication (also called hallucination)
This Photo by Unknown author is licensed under CC BY-SA.
Here's a session where ChatGPT attributes a quote to Mark Twain that is actually a popular quote supposedly by Abraham Lincoln. It’s a paraphrase.
“Whenever I hear anyone arguing for slavery, I feel a strong impulse to see it tried on him personally."--popularly attributed to Lincoln online.
The actual Lincoln quote: “While I have often said that all men ought to be free, yet would I allow those colored persons to be slaves who want to be, and next to them those white people who argue in favor of making other people slaves. I am in favor of giving an appointment to such white men to try it on for these slaves.” (UCSB American Presidency Project)
A microlesson on AI bias
Show a video that gives an example of racism and sexism compounded by AI
Another microlesson on bias: Invite students to read “How AI reduces the world to stereotypes” by Victoria Turk in restofworld.org
Biased Barbies
“THEY GAVE SOUTH SUDAN BARBIE A GUN”
Business Insider reporting on the Buzzfeed article “A list of AI-generated Barbies from 'every country' gets blasted on Twitter for blatant racism and endless cultural inaccuracies”
“They turned Thailand Barbie Russian”--@sighyam
“A few Middle Eastern Barbies wore a ghutra, a traditional headdress for men.”--Business Insider
A microlesson on AI'ls lack of understanding: share a chat session that suggests ChatGPT isn’t thinking, such as this session where it counts wrong.
Another way to demonstrate ChatGPT’s lack of real understanding: “correct” its right answer and watch it agree. Here I told it it had rounded the decimal incorrectly for ⅓.
What approach have you taken or might you take to introducing text generation AI and/or key elements of critical AI literacy?
�Scan the QR code
Or, go to https://www.menti.com/alirvxz51kke (in the chat)�Or, go to Menti.com and enter 5628 5048
Options for incorporating AI into our pedagogy
Combine AI literacy and discipline skills
Plan learning activities involving AI that
One approach: critique AI performance on specific skills we want to teach
This could involve students
We might
Template phrases for critiquing AI outputs from How Arguments Work (samples below)
Clarity
Accuracy
Juan Pablo Pardo-Guerra at University of California San Diego has documented his approach in TextGenEd: Teaching with Text Generation Technologies from the Writing Across the Curriculum Clearinghouse
"...[S]tudents are provided with an AI-generated text relevant to a course’s topics and focus and then asked to comment, review, and expand on it using a feature such as "track changes." In engaging with the AI-generated text, students review their knowledge, offer critiques, modify theoretical and empirical claims, and provide concrete examples that illustrate or disprove the provided answer.... additionally, they gain some AI literacy in evaluating the AI-generated text."
Critique AI summaries
Professional Writing for Healthcare: Writing & Revising Research Summaries with Artificial Intelligence by Heidi A. McKee in TextGenEd
Translating a Policy Document into Plain English by Timothy Laquintano, Lafayette College in TextGenEd
PerplexityAI integrates chatbot and search features and doesn't require sign-in. Like Bing Chat and Google's Bard, it will provide brief summaries of sources it links to. Ask students to read the sources and find inaccuracies in the summaries.
A bit of fun: critique summaries generated in tailored styles
In their TextGenEd contribution, Daniel Hutchinson and Erin Jensen of
Belmont Abbey College invite students to ask a language model to explain a difficult passage or concept using an example that incorporates the student’s pop culture reference of choice.
Then they ask students to evaluate how well the model’s example illustrates the meaning of the passage.
Critique an AI critical assessment
Students reflect on the differences between a ChatGPT critical assessment and a human-written one about the same New York Times article.
What did ChatGPT miss? What did its output get right?
How do those observations match what we learned about how language models work?
How might the student essay have turned out if the student had started with the ChatGPT output and revised from there?
What lessons do we draw from this comparison?
Critique AI "research"
Have students fact-check AI-generated claims and references to sources as a way to help them learn about academic research and peer review.
In my first-year writing class, we just did this through collaborative annotation of a chat session.
I asked students to check if sources were real and accurately described
ChatGPT output from a session on surprising AI facts:
AI Can Decode Ancient Scripts:
No such paper. No such author. (Yann Bengio mixes Yann LeCun and Yoshua Bengio, two famous AI scientists.)
Critical reflection on supplementary AI feedback
Using LLMs as Peer Reviewers for Revising Essays by Antonio Byrd, University of Missouri-Kansas City in TextGenEd:
"Revise your essay using a large language model (LLM) such as ChatGPT. First, revise your essay in response to peer review comments from myself and your classmates.
Cast careful judgment on the responses from the LLM, as the analysis may include misinformation or show that the LLM did not understand the intent of your prompt command."
I am advising on an app I am still getting permission to pilot:
Perhaps AI can supplement teacher, tutor and peer feedback without replacing them
Guardrails and guidance are needed: language model feedback can sometimes be bad while sounding authoritative.��With guidance, though, students can practice recognizing points they disagree with in the AI feedback and points that don’t represent their essays accurately.
Guardrails and nudges
Sample template phrases for reflecting on AI feedback (From How Arguments Work)
Reflect on ways the feedback doesn't fit our meaning or purpose
What first step(s) have you taken or might you take toward preparing students to work with AI?
�Click the right arrow if you're in the Mentimeter already�Scan the QR code
Or, go to https://www.menti.com/al7nmcnexgqv (in the chat)�Or, go to Menti.com and enter 43 89 24 1
Here’s to open discussion about AI among educators!
Resource list:https://bit.ly/AITextEdu
�Get in touch:
Twitter/X: @EnglishOER
LinkedIn: anna-mills-oer
Slides open for commenting: https://bit.ly/KCTCSAI
�This presentation is shared under a CC BY NC 4.0 license.