Exploring AI Pedagogy:

A Community Collection of Teaching Reflections

MLA-CCCC Joint Task Force on Writing and AI

February 22, 2024

Today’s Presenters

MLA Hosts: Janine Utell and Ayanni Cooper

​

Hosts from MLA/CCCC Task Force on Writing and AI:

Anna Mills (College of Marin)

Sarah Z Johnson (Madison College)

Elizabeth Losh (William and Mary University)

​

Contributors to the collection

Scott Rettberg (University of Bergen Center for Digital Narrative)

Anuj Gupta (University of Arizona)

Jill Walker Rettberg (University of Bergen Center for Digital Narrative)

​

​

​

What’s happening today?

1. About the task force
2. About the collection
3. Contributors’ remarks
4. Exploration of the collection: an activity with breakout groups
5. Questions, discussion, and an invitation to contribute

​

Slides with links open for commenting: https://bit.ly/ExploringAIPedagogyWebinar

Background on the Task Force on Writing and AI

Formed in November 2022 by the executive leadership of the Modern Language Association and the National Council of Teachers of English College Conference on Composition and Communication.

Highlights:

• Resource guide and blog
• Working Paper 1: Overview of the Issues, Statement of Principles, and Recommendations
• What AI Means for Teaching Webinar in July with 3,500+ registrants
• Statement for the White House Office of Science and Technology Policy
• Initial Guidance for Evaluating the Use of AI in Scholarship and Creativity
• Coming out very soon: Working Paper 2 on considerations for AI policy development.

Why a community collection of teaching experiments? (1)

• Our Spring 2023 survey and the response to our July webinar suggested educators are looking for immediate practical assistance on how to adapt teaching.
• As AI evolves, educators experiment with ways to teach about it and use it in pedagogy. A crowdsourced space allows for timely reflections.
• Making the reflections short and encouraging comments lowers the barriers to participation in collaborative inquiry and mutual assistance.
• We need not wait until we are sure we have a model approach to promote. A practical break-room conversation ethos helps. Let’s reflect on successes, failures, and results that leave us perplexed.

​

Why a community collection of teaching experiments? (2)

• We cast a wide net to include approaches to academic integrity and AI, critical perspectives on AI, and excitement about pedagogical possibilities, bringing these related topics into conversation.
• A crowdsourced collection complements curated, peer-reviewed lists. We join an ecosystem of spaces for sharing ideas for teaching about and with AI like TextGenEd, AIPedagogy.org, LearnwithAI.org, and CRAFT AI Literacy Resources. Our collection includes reflections on and links to peer-reviewed materials published elsewhere.

Highlights

• We reached out to educators working on these topics to pre-populate the collection before our soft launch in November 2023.
• Angela Gibson and Anatole Shukla of MLA built the space (thank you!).
• We have 55 reflections so far.
• Categories (we’re trying to keep it simple and add as needed): �Academic integrity, Bias in AI, Brainstorming, “Creative” output, Essay Writing, Recognizing AI-Generated Output, Text Generation
• Positive comments on the site and on X, LinkedIn, BlueSky, and Facebook
• Featured in Maha Bali’s One Year into ChatGPT: Resources & Possible Directions for Educators in 2024. “The reflections are truly helpful.”

Contributors as of February 20, 2024

Contributors’ Remarks

• Scott Rettberg (University of Bergen Center for Digital Narrative)

Authors Reviewing Authors Reviewing Authors Reviewing Authors…

Scott Rettberg

Professor of Digital Culture

Director, the Center for Digital Narrative

Co-author a book with AI and publish it in a week

In the Authors Reviewing Authors Reviewing Authors… experiment, our research group (professors, PhD students, MA students) set ourselves a goal to produce a book using ChatGPT 3.5 – the majority of the 129-page book was written (prompted) during a two-hour session on a Friday in September 2023. The editing took place over the weekend, and the book was published the following Tuesday.

Constrained writing with ChatGPT

The book was guided by a simple constraint. We asked the prompters to put their prompts in the form of “Write a review of Author X in the idiomatic voice of Author X.” For example: “Write a review of William Faulkner’s The Sound and the Fury in the idiomatic voice of Robert Coover’s The Public Burning” or “Write a review of William S. Burroughs’s Naked Lunch in the idiomatic voice of Jane Austen’s Pride and Prejudice.”

Reviews and Reviews of Reviews

Not all the reviews were positive. In fact, the blurbs of reviews from several well-respected national publications were obnoxious.

​

The Economist actually mentioned it. In real life!

But the guy from The Economist quite liked the project.

What did we learn? Have we learned anything?

• Some biases of the training dataset are also revealed by this type of exercise: the responses to prompts involving Norwegian literature, for example were much more cursory than those involving well-known works of American Literature
• The responses demonstrate the “power of the prompt.” Although most of the text was composed by ChatGPT, the responses are constrained by the human author who largely determines the style and tone of the texts. 

• The results of this exercise demonstrate some of the extents and limits of what ChatGPT’s training set “knows” about literature.
• It also demonstrated the ways in which the context, not only of a given prompt, but of the series of prompts and responses in a given conversation, have effects on the subsequent content.

Cyborg Authorship

• This is a dialogic genre of one-liners, probing a responsive system that has at its disposal a vast trove of literature and trivia, and can produce compellingly readable results when properly deployed.
• It is what I describe a process of cyborg authorship, in which human intelligence is interacting with language as a nonconscious cognitive system.

• This sort of experiment can be considered a form of serious play. Working with these kind of stylistic constraints, and engaging with the LLM in a sustained way, might help students better understand what these systems are and are not from a critical perspective.
• It may also help us to begin to understand AI Chatbots as creative writing environments.
• Lesson: AI will not write well for you, but you can write with it.

​

How to approach AI chatbots in writing and literature classrooms

• Maybe instead of being consumed by paranoia about our students turning in writing produced by AI, we should encourage them to understand AI chatbots as writing environments in which they have agency.
• When I give lectures on AI, I always point out that although LLMS are driven by math and computer programming, the application of these tools is driven by language. Industry now needs writers. If you want to get a good job in technology, study English or Comparative Literature.

• Fun produces engagement. Engagement provides opportunities for criticality.
• There is a great deal to learn about literary style from engaging with AI chatbots.
• These are a great environment for collaborative writing of various kinds. Learn about digital media by engaging with it together.
• AI is bad at many things that humans are good at. Contrasting generated writing with student writing might even be empowering.

​

​

​

Get your free copy today!

Bonus Tracks: More on AI Writing

• The Center for Digital Narrative

Norwegian Center of Research Excellence focused on algorithmic narrativity.

• Cyborg Authorship: Writing with AI – Part 1: The Trouble(s) with Chat GPT

An essay published in the Electronic Book Review on ethical and systemic problems with ChatGPT in the context of cyborg authorship.

• Republicans in Love (slideshow) (book)

A text-to-image generation project primarily during the month following the November 2022 United States Congressional that explores the extent to which platforms such as DALL•E 2 can be used for satirical literary purposes.

• Cyborg Authorship Lecture (video) (slides)

Talk exploring the relationship between humans and AI in the realm of authorship. While AI might not possess human-like consciousness, they can still be creative. Large language models can serve as platforms for cyborg authorship.

• Off Center (podcast)

Interviews with leading experts on digital narratives of all sorts.

Contributors’ Remarks

​

• Anuj Gupta (University of Arizona) �
• Big existential fear among many of us and our students: �Generative AI will replace us and our writing!

• A constellation of pedagogic and scholarly efforts currently underway to tackle these fears: Critical AI literacy �
• A new genre of writing I see emerging around me through which writers are exerting their critical thinking and creativity in an AI writing landscape: AI Prompts and Prompt engineering [this is what I’m studying for my PhD dissertation + using in my graduate assistantships at the University of Arizona]�

​

​

• A narrative writing exercise during a workshop I did with design students in Gujarat, India: Create a character for a story. Use ChatGPT to support your work.

​

• Then I started sharing a range of prompts with different rhetorical strategies in them to complete this task and together we analysed the kinds of outputs that ChatGPT produced in response to them.

​

• Here are some examples.

(Source: Gupta 2023)

​

A screenshot of a sample ChatGPT prompt from Gupta 2023

• As the activity progressed, I introduced them to various prompt-writing principles. �
• For example, I told them how ChatGPT produces better responses when we make our prompts as detailed as possible by adding more data about the specific contexts we are writing in. �
• Using OpenAI’s guidelines for prompting, I taught them how certain kinds of formatting help ChatGPT better understand our instructions. �
• Let’s look at an example

​

(Source: Gupta 2023)

A screenshot of a sample ChatGPT prompt from Gupta 2023

• “We can conceptualize prompts as an emerging digital genre using which humans engage in the social action of interacting with ChatGPT.

​

• “Prompt engineering then is the process of iteratively designing prompts to elicit desired outputs from AI tools while critically assessing the outputs & reflecting on their usefulness.”

​

• “Additionally, prompting techniques are elements that make up rhetorical moves in prompts”

​

(Source: Gupta 2023, Gupta 2024)

​

• “ [In response to the previous prompt], when students [...] critically analyzed ChatGPT’s output [...] some students noted that most of the characters ChatGPT generated were Westernized and not representative of their lives [in Gujarat, India].

• “Thinking about prompts as a genre of writing that requires iterative revision and critical thinking enabled them to creatively use prompt engineering techniques to make ChatGPT practically address this critique.”

​

(Source: Gupta 2023)

​

A screenshot of a sample ChatGPT prompt from Gupta 2023

Takeaways

• Conceptualizing ChatGPT prompts as a digital genre can help us teach our students to practice critical thinking and creativity when using AI tools as writing support in their writing. �
• This is one of the important ways in which we can address our collective fears about AI replacing us or our students.

​

• As teachers of writing and language, we already have rhetorical expertise on how to create good prompts! �
• To learn more about different rhetorical strategies for prompting, we should start experimenting and also explore emerging best practices which are splattered all over the internet. You can look at some of the resource I’ve put together in my existing resources on prompting: Gupta 2023, Gupta 2024. �
• We’ve got this! Good luck and may the force be with us all :)

​

​

​

Contributors’ Remarks

• Jill Walker Rettberg (University of Bergen Center for Digital Narrative)

​

Jill Walker Rettberg

Professor of Digital Culture

Center for Digital Narrative

Based on our research

Premise: art, games and stories are a key site for public debates about and the expression and negotiation of desires and fears about changing technologies.

Auditing AI models

FAT - e.g. make algorithms transparent, improve them

—> gender bias in facial recognition (Joy Buolamwini)

Gender Shades project, Joy Buolamwini at MIT Media Lab

Problem 2: censorship

Dodge, Jesse, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. “Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus.” arXiv. http://arxiv.org/abs/2104.08758.

Github: List of Dirty, Naughty, Obscene and Otherwise Bad Words

bookcorpus 2

Bandy, Jack, and Nicholas Vincent. 2021. “Addressing ‘Documentation Debt’ in Machine Learning Research: A Retrospective Datasheet for BookCorpus.” arXiv. http://arxiv.org/abs/2105.05241.

Model cards proposed in 2019

Now let’s explore! Which reflections stand out to you?

• 7 minutes: Browse the site and copy links to reflections that stand out to you. Paste the links under labels that fit them in the Padlet at https://bit.ly/InterestingReflections.
• Sounds fun–I want to try it
• This will help students understand and critically engage with AI
• A good way to meet non-AI learning goals
• Intriguing; makes me want to learn and reflect
• I have questions or concerns about this reflection

​

• 7 minutes: Join a breakout group and share what you found that excites, intrigues, frustrates, or perplexes you,

An Invitation to Comment and Contribute

Comment at the bottom of each reflection (no account needed)

Contributing just involves a brief Google Form open to all teachers in higher education.

• What did you try? How did you teach about or incorporate AI? What was the goal? You can link to more information later. (Max: approximately 250 words)
• How did it go? Pros and Cons? Student reactions? What might you do next time? (Max: approximately 250 words)
• You can share links, images, and contact information if you like.
• We welcome reflections from those with extensive background and those just starting to think about AI.

Questions and Discussion

• What questions or comments do you have? Please share in the chat.
• How could we improve the user experience and the collection itself?
• What seems particularly useful?
• What might you contribute?
• What kinds of teaching reflections do you wish others would contribute?

​

​

There are many ways to participate!

Please consider sharing your comments or contributing your reflection.

​

​

Visit our sites at

exploringaipedagogy.hcommons.org

aiandwriting.hcommons.org

Slides for this presentation: https://bit.ly/ExploringAIPedagogyWebinar