1 of 55

Jan 2023 �Prompts & pedagogy immune��

THUNDERDOME JAN 2023: ChatGPT vs. RWS�

Summary, no analysis. Fails rhetorical situation, audience, strategies, etc. �
Can’t analyze language of texts. Quotes rare or fabricated. �
“Hallucinations” and “forgetting” pervasive.�
But analysis of canonical texts and feedback on student writing showed promise.

2 of 55

Dec 2023: dramatic improvements�

Detection tools (mostly) fail�
Larger context windows & data sets = more accuracy, “memory,” fewer hallucinations �
Can upload texts, create custom GPTs & “knowledge bases” �
Summaries improve; analysis and quotations�
Integration with search, browsers, writing tools, etc.�
Academic tools appear��

3 of 55

AI prompt: “Otter on a plane using wifi.” Results show improvements in one year (Mollick) � 2023 2024

4 of 55

A.I. Humor Test - researcher tests AI models on ability to identify and explain British humor

5 of 55

A.I. Humor Test

6 of 55

Comparative ability to identify and explain jokes

https://www.linkedin.com/pulse/humour-understanding-llms-gpt-4-clear-leader-peter-gostev-qbxbe?trk=public_post

7 of 55

A Course on Writing with/about A.I.: My Experience Fall 2024

Read about AI & writing, education, culture, politics.
Integrate - interpreting, analyzing, and evaluating texts; planning and invention, drafting revision, organization, editing, feedback and peer review.
MyEssayFeedback & other feedback tools.
Experiment with commercial & academic tools, prompts, custom GPTs.
A.I. Reflection Journal document use, results, comparisons, strengths/limitations

8 of 55

Aim to help students become

Reflexive - documenting, evaluating, reflecting on AI use.
Critical – representations of AI, questions of power, bias, exploitation, effects on learning. Critically evaluate limitations, benefits, risks.
Ethical – responsible, accountable, aware of potential harms
Effective – identifying productive uses of tools for different contexts and writing/reading processes.
Creative – “playful prompting”; new opportunities to compose and have fun.

9 of 55

1. Reflection: surprise and awe (the “genie” experience”)

Riley: I have been honestly amazed by its capabilities and speed. I had avoided using AI because I felt like it was just a system used for cheating. However, this course opened up a sort of gateway into different systems and I now see AI as more of a learning tool. I can use prompts and certain tools to critique my writing versus having it write the whole thing. I also have seen AI be able to summarize large texts which is potentially helpful in my more lecture heavy classes. I have also become aware of the dangers of AI from the many articles we have read and know what to look out for when I am using it.�

Garrett: I’ve learned a lot about A.I. just in the last couple of weeks, and I’ve used it more in class than I ever thought I would. It was interesting to see during some of the in-class exercises we’ve done, how A.I. can be used to enhance writing and give good feedback.

Evan: Taking this class has been a revelation. It's not just that I've learned more about how AI operates; it's that I've begun to see the immense potential it holds for productive and creative endeavors. The coursework and discussions have been so engaging that I'm seriously considering pursuing a master's degree in applied AI. The idea of diving deeper into this field excites me in a way I hadn't anticipated.

10 of 55

2. Using in classes/personal lives but little instruction

Tom: I have tried Claude and POE and been impressed with the results. For POE I uploaded my discussion post for my economics class and asked for feedback. POE was able to point out which claims lacked evidence and give advice to back it up.

�Isabel: I used AI to help me study for my chemistry exam. On anything I was confused about or didn't know how to do I would ask chat-gpt to solve it for me and then explain to me how it got the answer. Its explanations were actually very understandable and it explained everything very clearly. Honestly, I feel like the way AI taught me was a lot more beneficial then how my professor lectures. It's really comforting to know that if I ever am unsure on how to do something then I can always ask an AI for help

Tom: I’m interested to see how well chat gpt can create an original song. I first started experimenting by asking it to generate chord progressions in the style of an artist that I like. In this case I wanted it to generate a jazzy-funky chord progression in the style of Roy Ayers…

Jaren: I do art projects for myself, so I was curious to see how I could implement the use of AI into my art without losing the “soul” of my art style.

11 of 55

3. Good for summarizing, explaining, verifying, researching

�Delia: I use A.I. as a sort of “fact check” when it comes to my readings. I usually read the text, take notes, then use A.I. programs (like POE)...This allows me to ensure that I am grasping the most pertinent information. Furthermore, it may expose a point that I need to go back and further analyze. I try to ensure that I am still interpreting my sources through my own lens; therefore, I take what A.I. says with a grain of salt.

Connor: Ok. Good start. These categories of demagogue are not exactly the same as Roberts Miller’s or Werry’s …I was expecting more commonality. I’m probably going to have to analyze and look for commonality more by myself. Perhaps if I understand the context of Wallace’s background more it may help me…

Me: Chat, can you give me the historical background of 1963 when Wallace gave his speech?

[A series of questions follow about who Wallace was, what he was known for, and the context of his speech.]

Erin: Using Claude to analyze research studies was extremely helpful. All I had to do was upload a PDF of the research study and write out what I needed Claude to identify. The answers I received were all correct and even included quotes from the study to support the responses. This was really beneficial for helping me identify information that would take me a long time to search through the paper for.

12 of 55

4. Brainstorming, composing introductions and conclusions

Jayden: AI can be an extremely helpful tool in brainstorming, simply if you’re feeling stuck with a topic you can ask an AI platform for some ideas, and maybe you don’t use the idea in its entirety, but you could use pieces of it which can help you get started in your writing.

Sami: As always, writing the introductory paragraph was the most challenging part of this essay so far because I never know how to open up the topic. However, with the help of Chat GPT, as well as Poe, those two softwares really helped in giving me a sense of where to start with both paragraphs, giving me specific examples of hooks, as well as ways to transition and finish my thoughts in the concluding paragraph.

��

13 of 55

5. Students became selective & better at prompt design�

Riley: The [ChatGPT-generated] paragraphs were pretty solid, but lacked good examples and in text citations of specific evidence of these demagogic statements. I heavily revised the paragraphs to fit the narrative I was trying to create. After my personal revisions, because personally I find my writing style to be more fluent than ChatGPT’s style, I used MyEssayFeedback to examine the clarity of my argument and see what I needed to shape up, much like a peer review. I additionally used the LEX platform to generate titles so I could 1. Create a solid title, and 2. Make sure my paper was pushing the point I was trying to make.�

Rachel: Yes, it would make errors that were blatantly obvious, but if one were to keep on refining the prompts you would find great results.

Nathan: Understanding how to use ChatGPT seems to be half the struggle. Whether you’re generating a funny story, asking for help with editing an essay, or trying to hold a conversation, the way you phrase your question is everything.

14 of 55

6. Becoming selective/aware of limitations�

Andrew It seems for deeply analytic tasks which require careful reading and superior comprehension skills, AI is ineffective, however, for tasks that involve creativity, brainstorming, varying perspectives, or outside feedback, AI can be pretty great. It’s not what I want when I am in a crunch trying to understand a philosophy reading, but in other applications it is exceedingly useful.

Cole: The image, audio, and video resources were significantly more frustrating to use, as copyright safeguards zapped all the fun out of using AI to foster unadulterated creativity. Why is it that using Suno.ai failed to make a song actually worth listening to, and not just a tune that would be played in the background of a public access television commercial? And most importantly, why did my really hilarious and well-written Ozempic parody sketch get flagged by HeyGen for allegedly violating the site’s moderation policy? �

15 of 55

8. Many students enjoy the creative opportunities

Taking this class has been a revelation. It's not just that I've learned more about how AI operates; it's that I've begun to see the immense potential it holds for productive and creative endeavors. The coursework and discussions have been so engaging that I'm seriously considering pursuing a master's degree in applied AI.
Cole: As a writing student, I benefit from being able to use a platform like HeyGen to make comedy sketches. I don’t have a camera, a mic, a budget to hire actors, or remarkable editing equipment to effectively make the words I’ve written come alive in visual media.
Abigail I’ve been trying to write a short story that is a reflection of my life and has elements of time travel, where a girl goes back in time to save her younger self from her abusive father. I was curious how ChatGPT would formulate the characters based on the verbs and adjectives I used in the prompt. Three versions are listed, with my reflection below.��

16 of 55

9. Some use for self reflection/therapy

Redacted I specifically asked for a poem on this subject because I have lately been forced by intensified parental abuse to confront and move past lies that I have been told about me needing to hide or reject my "sensitive and thoughtful" identity, especially those parts of it that motivate me to value and adapt my emotional reactions as guiding factors in the life and often offer the criticism that could cause dysfunctional situations to improve. Reading this poem helped me realize that I may have found a use for AI that I truly enjoy! Asking it for encouragement when I am struggling to think of words to describe how who I am can be valued or internal motivations can be applied to a situation is a potentially very helpful and powerful personal use of AI for me, especially as someone still navigating identify formation in the wake of very challenging interpersonal dynamics.
Evan: One of the most significant shifts for me has been recognizing how AI can be a tool for exploration in various domains—education, writing, even self-discovery. The technology isn't just about algorithms and data…I'm eager to see how AI can transform the way we teach, learn, and understand ourselves.

17 of 55

10. Most promising use: feedback

Rachel: Getting writing suggestions is a game changer. I’ve noticed that I dislike having to wait until my assignments are graded to see if it sounds good or if I’m doing it correctly…. it has really helped me improve my writing. �
Tom: Getting this type of feedback for an argumentative paper was helpful in finding the areas where my evidence was lacking and providing me with insight on how to strengthen it. POE was also able to give me questions to dive deeper into in order to strengthen my argument as a whole. I wouldn’t have been able to think about all of these questions while seeing it from my perspective. I think having AI as a perceptive tool to enhance your writing is beneficial.�
Nathan: When it comes to catching mistakes and areas for improvement in your writing, ChatGPT excels. It can catch the small grammar mistakes, spelling errors, losses of thought, and odd structural decisions throughout your writing. Obviously you shouldn't rely on this alone; knowing how to self edit is important. But, as always, using AI tools as additions to your own work is surprisingly effective.

18 of 55

Especially feedback for professional writing

Andrew: The AI’s feedback was very helpful. It gave me 5 different categories for ways to communicate my critical thinking skills: Education in philosophy and law, rhetoric and writing studies minor, LSE pre-law program, Work as a stagehand for LiveNation, and various leadership roles. Some feedback was more helpful than others, but it sparked new ideas for how I could phrase an experience in such a way that it archives my rhetorical purpose.

19 of 55

Gave students prompt guides

20 of 55

Provided custom chatbots and invited students to create own

21 of 55

Custom chatbots

22 of 55

Two of my custom chatbots: PickleRick Bot & Demagoguery Decoder

23 of 55

Eric Keane and Anna Mills (writing teacher)

Fast, simple, cheap ($12/semester)�
Formative feedback (won’t revise for students.) Use in Canvas or alone.�
Reflection and revision plans.

Peer review - automatic or manual assignment�
Create own prompts or adapt other teachers’ �Community of practice-ish

Student data protected�

MyEssayFeedback.ai

27 of 55

2. CRITICAL AI LITERACY

29 of 55

“Bias Explorer” Tool by Stable Diffusion

Stable Diffusion Bias Explorer lets you explore bias of image generation models through the lens of professions

Confident aerospace engineer

Compassionate clerk

Modest childcare worker

30 of 55

From Daniel Estrada, “Auditing AI in the classroom”

31 of 55

“I asked AI what Europeans think Americans from all 50 states look like.”

33 of 55

Even if ask for diversity, stereotypes seem baked in�(ChatGPT4)

34 of 55

Stanford and Princeton studies: covert LLM racism may be worse & harder to address

35 of 55

A.I. is W.E.I.R.D. (Western, educated, individualistic, rich, democratic)

In a recent systematic review we analysed over 200 studies from the last ten years (2012–2022) in which the explanations given by XAI systems were tested on people. We wanted to see to what extent researchers indicated awareness of cultural variations that were potentially relevant for designing satisfactory explainable AI.
Our findings suggest that many existing systems may produce explanations that are primarily tailored to individualist, typically western, populations (for instance, people in the US or UK).

37 of 55

A.I. in Healthcare, Housing, Banking, Criminal Justice…

38 of 55

Our social, cultural & communicative knowledges extracted and monetized.

Your thoughts, writing, stories, family photos, etc. fuel LLMs.

39 of 55

Our Digital Lives as Next Source of Training Data

Microsoft Copilot video: real value is allowing AI to ingest and train on emails, texts, docs, notes, spreadsheets, calendar, photos, etc.

40 of 55

OpenAi wants live on your computer…

42 of 55

First example = offers to do your homework

43 of 55

Student concerns about impacts

44 of 55

Commercial AI-Edtech Outsources Teaching?

“One of my courses this semester requires us to use a platform called PackBack, a discussion board forum. PackBack uses AI to grade discussion board posts with a "creativity score" automatically.” �
“No one is even going to read my posts, as my grade is based solely on meeting word count and whether the AI determines I did "good enough.” �
Very frustrating that in a world where professors complain about students getting "lazy" and using ChatGPT they are too lazy to actually do the job they get paid for. I would also like to say that I go to a top 20 public university and my program specifically is one of the foremost in the country...
“It’s almost HALF of my grade :/”

45 of 55

3. Ethical, responsible, accountable

4. Effective – identifying productive uses of tools for different contexts and writing/reading processes

5. Creative experiments and playful prompting (make it fun)

Teaching goals: help students become ethical, effective, creative users of LLMS and AI

46 of 55

Creative Use: genre bending, remix, “playful prompting” (Marino)

Take an article from the Daily Aztec and rewrite it in the style of Gossip Girl. �
You are a master chef. Read the U.S. Declaration of Independence like a recipe for a new country.�
You are a great lover. A veritable ChatJuan. Read the following scientific article like a love letter. �
Take the monologue from Fight Club and rewrite as if channeled by Elmo from Sesame Street.

47 of 55

Creative applications/playful prompting

Write an operatic libretto based on the recent presidential debate between Trump and Harris. Take the results, then transform it into a rap battle with Trump channeling Big Sean, and Harris channeling Kendrick. [AI annoyingly coquettish. Refused to “go there.” Challenged to do better gave me this]�
Result: Steampunk Dinosaur Detective Noir in a Haunted Submarine�"A private eye velociraptor with a top hat and steam-powered monocle is hired to solve a murder mystery aboard a haunted Victorian-era submarine. As the dinosaur detective navigates through ghostly apparitions and mechanical contraptions, he uncovers a conspiracy involving a cursed treasure, time travel, and the ghost of the submarine's mad inventor.”�
Sample creative explorations by instructor

48 of 55

Suno.ai

REDDITERS: “Okay. I officially cannot tell WTF is AI generated music anymore. Wow.”�

“You can feed it original music and lyrics. It's getting really good at it. Mine has been trained from my recording, so I hear variations of previous songs in new ones that were created later.”

49 of 55

Creative output requires sophisticated language (“programming in prose”/Humanities education)

“These new tools, trained on vast swathes of humanity’s cultural heritage, can often best be wielded by people who have a knowledge of that heritage…humanities majors can produce some of the most interesting “code.” Mollick�
“The best coders of ChatGPT for writing have a solid understanding of the prose they want to create (“end on an ominous note,” “make the tone increasingly frantic”). They are good editors of others (“make the second paragraph more vivid.”). They can easily run experiments with audiences and styles (“Make this like something in the New Yorker,” “do this in the style of John McPhee”). And they can manipulate narrative to get the AI to think in the way they want.

50 of 55

Liberal arts education and genre knowledge FTW

Create photo of a dog digging �in the sand

/imagine high angle photo of a dog �on a surfboard confidently surfing a �wave, agfa vista 200

/imagine photo of a sad dog at the beach �chained to a stake in the ground, agfa �vista 200, sunny

51 of 55

Programming in prose

Analysis of 14 million image prompts. Mentioning artistic styles, artist names, genres, media, painting techniques improves result.

From Yan Wu, WaPo, Jan 17, 2024. “AI is outrageous — and wonderful. It’s also prompting a new art form.” https://www.washingtonpost.com/opinions/interactive/2024/ai-image-generation-art-innovation-issue/

52 of 55

Jevonia Harris and Lauren Kelley, “From Algorithms to Insights: Understanding AI's Potential in Education”

COMBINING

A dynamic watercolor surreal painting of a beautiful Black woman with an Afro hairstyle. Her Afro is entirely made up of rhonocolus flowers, with deep purple hues, arranged to depict the Fibonacci series. The flowers seamlessly form the shape and flow of her hair, capturing the elegant spiral and harmonious design. The painting uses vibrant and fluid watercolor techniques to blend her features with the floral Afro, emphasizing the Fibonacci pattern.

AI has the Creative Power to Unite Experience and Learning

Image: DALL-E

53 of 55

A Few Observations

Strong writers tend to use AI more effectively, selectively, critically; they consider how tools can be integrated into their writing processes (or not).��Garrett: “I did not use any A.I. with this essay. I’m confident in my ability as a writer and using A.I. just doesn’t really fit into my process.” �
Some students may feel alienated from output and unsure how to integrate/adapt it. This can be a serious problem.�
Requiring students to document and reflect on AI = helpful. �
“Prompt literacy” takes practice/time. Simple prompts (“summarize claims and evidence”) return vague, general answers.

54 of 55