Overview

These materials provide students with sentence starters to help them iteratively prompt, interrogate, and engage with generative AI output.

What Students Need to Know

You should be familiar with the general background information on generative AI. You should also be familiar with the inherent limitations and biases of generative AI.

Remember:

Do NOT put sensitive personal information or confidential information into generative AI.
Hallucinations: Chatbots may repeat untruths, or they can make things up. You are responsible for the accuracy of anything you incorporate.

Learning Goals

AI: Students will be able to iteratively prompt and interrogate generative AI to effectively refine generated output to meet their goals and keep the human in charge of the interaction.

Instructions

Generative AI is used most effectively if you engage with it conversationally and iteratively. This means that after you give it the first instruction and receive its output, it’s your turn to ask the AI questions (see the box for sample phrases and PAIRR guidance), make it clarify anything that was unclear, expand on sections, or revise parts of the output. You can do this as much as you want, AI has infinite patience and you can ask for as many examples as you want.

Laura Yost calls this “Guerrilla Prompting” -- “the strategic and iterative practice of challenging generative AI output through thoughtful follow-up questions that confront LLM assumptions, biases, oversights, contradictions, errors, and shortcomings.” We call it “Being the Boss” -- making sure that the AI gives the human the output the human wants and needs. And sometimes you can even have the AI criticize itself, try a prompt like “What weakness would a reviewer find in this proposed language?,” “What is missing from this output?” or “What would be the best counterargument to this claim?” See Yost’s blog for an example of this kind of iterative, active prompting.

Have you noticed that generative AI always wants to be your buddy? The media has noticed an increased “sycophancy.” So it can be really hard to get useful, constructive criticism. One of the ways that can work for this is to give the AI a persona, a harsh critic, a social media troll, Reviewer #2 (ask your favorite academic). Sarah Senk & Taiyo Inoue have a great blog on this (and the end video is worth a watch).

Here are some phrases that might be useful:

What do you mean by …
Explain [this] in more detail …
Can you explain more about…
Can you explain again, using other words?
Explain [this] as if I’m [in 8th grade, new to engineering, etc.]
Can you provide an example of …
Do you have any suggestions for how to …
Would this suggestion still be accurate if …
How does this suggestion work with your other suggestion to …
I am confused about what you said about [describe the part of the generated output that is confusing to you. You can quote directly from the output or summarize the part you have selected, as long as you clarify which part you are confused about.]
If you think the suggestion missed something already in the original text:

I thought I already implemented this suggestion when I said “[quote the part of the text that you believe addresses the suggestion]”. Do I still need to change the text to implement this suggestion? If I do still need to revise, be specific in explaining why what I wrote does not address the suggestion.

If you think the information in the suggestion is factually incorrect or inaccurate:

Wouldn’t [describe the information in the suggestion that you believe is factually incorrect] be incorrect? If this is actually correct, please explain in detail why the information is true.

If you think the suggestion is biased or relies on biased information:

This suggestion sounds biased because [describe why you believe the suggestion is biased. Does it seem to be skewed toward one perspective, or excluding other perspectives? Does the suggestion miss out on some important context?] . Explain your suggestion in more detail, taking into consideration the perspectives of [describe the perspectives you believe the suggestion misses or excludes] .

If you are worried that implementing the suggestion could negatively impact another aspect of your writing:

If I implement this suggestion, wouldn’t this affect my writing by [describe the concern you have about possible consequences for implementing the suggestion. Are you concerned that this would make the text too long, or not concise enough? Are you worried that this would be redundant with another part or your writing? Describing your worry in detail will help the AI tailor or explain the suggestion to fit your writing goals and concerns.] ?

After you have asked the generative AI tool for these explanations, look over the explanations and decide to accept or reject the suggestion provided. Remember, sometimes the AI is wrong and you are ultimately the author of the text.

Activity

Complete the following activity and turn it in as directed by your instructor.

Using any generative AI tool you’d like (e.g., ZotGPT), create a very simple, basic prompt on any question of your choice (e.g., “What are three ways generative AI can improve my writing?”).
Read the tool’s output for clarity and accuracy. Prompt the tool to revise the output in one way that will improve the response (e.g., use your own words or one of the suggestions above to correct something that was wrong or add something that was missing--or try to get it to critique using a fun persona).
Take a screenshot of-- or copy and paste-- (a) your original prompt, (b) the initial output by the tool, (c) your follow up prompt, and (d) the revised output, into a document with your name, class section, and a couple of sentences with your thoughts on the ways in which you improved the output and how whether the original or revised version was more useful.

tt/6/21/24