Towards Transparency: How Can We Distinguish AI from Human Text Going Forward?

The Future of Writing: A Symposium for Teachers

University of Southern California

Anna Mills, May 1, 2023

Licensed CC BY NC 4.0

Welcome!

I like to approach this in a spirit of inquiry. It’s a complex topic, and I don’t claim to have the answers.�

Feel free to ask questions in the Q&A as we go, and I’ll try to pause periodically to respond.

​

Slides (open for commenting): https://bit.ly/TowardsTransparencyGPT�

What to expect

• Reasons to distinguish AI from human text
• How might we distinguish AI text without software?
• Pros and cons of classification software
• Possible ways forward

​

​

​

Presentation by Anna Mills, licensed CC BY NC 4.0.

Let’s take as a starting point the idea that language models like ChatGPT can now produce�passable prose of many kinds

Presentation by Anna Mills, licensed CC BY NC 4.0.

Are there settings where we want to know if text is AI-generated? (For me there are)

Presentation by Anna Mills, licensed CC BY NC 4.0.

Let’s think not just about student writing but about writing more broadly

• Higher ed discussions about distinguishing AI from human text have focused on academic integrity and picked up on polarized debates about plagiarism detection.
• Clearly, though, there are many other social contexts where merging AI and human writing will have consequences.
• What if we back up and look at AI vs. human text in society more broadly before deciding on an educational approach?

Presentation by Anna Mills, licensed CC BY NC 4.0.

We might want to determine which of the following categories each kind of text falls into, depending on the situation

• Not okay to auto-generate. Should be human written.
• Okay under some circumstances but I want to know it’s AI
• Okay to auto-generate (wholly or partly) but I want to know it’s AI
• Okay to auto-generate (wholly or partly), and I don’t need to know if it is AI.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Maybe knowing if text is AI generated is even a right? From the Blueprint for an AI Bill of Rights:

How important is it to know if the following kinds of text are AI generated?

Rate the different kinds on Mentimeter!

• Go to Menti.com
• Enter 6450 1889

Possible reasons to want to know

• To focus our energy on human communications
• To preserve the possibility of text being viewed as genuine human communication
• To screen out a flood of cheaply produced, highly customized AI content (spam, disinformation tailored to us, for example)
• To flag AI text for closer scrutiny for bias and accuracy
• To get more information about what shaped the text
• To recognize that unknown authors statistically averaged with a language model shaped the content
• To assess student learning

Presentation by Anna Mills, licensed CC BY NC 4.0.

Textpocalypse, anyone?

“What if, in the end, we are done in not by intercontinental ballistic missiles or climate change, not by microscopic pathogens or a mountain-size meteor, but by … text? Simple, plain, unadorned text, but in quantities so immense as to be all but unimaginable—a tsunami of text swept into a self-perpetuating cataract of content that makes it functionally impossible to reliably communicate in any digital setting?”

– Matthew Kirschenbaum, “Prepare for the Textpocalypse,” The Atlantic, March 8, 2023

Presentation by Anna Mills, licensed CC BY NC 4.0.

Kirschenbaum forsees “a debilitating amalgamation of human and machine authorship. ”

“Think of it as an ongoing planetary spam event, but unlike spam—for which we have more or less effective safeguards—there may prove to be no reliable way of flagging and filtering the next generation of machine-made text.”

– Matthew Kirschenbaum, “Prepare for the Textpocalypse,” The Atlantic, March 8, 2023

Presentation by Anna Mills, licensed CC BY NC 4.0.

Just today, AI scientist Geoffrey Hinton left Google warning of something like a “textpocalypse”

“"His immediate concern is that the internet will be flooded with false photos, videos and text, and the average person will “not be able to know what is true anymore.”"

“‘The Godfather of A.I.’ Leaves Google and Warns of Danger Ahead” The New York Times, May 1, 2023

Presentation by Anna Mills, licensed CC BY NC 4.0.

AI text is based on human sources that should be credited: “Data Dignity”

“At some point in the past, a real person created an illustration that was input as data into the model, and, in combination with contributions from other people, this was transformed into a fresh image. Big-model A.I. is made of people—and the way to open the black box is to reveal them.

This concept, which I’ve contributed to developing, is usually called “data dignity.” ”

Jaron Lanier, “There Is No AI,” The New Yorker, April 20, 2023

​

Presentation by Anna Mills, licensed CC BY NC 4.0.

Text helps us build a human relationship if we know it comes from a human

Presentation by Anna Mills, licensed CC BY NC 4.0.

For example, we may want to get to know someone via text

Presentation by Anna Mills, licensed CC BY NC 4.0.

If we value writing as a way to assess student learning, we will want to distinguish AI text from student writing

Even with ungrading approaches, we are assessing some baseline competence necessary for passing

Presentation by Anna Mills, licensed CC BY NC 4.0.

Doesn’t “embracing” student use of AI text generation also require some non-AI student work we can assess?

• ChatGPT on ChatGPT might seem unlikely now, but when it’s as ubiquitous as autocomplete or Grammarly?
• How would we even assess whether students are learning how to work with AI if we don’t know whether the prompts and changes to the AI output students turn in are themselves AI?

​

Presentation by Anna Mills, licensed CC BY NC 4.0.

I designed a module on Canvas Commons that asks students to reflect on the differences between ChatGPT’s critical assessment and a human-written assessment

Presentation by Anna Mills, licensed CC BY NC 4.0.

A Canvas Commons module

But anyone could use ChatGPT to comment on ChatGPT output.

If you (always or sometimes) want to know if a text was AI-generated, why? What reason or reasons do you find most compelling?

• Go to Menti.com
• Enter 1531 7593

Presentation by Anna Mills, licensed CC BY NC 4.0.

If we do want to distinguish AI from human text, at least in some circumstances, how might we do it?

Some consider this futile, but let’s consider

Dr. Sarah Elaine Eaton:

“Hybrid writing…will be the norm. Trying to determine where the human ends and where the artificial intelligence begins is pointless and futile.”

“6 Tenets of Postplagiarism: Writing in the Age of Artificial Intelligence”

Presentation by Anna Mills, licensed CC BY NC 4.0.

• Maybe we can tell if the text is AI?
• Maybe we can decide to trust each other to label AI text?
• Maybe we can have people write without access to AI?

Maybe we can tell if the text is AI? (Use our noses?)

Twitter thread from Lauren Goodlad of Critical AI

Presentation by Anna Mills, licensed CC BY NC 4.0.

What would be the giveaways?

Twitter exchange between Lauren Goodlad and Maha Bali

I’m skeptical. How many of us can tell, how much of the time, and for how long?

I’m not as confident as Maha Bali and Lauren Goodlad that I can tell. Sometimes student writing has formulaic features. And ChatGPT can be “voicier.”

Can we be sure language models will stay mostly generic, bland, superficial in their responses?

Can we be sure they won’t be able to imitate individual styles and levels of proficiency with Standard English convincingly?

Presentation by Anna Mills, licensed CC BY NC 4.0.

Trust- and motivation-

based approaches

Let’s go back to why we assign writing. Not for the product. For the thinking process.

“A fundamental tenet of Writing Across the Curriculum is that writing is a mode of learning. Students develop understanding and insights through the act of writing. Rather than writing simply being a matter of presenting existing information or furnishing products for the purpose of testing or grading, writing is a fundamental means to create deep learning and foster cognitive development.”

-- The Association for Writing Across the Curriculum

Presentation by Anna Mills, licensed CC BY NC 4.0.

First and foremost, emphasize purpose and engagement

• Emphasize how writing helps us think and form our own ideas and voice.
• If students see meaning in the writing assignment and understand what they will get out of wrestling with it, they are more likely not to resort to a text generator.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Discuss the ethics of transparency with students

• Have an open discussion at the beginning of the course–when do we need to know how AI was used in writing? Come up with examples of when it matters, when it doesn’t.
• Seek student input or collaboration on AI policy formation
• What do students think will deter misuse of AI?
• What values do students see as important here?

​

Presentation by Anna Mills, licensed CC BY NC 4.0.

Maha Bali, Agentic and Equitable Educational Development For a Potential Postplagiarism Era

“I also feel like our relationship with AI will become entangled and complex, where it will be difficult to determine where our agency begins and where the machine’s influence on us is stronger – but I sense that it is important for students’ metacognition that they remain aware of how they’ve integrated AI into their thinking��… this is why I suggest an approach of “transparency“, not just of students disclosing where they got some ideas/text from in the sense of attribution, but also of reflecting on where they used AI in their process and why that was helpful or how it needed tweaking, etc..”

Presentation by Anna Mills, licensed CC BY NC 4.0.

Trust may be one good path and may suffice in many situations

But will it in others? In some settings, incentives and competitive pressures will be strong.

​

• Customized phishing
• Disinformation campaigns: blogs, social media posts
• Advertising: product reviews

Presentation by Anna Mills, licensed CC BY NC 4.0.

Restrictions on access to AI during writing. Even if these would fix the problem in education, they wouldn’t work around text in other settings.

• Proctored in-class writing? Maybe sometimes, especially for formative purposes, but this doesn’t give enough time to develop writing.
• Writing by hand? Maybe occasionally, but not for summative assessment. Seems too far removed from common practice. Anxiety and disability barriers.
• Surveillance software? Disturbing. Likely inaccurate and biased against marginalized people.
• Blocking particular AI software on a network? Ineffective

​

Presentation by Anna Mills, licensed CC BY NC 4.0.

AI Text Classification

Detectors: How good are they?

Q: Can software check if text is AI-generated?

A: There is software designed to classify text as “likely AI” or not.�

• However, it is not reliable.
• And it is not expected to become reliable.

Presentation by Anna Mills, licensed CC BY NC 4.0.

First, a caution: ChatGPT doesn’t “know” if it wrote something!

Don’t paste student work into ChatGPT and ask, “Did you write this?”

• First, it’s not legal to share student work without permission.
• ChatGPT may well give an answer. The answer might be right, but it might be wrong.
• ChatGPT has not been tested for this purpose. (OpenAI has a different AI text classifier system.)
• ChatGPT has no access to a database of its own outputs to check against.

Presentation by Anna Mills, licensed CC BY NC 4.0.

How does AI detection compare to (traditional) plagiarism detection?

Differences

• No access to a database of AI text for direct comparison (unless companies create and share such)
• Detection is based not on direct similarity but on probabilistic guesses based on features of the text
• ChatGPT and other AI tools can tweak the text to evade detection

Presentation by Anna Mills, licensed CC BY NC 4.0.

100% reliable AI detection? Not likely

“We’re extremely unlikely to ever get a tool that can spot AI-generated text with 100% certainty. It’s really hard to detect AI-generated text because the whole point of AI language models is to generate fluent and human-seeming text….”

– “Why detecting AI-generated text is so difficult (and what to do about it)” by Melissa Heikkilä, MIT Technology Review, February 7, 2023

Presentation by Anna Mills, licensed CC BY NC 4.0.

AI Text Detection Software

• TurnItIn’s detector
• OpenAI’s AI text classifier
• Crossplag AI content detector
• CheckGPT (Packback)
• ZeroGPT
• GPTZero
• OpenAI’s older detector
• AIWritingCheck.org

Presentation by Anna Mills, licensed CC BY NC 4.0.

Turnitin claims a “less than 1/100 false positive rate,” but is that accurate?

• “Our AI writing preview has been trained on academic writing with high efficacy rates and can identify 97% of AI writing”
• But the company has shared no data at all, let alone external peer reviewed studies
• They didn’t test their system on the most sophisticated, recent AI software, ChatGPT running GPT-4.

Presentation by Anna Mills, licensed CC BY NC 4.0.

The Washington Post found an example of a false positive

GPTZero labeled the Bill of Rights “likely AI”

On April 17, 2023, I retested a popular Reddit experiment using GPTZero.me.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Caution: false positives = false accusations?

• All these tools currently may get it wrong: sometimes they label human text as “likely AI” and sometimes they label AI text as likely human.

​

• This is not likely to change. I have not heard any experts optimistic about these tools ever being able to eliminate false positives, though the rates could well improve.

Presentation by Anna Mills, licensed CC BY NC 4.0.

How will students feel when we tell them their writing may be wrongly flagged as likely AI?

To help students prepare for the possibility of false positives, Turnitin advises “Establish your voice: Make sure that your writing style and voice are evident to your instructor.”�

• But is it fair to ask students to write in such a way that proves their humanity?
• How will the student know what this means, how to do it, and how to make sure that the instructor will perceive it as their voice?
• As AI gets “voicier,” this may not work.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Caution: Is detection a good use of money?

• Turnitin’s detector is free for now,

BUT

• “Beginning January 1, 2024, only customers licensing Originality or TFS with Originality will have access to the full AI writing detection experience”--Turnitin

Presentation by Anna Mills, licensed CC BY NC 4.0.

Even according to Turnitin, detectors will often be inconclusive

• “Rely on relationships with the student: This kind of judgment should never be made without a respectful dialogue with the student.”�
• “After review, if the evidence isn’t clear, give the student the benefit of the doubt. All the right conversations have taken place, with all the right questions asked, and there’s still uncertainty, the student cannot be penalized based on that.” –Turnitin�
• My note: Certainly, this is time intensive and raises equity concerns about how the judgments will be made and which students will be affected.

Presentation by Anna Mills, licensed CC BY NC 4.0.

It’s currently simple for students to get around detection with paraphrase software like Quillbot.�

From the Washington Post on Turnitin’s AI detector: “It couldn’t spot the ChatGPT in papers we ran through Quillbot, a paraphrasing program that remixes sentences.”

Presentation by Anna Mills, licensed CC BY NC 4.0.

Free software explicitly designed to get around AI detectors

Presentation by Anna Mills, licensed CC BY NC 4.0.

Still, I’m not convinced we should give up on detection. Others disagree.

Presentation by Anna Mills, licensed CC BY NC 4.0.

I’m thinking of the situations in which we want to know if text is human or AI. I’m thinking of the limits on trust, restricting AI, and guessing, in education and beyond.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Researchers see several ways forward for improving AI text classification

• Language models could watermark text outputs (choosing certain words over others according to recognizable statistical patterns) in ways that are not noticeable to readers but can be detected by software.
• Models could keep a database of outputs to certify whether a particular text sequence has ever been auto-generated.
• Models themselves can be used to determine whether paraphrased text is semantically the same as a AI-generated output.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Researchers from UMass Amherst and Google suggest that language models could keep a record of what they generate. They envision an API that “stores every sequence generated by their LLM in a database” and “offers an interface that allows users to enter candidate AI-generated text as a query.” A language model is used to check paraphrased text against the database of outputs.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Turnitin is developing paraphrase detection

From Turnitin’s FAQ:

“Turnitin has been working on building paraphrase detection capabilities – ability to detect when students have paraphrased content either with the help of paraphrasing tools or re-written it themselves…We have plans for a beta release in 2023…for an additional cost.”

Presentation by Anna Mills, licensed CC BY NC 4.0.

Some students want teachers and students to use detectors. They don’t want to be at a competitive disadvantage.

Boston University students in a data science class with Professor Wesley Wildman developed a policy that allows different options for AI use but involves detectors extensively.

​

https://www.bu.edu/articles/2023/student-designed-policy-on-use-of-generative-ai-adopted-by-data-sciences-faculty/

Presentation by Anna Mills, licensed CC BY NC 4.0.

From the student-

developed policy adopted by the Data Sciences Department at Boston University

• “Students shall…Employ AI detection tools and originality checks prior to submission, ensuring that their submitted work is not mistakenly flagged.”
• “Instructors shall…Employ AI detection tools to evaluate the degree to which AI tools have likely been employed.”

Presentation by Anna Mills, licensed CC BY NC 4.0.

Could student desire for detectors just be evidence of the pressures on them/identification with oppressive power structures they’ve learned?

My response: Perhaps, but we should still listen and respond. It seems condescending to dismiss a desire for classification software as false consciousness.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Detection as deterrent: �Point out that what’s generated by AI might be labeled as AI, sooner or later

• No one should assume that AI text is undetectable.
• As the software evolves, what’s not detectable now might become retroactively detectable.
• Remind students that their next teacher or school or workplace might use detection software.
• Share how we use AI and label it and any stories of being tempted to use it without labeling.

Presentation by Anna Mills, licensed CC BY NC 4.0.

What if everyone, including students, had access to classification software, free?

• Organizations could use it to (imperfectly) filter out AI content masquerading as human (editorial submissions, spam, reviews, social media, disinformation, manipulation).
• When in doubt about a communication, individuals could get an estimate of how likely it is to be AI.
• Students could get reassurance that their own writing is not being flagged as AI (Turnitin does not allow teachers or instititions to see their own writing’s AI likelihood score before submitting).

Presentation by Anna Mills, licensed CC BY NC 4.0.

Advocate for government support for AI text classification as a social good

If transparency is a social good in education and in other areas, let’s advocate for a government role in promoting human-nonhuman distinction to protect human communication

• Regulation might require companies running LLMs to keep a record of outputs
• Governments might offer incentives or prizes for advances in classification techniques
• Regulation could ban software and marketing designed to help people pass off AI text as human (like undetectable.ai) 

That’s my current position… What is yours?

While I don’t think we should use classification software punitively, I do think we should pursue it.

If used cautiously, it can help us maintain space for the human in textual communication.

Its existence can help promote

• awareness of our own agency as writers
• reflection on how we are using AI and on the provenance of the AI text.

Presentation by Anna Mills, licensed CC BY NC 4.0.

Presentation by Anna Mills, licensed CC BY NC 4.0.

• More resources on AI text classification from my WAC Clearinghouse list

Articles about AI text detection especially in higher ed

What are your thoughts on whether and how we might move towards transparency around AI text?��Comments? Questions?

�Anna Mills�armills@marin.edu, @EnglishOER

Slides: https://bit.ly/TowardsTransparencyGPT��This presentation is shared under a CC BY NC 4.0 license.