CC by SA 3.0, Jens Mohr

Undeclared AI-generated text in Wikipedia: �A tale of caution

Mathias Schindler

Wikipedian from Germany

Jambo!�Hello!�Guten Tag!

• Wikipedia autor since 2003
• Co-Founder of Wikimedia Germany
• Board member from 2004 to 2006
• Wikimedia Germany employee from 2010 to 2014
• I now work for a tech company (we use AI)
• This presentation is unrelated to my current employment, I am here as a Wikipedian

About me

0. Prologue

1. This is a story about AI

• References in Wikipedia are important
• Printed literature often has ISBN, ISSN or DOI identifiers
• MediaWiki handles ISBN references

ISBN 10 and ISBN 13 have a checksum at the last digit. A single wrong or flipped number will cause the checksum test to fail

ISBN’s coolest feature: Checksums

No rocket science

Just download the Wikipedia dump

Write a parser for the article section and scan for ISBN strings or ISBN templates

extract the ISBN

Calculate the checksum

So I wrote a Checksum checker

• Wrong ISBNs because of mistakes, missing or additional numbers
• Wrong ISBNs because the publisher messed up

What I hoped to find

• Wrong ISBNs because of mistakes, missing or additional numbers
• Wrong ISBNs because the publisher messed up
• Something else

What I found

• Articles with two or more Checksum errors in ISBN references
• Book titles and author names sounded familiar and plausible given the topic of the article
• I could not find the literature in any catalogue

The reference section was hallucinated, none of the literature did exist (in this universe, at least)

The articles were in part of fully ChatGPT generated, ChatGPT had added a list of references that made it look like supporting literature did exist.

Books �that do not exist

2. This is a story about AI and humans

What is at stake here?

Checksum errors are not the main issue, they are merely a symptom of the current shortcomings of LLMs.

Forged literature references are not the only issue.

The core issue is the contamination of Wikipedia with anti-knowledge. Text that “could be true”, that camouflages as information.

There is a deeper, darker irony here about LLMs poisoning the very well they intend on drinking from

Articles created by LLMs will most likely be unmaintainable and serve as a justification for further deterioration

• As far as I can tell, there is one (in dewp at least)
• Use of LLM for article creation is not permitted
• Use of LLM for article creation is considered vandalism
• These users will be sanctioned, including banned
• Articles will be deleted or reverted

Is there a consensus?

• Maybe a bit
• There is more evidence about ChatGPT generated articles
• Effort for cleanup exceeds effort to contaminate Wikipedia
• ChatGPT and other tools are gaining functions, including features that actually search the web for linkable content (sometimes, based on subscription level)

Does it work?

Ignorance

Neutral Working theory: People see LLMs as a tool to compensate for missing skills or at least to masquerade it enough.

Assume good Faith Working theory: People don’t know and/or don’t care how LLMs work and genuinely believe that they are contributing to Wikipedia in a positive manner

Cautious working theory: People see LLMs as a tool to change Wikipedia according to their agenda, using brute force and overpowering numbers with machine help

What other scenarios are conceivable? Is there a motivation possible?

Just one question: Why?

Inability

Agenda

Something else?

Show me �the prompt, �please.

3. This is a story about humans

I could not have done it without the help of AI

In this case, the code was written with Claude and later rewritten in VS Code and Copilot with Claude

There is a beautiful metaphor in there about fighting AI generated text with AI generated code

Remember the checksum checker?

It is entirely possible that some of the shortcomings of current models will disappear or be mitigated

Maybe LLMs will start to hallucinate ISBNs with correct checksums

There are some rather beautiful examples of LLM usage on scanning Wikipedia content (with some caveats)

AI is here to stay

• AI is never going to take responsibility
• Responsible, thoughtful humans utilizing AI can change the world for the better
• Responsible, thoughtful humans will refuse to use AI when it does not work as one could have hoped for
• The key requirement is open declaration of LLM usage, at its bare minimum

It is about humans

• How do we communicate LLM capabilities and limitations properly?
• Are there more tools available to identify and remove LLM-generated content?
• Beyond moral responsibility, are there avenues for making manufacturers of GenAI systems liable for their users’ actions?
• Could this be the end of the road for the concept of an encylopedia, after all?

It is about Wikipedia

My ToDo

​

• Verification of literature references against library catalogues
• Recurring checks in various languages with special lit templates

🙏

https://github.com/MathiasSchindler/bibtest

https://github.com/MathiasSchindler/dewp-parse