​

A Week's Work in Six Hours: �How LLMs will (Probably) Revolutionise the Study of Humanities… But Not Yet

Ian Miell

ian.miell@container-solutions.com

@ianmiell

@ianmiell

Who I Am / Who You Are

• Work @ Container Solutions
• Spent 15 years in online gambling
• Five years at Banks building�Kubernetes platforms, now consult
• Blog for CS and at�https://zwischenzugs.com
• Books on Docker, Bash, Git, Terraform
• I still dabble in software (Bash FTW Zack!)

@ianmiell

@ianmiell

Background

• I did a History degree 1994-1997

​

• Initial toying with LLMs:�bad at writing undergraduate-level history essays…

​

@ianmiell

@ianmiell

The Genius of Douglas Adams

​

He had found a Nutri-Matic machine which had provided him with a plastic cup filled with a liquid that was almost, but not quite, entirely unlike tea.�

Douglas Adams, The Hitch-Hikers Guide to the Galaxy

@ianmiell

@ianmiell

Background

​

• How could I use LLMs and billions of transistors to help me write essays now?

So….

​

@ianmiell

@ianmiell

Challenge!

Can I write a top-quality essay using LLMs in under two days?�Two rules:�1) No peeking at the original essay �2) No reading the texts

@ianmiell

@ianmiell

@ianmiell

@ianmiell

How to Write a History�Essay in 1996

• Time: one week
• Number of lectures: 0
• Books to read: 12
• Articles to read: 6
• Words written: 2100
• Cheap red wine: far too much

@ianmiell

@ianmiell

Writing an Essay in 2024

• Download the sources
• ChatGPT essay plan
• Explore topic with questions
• Private GPT as a research tool
• [Agents, Crews, Tools]

@ianmiell

@ianmiell

How to Write a History Essay in 2024

• Sourcing sources
• Only 1 hour!
• grep-able

@ianmiell

@ianmiell

How to Write a History Essay

in 2024 - Step One

• Generate Essay�Plan
• ChatGPT Shines

​

https://github.com/ianmiell/llm_history_essay

​

@ianmiell

@ianmiell

@ianmiell

• Used PrivateGPT�to explore topic�and get quotes using�Retrieval Augmented�Generation
• GitHub repo:

@ianmiell

@ianmiell

LLM 'Crews' Experiment

• Simulate the student/teacher dynamic
• Disappointing results

@ianmiell

@ianmiell

Observations

• Toil significantly reduced
• Limits of 'I' in AI quickly found
• I wanted/needed to actually read the �texts to improve on 1996
• Llama2:70b returned a much�better essay plan than ChatGPT
• As a speed-learning tool it's�unparalleled

Q: Where is this image from?

@ianmiell

@ianmiell

@ianmiell

@ianmiell

Augmented LLMs

"You are going to make [a cup of tea] for me. Keep quiet and listen.” And he sat. He told the Nutri-Matic about India, he told it about China, he told it about Ceylon. He told it about broad leaves drying in the sun. He told it about silver teapots. He told it about summer afternoons on the lawn. He told it about putting in the milk before the tea so it wouldn't get scalded. He even told it (briefly) about the history of the East India Company.�Douglas Adams, The Hitch-Hikers Guide to the Galaxy

@ianmiell

@ianmiell

What Happened Next

​

​

@ianmiell

@ianmiell

​

​

@ianmiell

@ianmiell

AI: Academia's Underestimated Revolution

​

• Academia not ready for this:
• 'the lofty reply from the Fac / Uni was that AI could only summarise, it wasn't analytical, we had nothing to worry about "for many years". […] I don't think universities have begun to get a handle on AI'
• Comms via the Orange Site with educators concur
• Binary view of AI (AGI or not AGI?)
• We've seen this before…

@ianmiell

@ianmiell

A Historical Analogy

• 'History repeats itself, first as tragedy, second as farce.' -?

​

• The history of Chess offers a glimpse of where this might go.

​

• Five-phases of Chess AI…

@ianmiell

@ianmiell

Chess AI - A History (I) - Fakery

• Mechanical Turk
• Built in 1770, exhibited for 85 years.
• "a complicated piece of clockwork�which ingeniously misguides and�deludes the observers"
• 'Beat' Napoleon and Benjamin�Franklin

Q: When was the first 'real' chess computer built?

@ianmiell

@ianmiell

Chess AI - A History (II) - Mechanisation

• El Ajedrecista, built by Leonardo Torres Quevedo.
• Played rook and king vs king endgame
• Could identify illegal moves and checkmate the opponent
• Impressive toy

@ianmiell

@ianmiell

@ianmiell

@ianmiell

Chess AI - A History (III) - Computerisation

• Co-designed by Ken Thompson
• Awarded 'master' status in 1983
• People still believed that computers would never beat humans, as they lacked 'insight'

@ianmiell

@ianmiell

Chess AI - A History (III) - Hubris

@ianmiell

@ianmiell

Chess AI - A History (IV) - Capitulation

• 1997 - Garry Kasparov�defeated in a match�with IBM's Deep Blue
• It was the first time he�ever lost a 'best of n'
• Still maintains he lost�unfairly

@ianmiell

@ianmiell

Chess AI - A History �(V) - Alien Chess

• The Alphazero/Deep�Learning age
• 'Self-learning' - onanistic learning
• The AGI of Chess, greater 'intuition' than best humans

@ianmiell

@ianmiell

@ianmiell

@ianmiell

Where We Are

​

• LLMs can't 'write the essay for you', even if you are a capable historian using private LLMs with RAG
• They (and the internet, and transistors) can save you a lot of time compared to old-fashioned research
• There's still no substitute for independent study
• How long will this last?�Based on chess, 20 years?

@ianmiell

@ianmiell

The Future?

​

On the delivery plate of the Nutri-Matic Drink Synthesizer was a small tray, on which sat three bone china cups and saucers, a bone china jug of milk, a silver teapot full of the best tea Arthur had ever tasted, and a small printed note saying “Wait”.

Douglas Adams, The Restaurant at the End of the Universe

@ianmiell

@ianmiell