“Zwischen Tool und Forschungswerkzeug”. Generative KI in den (digitalen) Geisteswissenschaften�
Zwischen Codices und Codes
Innovative Perspektiven in der Mittelalter- �und Frühneuzeitforschung. �Ringvorlesung des IZMF. 21.10.2024
Christopher Pollin
Digital Humanities Craft OG�www.dhcraft.org �Institut für Digitale Geisteswissenschaften,�https://digital-humanities.uni-graz.at
Aerosol paint illustration of a golden robot working with books, stacks of medieval documents in front, full body shot, white background, Ultra HD
“Zwischen Tool und Forschungswerkzeug”. Generative KI in den (digitalen) Geisteswissenschaften
Generative KI
“A toy, until it isn’t”
Grundlagen LLM und Prompt Engineering
“Tools on demand”
o1-preview
“The Intelligence Age” (!?)
Aerosol paint illustration of a golden robot working with books, stacks of medieval documents in front, full body shot, white background, Ultra HD
Erstellung eines Videos mit Sora��Analyse des Videos mit GPT-4 und �Erstellung eines Storyboards für eine „Naturdokumentation“.��Klonen der eigenen Stimme��Erstellung eines Films, in dem eine Stimme das Geschehen beschreibt��Übersetzung in andere Sprachen
Generative KI … in 03:26
Late nights, dino delights
OpenAI. Sora. �https://openai.com/sora
Matthew Berman. OpenAI's "World Simulator" SHOCKS The Entire Industry | Simulation Theory Proven?!. https://youtu.be/BH9FU7Gd6v8?si=mR1bUre_FVLaQUA-
AI Explained. Sora - Full Analysis (with new details). https://youtu.be/nYTRFKGR9wQ?si=V3QTXC9gp2Z3o2yE
“A toy, until it isn’t”
Ethan Mollick. Gradually, then Suddenly: Upon the Threshold. https://www.oneusefulthing.org/p/gradually-then-suddenly-upon-the
Hyper realistic diptych of transforming objects with white background, child's hand holding building blocks. adult hand writing in a planner. Warm to cool color shift. Depth-of-field blur techmecha-cat in the style of Tekkonkinkreet anime --style raw --v 6.1 --ar 16:9 --q 2
Tools
NotebookLM ist ein brauchbares Tool!
“Large Language Models (LLM) are �like having a Zip-File of the internet”�[nur nicht mehr vollständig entpackbar]
Andrej Karpathy. [1hr Talk] Intro to Large Language Models. https://www.youtube.com/watch?v=zjkBMFhNj_g&list=WL&index=16
Sureal painting of a hyper realistic and sureal gigantic yellow folder with zipper, like a desktop icon, ultra detailed, salvador dali desert background, landscape, by Frederic Edwin Church –v 6.1
Skalierung
Embeddings
Hochdimensionale Vektordarstellungen von Token
The King doth wake tonight and takes his rouse
“Modern English”
The King doth wake tonight and takes his rouse
The King wakes up tonight and begins his celebration
The King wakes up tonight and begins his celebration
“Shakespearean English”
“Let’s think step by step!
�“Prompting is weird. Prompting matters.”
Ethan Mollick. Captain's log: the irreducible weirdness of prompting AIs. https://www.oneusefulthing.org/p/captains-log-the-irreducible-weirdness
Prompt Engineering ist die Kunst und Wissenschaft, präzise und effektive Eingabeaufforderungen (Prompts) für KI-Systeme wie LLMs zu formulieren, um gewünschte Ausgaben oder Verhaltensweisen zu erzielen.
�Chain of Thought = �“Simulation eines Pseudo-Reasonings”
Daten-
erfassung
Daten-
transformation
Daten-�modellierung
Daten-�visualisierung
…
Pollin, C. (2024). Workshopreihe “Angewandte Generative KI in den (digitalen) Geisteswissenschaften” (v1.1.0). Zenodo. https://doi.org/10.5281/zenodo.10647754
95% o1-preview mit ein bisschen Prompting: eine WebApp einer interaktiven Treemap aller Objekte in “Stuben” von Schlössern (https://realonline.imareal.sbg.ac.at); SPARQL Query als JSON Result.
CROWN.
Untersuchungen zu Materialität, Technologie und Erhaltungszustand der Wiener Reichskrone.
Sunburst aller Fassungstypen und aller ihrer Datenfelder + Frequenz
Aufstellung aller Objekte und der Anzahl der Granalien (“Goldkügelchen-Ornamente”).
~95% GPT-4o und Claude 3.5 Sonnet generiertes interaktives Daten-Dashboard �(Excel, Plotly Dash Python)
GPT o1-preview Moments
Learning to Reason with LLMs. https://openai.com/index/learning-to-reason-with-llms�OpenAI Releases GPT Strawberry 🍓 Intelligence Explosion!. https://www.youtube.com/watch?v=NbzdCLkFFSk �Something New: On OpenAI's "Strawberry" and Reasoning. https://www.oneusefulthing.org/p/something-new-on-openais-strawberry �Explaining OpenAI's o1 Reasoning Models. https://www.youtube.com/watch?v=jrA47yocyV0 �Scaling: The State of Play in AI. https://www.oneusefulthing.org/p/scaling-the-state-of-play-in-ai
Can ChatGPT o1-preview Solve PhD-level Physics Textbook Problems?. https://www.youtube.com/watch?v=a8QvnIAGjPA
ChatGPT o1 preview + mini Wrote My PhD Code in 1 Hour*—What Took Me ~1 Year. https://youtu.be/M9YOO7N5jF8?si=-lYWaQ1LvgmzHnHQ
OpenAI o1-preview answer unformalized physics questions. https://www.youtube.com/watch?v=wAnkM10FByY&list=WL&index=32
„Bauen Sie Ihre �eigenen Werkzeuge“
Skalierung
https://epochai.org/blog/can-ai-scaling-continue-through-2030
AI Explained. 8 Most Controversial Terms in AI, https://www.coursera.org/learn/8-most-controversial-terms-in-ai-explained
Ethan Mollick. Scaling: The State of Play in AI. https://www.oneusefulthing.org/p/scaling-the-state-of-play-in-ai
ARC Prize is a $1,000,000+ public competition to beat and open source a solution to the ARC-AGI benchmark. https://arcprize.org.
I Won't Be AGI, Until It Can At Least Do This (plus 6 key ways LLMs are being upgraded). https://youtu.be/PeSNEXKxarU?si=pqkqcbrAHa58W1jg.
Francois Chollet - LLMs won’t lead to AGI - $1,000,000 Prize to find true solution. https://youtu.be/UakqL6Pj9xo?si=f8f_GKJX1nOQUmoW.
A new initiative for developing third-party model evaluations. Anthropic. https://www.anthropic.com/news/a-new-initiative-for-developing-third-party-model-evaluations?s=09
Dr. Chollet. General Intelligence: Define it, measure it, build it. https://youtu.be/nL9jEy99Nh0?si=lY_FfS4aYRY6b2kX
Was kann ein LLM eigentlich?�https://www.patreon.com/posts/was-kann-ein-llm-112314055
“Let’s think step by step” skaliert!
LLMs Still Can't Plan; Can LRMs? A Preliminary Evaluation of OpenAI's o1 on PlanBench. https://www.arxiv.org/abs/2409.13373
OpenAI: ‘We Just Reached Human-level Reasoning’. https://youtu.be/qaJJh8oTQtc?si=Cn8x30DxESnMDfT4
Sam Altmann. The Intelligence Age. https://ia.samaltman.com. OpenAI
Leopold Aschenbrenner. SITUATIONAL AWARENESS. The Decade Ahead. https://situational-awareness.ai (OpenAI)
Dario Amodei. Machines of Loving Grace. https://darioamodei.com/machines-of-loving-grace. Antrophic
Fazit
Vielfalt der Werkzeuge und Arbeitsmethoden �(z.B. Prompt Engineering, Anforderungsanalyse, Prozessmodellierung) ist entscheidend.
“Expert in the Loop”: Domänenwissen und Wissen über Tools und Handhabe.
Digital Humanities optimale Ausbildung für Angewandte Generative KI (?!)
“o1-Momente” ersetzen “GPT-4-Momente”.
“Tools on Demand”
“The Intelligence Age” (!?)
Meinung: Modelle werden im kommenden Jahr deutlich besser sein!
Skalierung im Pretraining: “Weltwissen”� Skalierung in der Inferenz “(Pseudo-)Reasoning”� Tools, UI, Workflows, Prompt Engineering, …
Anhang
Big Context Window
Hatten Sie schon einmal ALLE Paper zu einem Thema gleichzeitig im Context Window?
Gemini 1.5 Experimental 0827 mit einem 2 M Token Context Window (https://ai.google.dev/aistudio)
PRISM -
Parameterized
Recursive
Insight
Synthesis
Matrix�(by Christopher Pollin)
# PRISM: Parameterized Recursive Insight Synthesis Matrix
You're an AI using the PRISM problem-solving method. For each task:
1. **Analyze**
- Identify objectives, constraints, resources
- Restate problem concisely
- Consider potential sub-problems for recursive analysis
2. **Parameterize**
- Set: Thinking Type, Focus Area, Depth, Timeframe
- Justify choices briefly
- Adjust parameters for sub-problems as needed
3. **Matrix Creation**
| Step | Description | Considerations | Outcomes | Branches | Rating | Convergence |
|------|-------------|----------------|----------|----------|--------|-------------|
| 1 | | | | T1.1 | [1-5] | |
| | | | | T1.2 | [1-5] | |
| | | | | T1.3 | [1-5] | |
| ... | | | | ... | ... | |
- Break problem into steps, identifying recursive sub-problems
- For each: describe, consider, predict, branch (2-3 thoughts), rate, converge
- Rating scale: 1 (Poor) to 5 (Excellent), based on relevance, feasibility, and potential impact
- For sub-problems, create nested matrices as needed
4. **Synthesize**
- Integrate insights from all levels of analysis
- Emphasize highest-rated thoughts and their interconnections
- Recommend solutions, addressing both main problem and sub-problems
- Identify uncertainties and potential areas for further exploration
Guidelines: Clear, concise, use Markdown, adapt to task complexity, explain if asked.
Start responses with: "Applying PRISM Method to [task]..."
Interactive Commands:
1. `/deepdive [topic]`: Initiate a Q&A session on [topic] with follow-up questions
2. `/compress`: Summarize current analysis in 3 key points
3. `/iterate`: Perform another cycle of analysis, incorporating new insights