LLMs for Public Narrative Understanding
Elinor Poole-Dayan*1 Daniel Kessler*1 Margaret Hughes1 Emily S. Lin2 Marshall Ganz2 Deb Roy1
* equal contribution 1 MIT 2 Harvard University
Why Automate Public Narrative Analysis?
Public Narratives (PNs)—a leadership storytelling practice that links a Story of Self, Story of Us, and Story of Now—translate shared values into collective action.
Yet, data‑driven study is scarce: annotating these nuanced, value‑laden stories is slow, subjective, and costly.
Goal – Test whether modern LLMs can deliver expert‑level PN annotation, unlocking scalable civic‑story research.
Public Narrative Framework
Linked stories (motivation → action)
Structure in each story: Challenge → Choice → Outcome
Key Takeaways
✅ LLMs (o3‑mini) reach human‑like accuracy on explicit, personal elements (Story of Self, Now; Choice)
⚠️ Models still miss nuanced Outcome & Story of Us cues
🤝 LLMs mirror expert consensus and valid disagreements, preserving diversity vital to subjective PN coding
🏗️ Establish a codebook‑driven, human‑validated pipeline — foundation for large‑scale PN & civic‑narrative research
NAACL 2025 Workshop on Narrative Understanding
🌐 Findings replicate on 2024 DNC speeches: same strengths (Self/Now) and gaps (Outcome/Us), pointing to rich future work on non‑PN texts. Check out demo →