1 of 15

2 of 15

The Problem

2 / 17

The Edison Papers Digital Edition hosts ∼154,000 digitized documents.

Browsing works. Metadata is rich.

But almost none of it is transcribed — so none of it is searchable as text.

149,619 documents without a transcription

4,147 done

2.7% transcribed.

3 of 15

Why This Is Hard

3 / 17

One real page from the archive (et0976):

Cursive, 1888, two different hands
Marginal notes crammed between lines
Words crossed out and replaced
Printed letterhead mixed with handwriting

This is the normal case, not the worst case.

4 of 15

How Transcriptions are Measured

4 / 17

Both metrics are edit distance, normalized by the reference length:

CER =

S + D + I

^Nchars

S + D + I

WER =

N

words

S, D, I = substitutions, deletions, insertions.

Note: insertions count on top but not below — so CER can exceed 1.0. A model that hallucinates scores 6.11, not 1.0.

Worked example

reference: dear sir, i enclose the report

prediction: dear si, i enclose the report now

	Characters	Words
Deleted	1 (r)	0
Inserted	4 ( now)	1 (now)
Substituted	0	1 (sir,)
Reference length	30	6
Score	0.167	0.333

WER is harsher — one wrong character condemns a whole word.

5 of 15

Benchmarked 8 Methods

5 / 17

Method	Type	Docs	CER↓	WER↓	$/doc

PaddleOCR-VL → Qwen	Hybrid	26	0.198	0.261	0.0076
docTR → Qwen	Hybrid	26	0.220	0.278	0.090
Qwen3.6-27B	VLM	26	0.218	0.276	0.0223
Qwen3.6-Flash	VLM	26	0.235	0.299	0.0069
Llama-4-Scout	VLM	26	0.234	0.324	0.0005
PaddleOCR-VL	OCR-VL	26	0.244	0.320	free
Mistral-Small-3.2	VLM	26	0.452	0.503	0.0003
docTR	OCR	26	0.516	0.823	free

26 documents from Volume 9, ground truth from the published book edition, every document hand-labeled with challenge tags.

6 of 15

Where Each Method Breaks Down

6 / 17

Green = lower error = better.

Crossouts and handwriting are hard for everything
Classical OCR (top row) simply cannot read

19th-century script

The hybrid (bottom row) is the greenest band overall

7 of 15

Why the Hybrid Performs Best in Our Benchmark

7 / 17

PaddleOCR-VL

local, free

+ faithful to the page – garbles hard glyphs

Qwen3.6-27B

cloud, paid

+ reads context – can hallucinate

Hybrid

draft anchors the model model repairs the draft

CER 0.199

draft text

+ page image

The model never sees a blank slate — it sees a draft it must justify against the image
That constraint is what suppresses runaway hallucination
The free half does the reading; the paid half only corrects

8 of 15

Archive Specific Prompting and Guardrails

Structural separation

Tag what the author didn’t write, inline: [Letterhead] [Stamp] [Marginal note]. Otherwise a researcher can’t tell Edison’s words from a mailroom stamp.

Entity gazetteer

A curated gazetteer repairs “Leadowcroft” → “Meadowcroft” – but only where the page image supports the correction

Conservative splitting

“When uncertain, prefer continuation over inventing a split” — a missed boundary is one click to fix; a spurious one is a mess. Duplicate scans are flagged on a separate axis.

Deterministic fields in code

Titles follow a fixed pattern on the live site, so they’re computed in code, never generated.

page image

shows “Meadowcroft”

OCR draft

“Leadowcroft”

gazetteer

contains “Meadowcroft”

output

“Meadowcroft”

only because the image supports it

9 / 17

All three prompts are versioned files; every stored transcription records which version produced it.

Once the method is fixed, the prompt is the only remaining lever on quality, so these are versioned files, not strings buried in the code. Four design decisions worth calling out. First, structural tags. A scanned letter isn't just prose — there's letterhead, receipt stamps, filing codes. None of that was written by the author. If it all flattens into one paragraph, a researcher can't tell Edison's words from the mailroom's stamp. So non-authored material gets tagged inline.

Second, the gazetteer — and this is the diagram at the bottom. OCR reliably fails on proper nouns, because there's no linguistic context to recover them from. 'Meadowcroft' comes back as 'Leadowcroft'. So the prompt carries a curated list of names, places, and organizations that recur across the collection. But a list of plausible names is also a list of things a model will happily hallucinate, so the instruction is deliberately one-directional: it may only repair a word already on the page, never introduce one. The prompt says 'a spelling aid, never content to add.'

Third, the split prompt is asymmetric on purpose — I'll come back to that in a second.

Fourth, and I think this is the most useful principle: we don't ask the model for the title. Titles on the live site follow a fixed pattern built from document type, author, recipient, and date. That's computable, so it's computed. Never ask a model for something you can compute — it just adds variance and cost

9 of 15

What Was Built: Two Pipelines

9 / 17

Existing Documents (transcription)

Discover

Extract

Transcribe

Review

Export

New Documents (transcription + indexing)

Upload

Split

Confirm

Draft

+ index

Review

Export

Shared: the same hybrid engine, the same storage layer (file:// today, s3:// by config), the same review UI.

New on the bottom row: bulk scans have no document boundaries and no metadata — both have to be produced.

10 of 15

Where Does One Document End?

10 / 17

A scan folder is an ordered pile of images. Nothing records where one letter ends and the next begins.

The model proposes boundaries page by page; the reviewer confirms or corrects every one before anything is transcribed.

Scaling: a 70-page folder exceeds the request-size limit, so it is split into overlapping windows dispatched 3 at a time — 2.26× faster on a real 71-page batch.

The boundaries problem is the interesting one. A scan folder is an ordered pile of images and nothing records where one letter ends and the next begins. Page five is not necessarily a continuation of page four. So the model gets asked a much narrower question than transcription — page by page, does this start a new document?

And then the reviewer confirms or fixes every boundary before anything gets transcribed. This is where that asymmetry comes in. The prompt explicitly says: when uncertain, assume continuation rather than inventing a split. Those two errors aren't equally expensive — a missed boundary is one click to fix, while a spurious split fragments a document and takes real cleanup. There was also a scaling problem here. A 70-page folder is too large for one request, so it gets broken into overlapping windows.

Originally each window depended on the previous one's answer, so they had to run in sequence. I restructured it so the window layout is computed from position alone, which means the whole plan is known upfront and can be dispatched three at a time. That measured 2.26 times faster on a real 71-page batch.

11 of 15

Nothing Counts Until a Human Accepts It

11 / 17

Real output — the same 1888 page from earlier. It read the cursive, tagged the letterhead, pulled the angled marginal note, dated it 1888-02-04. Cost $0.0016.

Every field editable, raw AI output never overwritten, export takes only accepted documents.

12 of 15

What It Costs to Run

12 / 17

Measured cost

One single-page letter Benchmark avg (multi-page) PaddleOCR-VL draft pass

$0.0016

$0.0076

Free

Projected: all 149,619 documents

If mostly single-page

If multi-page average

∼$245

∼$1,420

Why it can’t run away

A hard all-time spend cap, checked before every model call — not per run. Reached ⇒ drafting stops.
Adjustable live in the Settings menu, no restart
Human-paced: nothing runs in a background loop — a reviewer clicks, a bounded batch runs
Split proposal is resumable — a

mid-batch failure never re-bills finished pages

The bottleneck isn’t cost — it’s review throughput.

13 of 15

What’s Done

Benchmarked 8 methods; picked the hybrid on evidence, not intuition
Both pipelines working end to end

against the live archive

AI split proposal + full metadata indexing
Filtered Discovery - the reviewer picks what to transcribe
Cost caps, resumability, remote multi-device access
343 automated tests passing

13 / 17

149,619 documents

have a working path to transcription

$245 – $1,420

projected for the entire backlog

Every output provisional

a human accepts it before it counts

14 of 15

For Future Development: Archive → Research Platform

14 / 17

154,000 scanned documents

Omeka S — already existed

Machine-readable text + index

this summer’s work

Semantic search · APIs

future development

Each layer needs the one beneath it to exist first.

The original goal was to move the site from a static repository to a dynamic research platform. Semantic search and public APIs were always the destination — but neither is possible over images.

Now that the text and metadata exist:

Semantic search — embed the transcriptions so a researcher can search by meaning, not just keyword; the indexed places, names and subjects give it structure to filter on
Public APIs — expose transcriptions and their entity index for external digital-humanities work
Write-back to Omeka — close the loop so accepted transcriptions land on the live site, not just a CSV
Corpus-scale analysis — correspondence networks, topic and place trends across 150,000 documents

Which brings it back to where the project started. The original goal was to move this from a static repository to a dynamic research platform — semantic search, public APIs, new kinds of digital humanities work.

That was always the destination. But none of it is possible over page images. You cannot semantically search a JPEG.

That's the middle layer in this diagram, and it's what this summer produced: machine-readable text plus a structured index of places, names, and subjects, sitting on top of the scans that were already there. With that layer in place, the rest becomes tractable. Semantic search — embed the transcriptions so a researcher can search by meaning rather than exact keyword, with the indexed entities giving that search real structure to filter on.

Public APIs, so external projects can build on the collection. Writing accepted transcriptions back into Omeka so the loop actually closes. And corpus-scale analysis — correspondence networks, how topics and places shift over Edison's career — which simply isn't a question you can ask of 150,000 images.

The foundation is built. That's what it unlocks.

15 of 15

Questions?

15 / 17