From Pixels to Text
209/07/2025
Evaluating Open-Source OCR Models on Japanese Medical Documents
1
2
2
The Challenge
3
Topic
4
The engines
Paddle
3.0
6
Yomitoku 1.0
7
Tesseract
5.0
8
Candidate OCR Engines
Tesseract
Paddle
Yomitoku
9
01
02
03
How the models are evaluated
10
Text-Level Accuracy
Characters, symbols, number error rate
Quantitative
Processing time
Time from input to text output.
Character Error Rate
S = number of substitutions (wrong character recognized)�
D = number of deletions (a character missing)�
I = number of insertions (extra characters added)�
N = total characters in the reference (ground truth)
CER= S+D+I / N
11
How the texts are built to compare with ground truth
Concatenate output texts of the same row from top to bottom
12
How the models are evaluated
13
Layout fidelity
Compare detected text regions with ground truth
Qualitative
Content Diff
What types of contents differ from ground truth
Readability
How readable the recognized texts are
Developer Experience
Ease of setup
Documentation quality
Output for downstream processing
Image 1
Image 1 Result
15
| Edit Distance | Processing Time |
Paddle | 16.78% | 1.76s 🌟 |
Yomitoku | 0.97% 🌟 | 1.93s |
Tesseract | 33.57% | 9.87s |
Image 2
Image 2 Result
17
| Edit Distance | Processing Time |
Paddle | 10.95% | 6.11s |
Yomitoku | 3.08% 🌟 | 4.95s 🌟 |
Tesseract | 0.19% | 14.2s |
Image 3
Image 3 Result
19
| Edit Distance | Processing Time |
Paddle | 27.03% | 0.99s 🌟 |
Yomitoku | 9.3% 🌟 | 1.22s |
Tesseract | 78.2% | 4.66 |
Layout Fidelity
20
Tendency to fail to detect texts on the edge
Paddle
No major issue
Struggle to locate texts when documents have complicated layout
Yomitoku
Tesseract
Paddle |
Tendency to fail to detect or convert texts on the edge
21
Tesseract |
Struggle to locate where texts when documents have complicated layout
22
Content Diff/ Readability
23
Different versions of the same Chinese character
Tendency to fail to convert texts on the edge
Paddle
Some struggle with handling writing, seal, logo
NA
Yomitoku
Tesseract
Paddle |
Tendency to fail to convert texts on the edge
24
Paddle |
Mix up different versions of kanjis
25
Yomitoku |
Struggles with logo, handwriting and when data quality is very low
26
Effect on error rate when we improve image quality
Image 4 Sharpened
kernel = np.array([
[0,-1,0],
[-1,5,-1],
[0,-1,0]
])
sharpened = cv2.filter2D(img, -1, kernel)
28
Image 4 Contrasted
# Improve contrast using CLAHE
# (on L-channel in LAB color space)
lab = cv2.cvtColor(sharpened, cv2.COLOR_BGR2LAB)
l, a, b = cv2.split(lab)
clahe = cv2.createCLAHE(clipLimit=3.0, tileGridSize=(8,8))
l2 = clahe.apply(l)
lab = cv2.merge((l2,a,b))
contrast = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)
29
Image 4 Improved Error Rate
30
| Original | Sharpened | Contrasted |
Paddle | 27.03% | 18.90%↓ | 20.34%↓ |
Yomitoku | 9.3% | 8.14%↓ 🌟 | 6.4%↓ 🌟 |
Tesseract | 78.2% | 75.87%↓ | 75.58%↓ |
Developer Experience
Ease of setup
31
⭐️⭐️⭐️
Paddle
⭐️⭐️⭐️
⭐️⭐️
No GPU support
Yomitoku
Tesseract
Developer Experience
Documentation quality
32
⭐️⭐️⭐️
Paddle
⭐️⭐️
⭐️⭐️⭐️
Yomitoku
Tesseract
Developer Experience
Output for downstream processing
33
⭐️⭐️⭐️
⭐️⭐️⭐️
Paddle
⭐️⭐️⭐️
Shows text detection confidence rate
⭐️⭐️⭐️
⭐️
hOCR a legacy OCR output format that requires additional styling or processing to render in a browser.
⭐️
Page
└─ Block
└─ Paragraph
└─ Line
└─ Word
Yomitoku
Tesseract
Transparent output with intermediate steps
Easy to parse text and position pairing
Yomitoku 👑
Merits
Struggles only with poor handwriting and low-quality images.
Limited language support.
Licensing (CC BY-NC 4.0) does not guarantee commercial use.
Accurate recognition Consistently extracts text with minimal errors, even from complex layouts.
Fast processing
34
Limitations
Paddle
Merits
Layout sensitivity Error rate rises on documents with complex layouts.
Language coverage Broader than some models, but still limited.
Reliable output Delivers text recognition with high readability on fairly complex documents.
Fast processing
Versatile pipelines Supports a wide range of text processing tasks.
Open licensing Apache 2.0 license allows unrestricted commercial use.
35
Limitations
Tesseract
Merits
Low accuracy Performs poorly on real-world documents.
Slow processing Noticeably longer processing times.
Moderate accuracy Performs reliably on high-resolution documents with straightforward consistent text blocks.
Extensive language support Covers a wide range of languages.
Open licensing Apache 2.0 license permits unrestricted commercial use.
36
Limitations