1 of 32

Semi-Automated TEI/XML Encoding �of Modern Korean Novel

Academy of Korean Studies

Cultural informatics PhD. Student Gayeon Kim

2 of 32

Agenda

1

Why encode a novel?

2

The bottleneck — and what makes Korean harder

3

The system — a semi-automated pipeline with Gemini / AI Studio

4

The prompt as encoding manual

5

Live: one paragraph in, XML out

6

Where the model breaks — and why "semi," not "full"

7

Takeaways & open resources

3 of 32

Why encode a novel?

4 of 32

Why encode a novel?

▲ teiCrafter

(Link)

▲ Automated XML Tagging of the Hugo Schuchard Correspondence (Link)

5 of 32

Why encode a novel?

Prior work rarely shares the actual prompts used, and most remains at the level of proposals rather than implemented pipelines.

Existing XML auto-tagging work centers on drama and correspondence; unstructured prose and fiction remain largely untouched.

And no project has yet attempted semi-automated TEI/XML tagging tailored to the specifics of modern Korean Novel.

6 of 32

The bottleneck — and what makes Korean harder

So why use AI?

Time

: Hand-tagging a single short story takes a day or two; across 33 works, that alone is an entire project.

Expertise

: TEI has hundreds of elements; every annotator must know when to use persName vs. rs, yet our work was split across many hands in a training program.

Consistency

: If A's tagging of 『Kamja』 and B's of 『Nalgae』 diverge, the numbers start to lie the moment the two are analyzed together. A corpus becomes data only when it's consistent.

7 of 32

The bottleneck — and what makes Korean harder

So what makes Korean harder?

Mixed script

: A 1906 text and a 1950s text are written differently;

8 of 32

The bottleneck — and what makes Korean harder

So what makes Korean harder?

Mixed script

: A 1906 text and a 1950s text are written differently;

Particles

: Korean attaches particles to names

- 형식-ŭn | 형식-I | 형식-ŭl

Consistency

: One character is "Yŏngch'ae," then "she," then "the daughter," then "the master."

9 of 32

The bottleneck — and what makes Korean harder

Particles

: Korean attaches particles to names

- Hyongsik-ŭn | Hyongsik-I | Hyongsik-ŭl

Consistency

: One character is "Yŏngch'ae," then "she," then "the daughter," then "the master."

10 of 32

The system

11 of 32

12 of 32

13 of 32

The prompt as encoding manual

Role Assignment

You are an expert in digital humanities and the TEI (Text Encoding Initiative) standard, specializing in Korean literature. Your task is to process the provided input text and convert it into a single, complete, and perfectly-formed TEI XML document according to the following rules.

​

14 of 32

  1. TEI Header:
    • The document must have a <teiHeader>.
    • Inside <teiHeader>, there must be a <fileDesc>.
    • Inside <fileDesc>, create a <titleStmt> by inferring the <title> and

<author> from the text content. If they cannot be found, use “제목 미상” for the title and “저자 미상” for the author.

    • Inside <fileDesc>, create a <publicationStmt> with <publisher>한국학중앙연구원 디지털인문학연구소</publisher>.
    • Inside <fileDesc>, create a <sourceDesc> with a <p> tag stating that

the text was provided by the user.

    • After <fileDesc>, create an <encodingDesc> containing

<projectDesc><p>본 전자본은 TEI P5 지침(TEI ALL)에 따라 구조화함.</p></projectDesc>.

    • After <encodingDesc>, create the <profileDesc>.

​

​

Configuring the <teiHeader>

15 of 32

Configuring the <particDesc>

  1. Profile Description:
    • Inside the <profileDesc>, you MUST include a <langUsage> element containing <language ident="ko">Korean</language>.
    • Add a <textClass> with <keywords scheme="local"><term>근현대 한국문학

</term></keywords>.

  • Ensure a <particDesc> element with a <listPerson> exists.
  • In <listPerson>, all characters (persons) from the text must be defined. Each character must have a <person> tag with a unique 'xml:id' (e.g., using initials).
  • **Crucially**, inside each <person> tag, the character's name MUST be wrapped in a

<persName xml:lang="ko"> tag. Correct any entries that do not follow this format.

- Here is an example of a well-formed <particDesc>:

<particDesc>

<listPerson>

<person xml:id="ME"><persName xml:lang="ko">나</persName></person>

<person xml:id="JS"><persName xml:lang="ko">점순</persName></person>

</listPerson>

</particDesc>

16 of 32

Encoding the <body>

  1. Text Body:
    • The main body of the text must be enclosed within <text><body> and a

single <div> tag.

    • Every paragraph must be wrapped in a <p> tag.

17 of 32

  1. Entity Tagging:

- Identify and tag all named entities that are not already tagged:

<persName> for people, <placeName> for places, <orgName> for organizations.

  • **Particle Exclusion**: When tagging <persName> and <placeName>, you must exclude any attached Korean postpositional particles (조사). For example, "이형식은" should be tagged as "<persName>이형식</persName>은".
  • **Hanja/Chinese Characters**: For proper nouns with associated Hanja in parentheses, use the <foreign xml:lang="zh">羽</foreign>.

​

18 of 32

<persName>

<placeName>

<orgName>

Tagging through LLM reasoning

19 of 32

The Importance of Particle Separation

When tagging keeps the particle attached…

이형식 | 이형식은 | 이형식이 | 이형식을 → recognized as separate entities.

Normalizing every form of 이형식 into one canonical entity

20 of 32

Handling Hanja Tagging

Previous version: handled with <choice><orig><reg>

​

Source: 羽

Tagged: <foreign xml:lang=“zh”>羽</foreign>

Present version

- simply handled with the <foreign> tag

21 of 32

  1. Dialogue and Monologue:

- **Direct Speech**: Convert all direct speech/dialogue (indicated by "...") into <said> elements.

The <said> element MUST have a 'who' attribute referencing the speaker's 'xml:id' from

<listPerson> (e.g., <said who="#LHS">).

- **Internal Monologue**: Convert all internal monologues (thoughts, typically enclosed in '...')

into <said aloud="false"> elements. This element also requires a 'who' attribute.

- The spoken text itself should be inside the <said> tag, but it does NOT need to be wrapped in a

<p> tag.

  • **Speech Attribution**: Any narrative text indicating who is speaking (e.g., "형식이가 말했다") must remain outside the <said> element.
  • **CRITICAL RULE on Punctuation**: The surrounding quotation marks (double ", single ') or brackets (『』, 「」) **MUST remain OUTSIDE** the <said> element. The <said> tag should wrap ONLY the content of the speech.
  • Correct: "<said aloud="true" direct="true" who="#JS">얘! <rs ref="#ME">너</rs> 혼자만 일하

니?</said>“

- Incorrect: <said who="#JS">"얘! <rs ref="#ME">너</rs> 혼자만 일하니?"</said>

22 of 32

“얘! 너 혼자만 일하니?”

  • “<said aloud=“true” direct=“true” who=“#JS”>얘!

<rs ref=“#ME”>너</rs> 혼자만 일하니?</said>”

Direct Speech

Indirect Speech

‘이게 꿈인가...’

  • ‘<said aloud=“false” who=“#ME”>이게 꿈인가...</said>’

23 of 32

“얘! 너 혼자만 일하니?”

* “<said aloud=“true” direct=“true” who=“#JS”> 얘! <rs ref=“#ME”>너</rs> 혼자만 일하니?</said>”

Quotation-Mark Placement

Place quotation marks outside the tag

24 of 32

점순이가 말했다. “뭐해?”

* 점순이가 말했다. “<said who=“#JS”>뭐해?</said>”

Narration Separation

Narrator description vs. character speech — separation

25 of 32

6. Person and Reference Tagging:

  • **Proper Names**: When a character is mentioned by their actual name (e.g., "영채", "형식", "김장로"), use the <persName> element.
  • Example: <persName ref="#YC">영채</persName>
  • **References/Pronouns**: When a character is mentioned by a pronoun (e.g., "그", "그녀", "자기"), a nickname, a title, or a descriptive noun (e.g., "딸", "악한", "주인"), use the <rs> element with type="person".
  • Example 1 (Pronoun): <rs type="person" ref="#YC">그</rs>
  • Example 2 (Noun): <rs type="person" ref="#YC">딸</rs>
  • Example 3 (Descriptive): <rs type="person" ref="#BMS">악한</rs>
  • **Mandatory Attribute**: Both <persName> and <rs> MUST include the 'ref' attribute pointing to the correct 'xml:id' defined in the <listPerson>.

​

26 of 32

  1. Structure and Divisions:
  2. **Chapter Container**: Wrap each chapter in a <div> element.
  3. **Mandatory Attributes**: The <div> tag MUST include the following attributes:
  4. type="chapter": To indicate this division is a chapter.
  5. n="X": Where 'X' is the chapter number (e.g., "1", "2").
  6. **Chapter Title**: The chapter number or title text (e.g., "1", "Chapter One") must be placed inside a <head> element immediately after the opening <div> tag.
  7. **No Nesting**: Ensure that the previous chapter's </div> tag is closed BEFORE opening a new chapter's <div>. Do not nest chapters inside each other.
  8. **Example**:

<div n="1" type="chapter">

<head>1</head>

<p>...content of chapter 1...</p>

</div>

27 of 32

Finalizing the output

  1. **Cleanup & Final Output**:
  2. Do not include page numbers in the XML <body>.
  3. The final output must be ONLY the complete, refined XML document, starting with

<?xml version="1.0" encoding="UTF-8"?>. Do not include any commentary, explanations, or markdown formatting like \`\`\`xml.

28 of 32

Live: one paragraph in, XML out

29 of 32

Where the model breaks

and why "semi," not "full"

<Big Problem>

Faulty <rs> resolution

— the model struggles to infer reference from context

Surface-level tagging

— the model marks form, not meaning

30 of 32

Open resources

▲ GitHub (Link)

31 of 32

Open resources

▲ Zenodo v4 (Link)

32 of 32

Thank you

For questions, contact eiloppang@naver.com