2 of 13

Processing a large dataset

Objective: understanding change, uses of language, studying unstructured image and audio collections
Sources: full-text databases (ECCO, EEBO, Finnish Newspapers etc.); image, audio, video collections
Infrastructure: provide access to data, workflow and some tools

Metadata as a systematic source

Objective: Systematic study of subject matter; Quantitative study of material objects
Sources: World is full of different metadata collections ranging from libraries to museums to archives and private collections
Infrastructure: provide access to data, workflow, tools and scalable solutions how research groups provide input

With metadata we can approach more or less any cultural heritage collection!

There is no reason to limit our efforts to texts only. Materiality and material objects are a great source for all SSH fields.

Unstructured

Structured

3 of 13

Virtuous cycle of better data (never-ending process)

Combining harmonized bibliographic data to full-text sources -> Enables text mining and layout detection in a new way.
Using full-texts to enrich bibliographic data -> Feeding back to the loop, better quality data, detecting genre/subject topics for example.
Combining text reuse information to bibliographic data -> feeds back to FRBR/edition information.
The data need not be perfect. It never will be. It needs to be good enough / useable. When is it good enough? Depends on the research questions.

4 of 13

Interaction between research groups and infrastructure

The part of doing new research with respect to creation of structured data from unstructured (e.g. genre detection using AI) is research -> it is done in a research project
When this is scaled up and shared with others as structured data, this becomes also an issue for the infrastructure
It is important that different roles (that can vary) are negotiated early on

5 of 13

Creating new data with layout and full texts

layout & metadata analysis (materiality detection)

6 of 13

Sequential Genre / Subject Topic Change in Historical Texts

Most genre classification methods consider each document as a single genre (or set of probabilities), while in fact a book may contain subsections of different genres.
In HPC-HD we trained different models based on ECCO-BERT, and explore the genre change in one book.

7 of 13

Genre distribution in documents in ECCO

8 of 13

Genre change in a single book

Each book was divided into 512-token chunks and using ECCO-BERT-Seq we generate genre predictions for each.
Aim is to detect sections of genre within a single book, breaking it down into sequences of particular genres.
For example using burst detection (right) though this works best for finding ‘exceptional’ sections of a particular genre.

9 of 13

Sequential genre distributions within single books

10 of 13

Printmark Detection

Image clustering and classification -> more metadata

11 of 13

Printmark detection: Systematic approach to create metadata about printers and publishers

12 of 13

Data sources

Evolving set of tools

Research questions

APIs

Applications

Analysis libs/tools

ECCO

ESTC

Project goals

Research publications

Public tools, APIs, code

Refined data

BLN

EEBO

...

DFN

FENNICA

CERL

...

KUNGLIGA

Data refinement libs/tools

1 of 13