1 of 53

Connecting IIIF and Semantic Cultural Heritage Metadata for Discovery

Rob Sanderson, Julien A. Raemy, Kevin Page,

Daniel Sissman, Claire Knowles, Tom Crane

2024 IIIF Conference

Los Angeles, CA, USA. 6th June 2024

​

https://linked.art/

1

IIIF and Semantic�Metadata

2 of 53

Overview

  1. Introductions and Introduction
  2. Linked Art
  3. Data Model
  4. APIs
  5. Discussion!

​

2

IIIF and Semantic�Metadata

3 of 53

https://shirt.woot.com/offers/existential-whale

3

IIIF and Semantic�Metadata

4 of 53

The Discovery Use Case

https://github.com/IIIF/iiif-stories/issues/25

4

IIIF and Semantic�Metadata

5 of 53

With apologies to https://shirt.woot.com/offers/existential-whale

What is a Manuscript?

5

IIIF and Semantic�Metadata

6 of 53

Current Solution: Link to ... Something?

https://iiif.io/api/presentation/3.0/#seealso

6

IIIF and Semantic�Metadata

7 of 53

Is It Time To Re-Ask the Question?

  • June 2018 Discovery workshop at Stanford:
    • In IIIF Drive: “The Great Debate ‘18”
    • Outcome: “This is too hard!”�
  • Changes since 2018:
    • Significant overlap between communities & practice
    • Knowledge Graphs are popular
    • Technological advances make this feasible at scale
    • Linked Art exists!

​

​

7

IIIF and Semantic�Metadata

8 of 53

What is Linked Art?

A Linked Open Usable Data (LOUD) specification, collaboratively designed to work across cultural heritage organizations, allowing easy publication and use of our knowledge.

​

Linked Art provides a Standards based metadata profile,

… which Consistently solves problems from real data,

… is designed for Usability and ease of implementation, � … which are prerequisites for Sustainability

​

https://linked.art/

8

IIIF and Semantic�Metadata

9 of 53

Brief History of Linked Art / IIIF

2014

2020

2021

2015

2016

2017

2018

2019

American Art Collaborative�IIIF v2.0 APIs

AAC Target Model

Getty Extensions�IIIF v2.1.1

Kress, AHRC Grants, F2F 1-3�IIIF v3.0

CIDOC WG Approved

1.0

4.0

2022

2023

F2F 4,5

Stanford Discovery

9

IIIF and Semantic�Metadata

10 of 53

LOUD’s Audience is Developers

10

IIIF and Semantic�Metadata

11 of 53

Usable vs Complete: Target Zone

11

IIIF and Semantic�Metadata

12 of 53

Linked Art + IIIF Design Principles

  1. Scope design through shared use cases
  2. Design for international use
  3. Make easy things easy, complex things possible
  4. Avoid dependency on specific technologies
  5. Use REST / Don’t break the web / Don’t fear the network
  6. Design for JSON-LD, using LOD principles
  7. Follow existing standards & best practices, when possible
  8. Define success, not failure
  9. Separate concerns, keep APIs & systems loosely coupled
  10. Address concerns at the right level

https://iiif.io/api/annex/notes/design_principles/

https://linked.art/api/1.0/principles/

12

IIIF and Semantic�Metadata

13 of 53

Linked Art Design Specifics

  • Trivial to Implement (aka IIIF Level 0)� Possible to implement with hand crafted files on disk
  • Consistency across Representations� Each relationship in only one document
  • Division of Information across Representations� From the many to the few, and easy to determine
  • Identity and URI Requirements� One-to-one relationships are embedded, no URIs� The URIs for records do not have any internal structure

https://linked.art/api/1.0/principles/

13

IIIF and Semantic�Metadata

14 of 53

Baseline Theory You Need to Know

  • Knowledge Graph� A method of managing data by describing entities,� connected via named, semantic relationships into� a coherent network or graph
  • Entity� A thing (physical, conceptual, or beyond) of interest� e.g. a physical painting; the concept of oil paint
  • Relationship� The way in which two entities are connected� e.g. the painting has a material of oil paint

​

14

IIIF and Semantic�Metadata

15 of 53

  • Conceptual Model
    • Abstract way to think about the world, �holistically, consistently and coherently�
  • Ontology
    • Shared set of terms to encode that thinking�in a logical, machine-actionable way �
  • Vocabulary
    • Curated set of sub-domain specific terms, �to make the ontology more concrete

Model

Ontology

Vocabulary

Data Model Standards

encoded�by

refined�by

15

IIIF and Semantic�Metadata

16 of 53

Implementation Standards

encoded�by

refined�by

specialized by

available by

Model

Ontology

Vocabulary

Profile

API

A Profile is a selection of appropriate abstractions, to encode the scope of what can be described.�

�An API is a selection of appropriate technologies, to give access to the data managed using the profile.

16

IIIF and Semantic�Metadata

17 of 53

LUX – Yale Collections Discovery

17

IIIF and Semantic�Metadata

18 of 53

LUX – Yale Collections Discovery

18

IIIF and Semantic�Metadata

19 of 53

Getty Museum

19

IIIF and Semantic�Metadata

20 of 53

Getty Research Collections Viewer

20

IIIF and Semantic�Metadata

21 of 53

O’Keeffe Museum

21

IIIF and Semantic�Metadata

22 of 53

Van Gogh World Wide

22

IIIF and Semantic�Metadata

23 of 53

Where is Linked Art Today?

  • Specifications have a solid, stable core
    • Documentation needs to be finalized (almost done!)�
  • Multiple implementations in Production
    • All use slightly different versions
    • Agreement to update when 1.0 is nailed down

23

IIIF and Semantic�Metadata

24 of 53

Linked Art Community Process

  • Calls every two weeks via Zoom
    • Wednesday 8am LA, 11am NY, 4pm UK, 5pm EU
    • Agendas and notes in Google Docs�
  • Slack channel, Google Group, Face to Face meetings�
  • Issues and the specifications are managed via github:� https://github.com/linked-art/linked.art

Based on IIIF’s Process

24

IIIF and Semantic�Metadata

25 of 53

Linked Art Data Model

Don’t Panic!

​

Target: Cross-domain foundational ontology that can be made more precise through domain-specific vocabulary�

  • Core Classes
  • Data Patterns

​

25

IIIF and Semantic�Metadata

26 of 53

Objects and Works

  • HumanMadeObject�A physical thing you can touch (even fossils, meteorites)
  • DigitalObject�A file on a computer somewhere
  • PropositionalObject�An abstract work, not textual or visual (e.g. exhibition idea)
  • LinguisticObject�A textual work (e.g. the text of the Lord of the Rings)
  • VisualItem�A visual work (e.g. the image of The Night Watch)

26

IIIF and Semantic�Metadata

27 of 53

Connecting Classes

  • Person�An individual capable of taking intentional action (humans)
  • Group�More than one Person capable of collective action
  • Place�A fixed geographic location, describable using lat/long
  • Set�An unordered group of other entities
  • Period, Event, Activity�Something that occurs in time and space

​

​

​

27

IIIF and Semantic�Metadata

28 of 53

Concepts

  • Type�A category or classification of any sort (landscape)
  • Language�A human language (Dutch)
  • Material�A classification of matter (oil paint, canvas)
  • MeasurementUnit�A unit for understanding a dimension value (cm, seconds)
  • Currency�A unit for understanding a monetary value (euro, dollars)

​

28

IIIF and Semantic�Metadata

29 of 53

Running Example: The Night Watch

29

IIIF and Semantic�Metadata

30 of 53

URI, Class, Label

{

"@context": "https://linked.art/� ns/v1/linked-art.json",

"id": "https://rijks.nl/o/sk-c-5",

"type": "HumanMadeObject",

"_label": "Nightwatch"

}

Note: URIs are not real, but compressed to fit into the slides

30

IIIF and Semantic�Metadata

31 of 53

Classification

{

"@context": "https://linked.art/� ns/v1/linked-art.json",

"id": "https://rijks.nl/o/sk-c-5",

"type": "HumanMadeObject",

"_label": "Nightwatch",

"classified_as": [{� "id": "https://…/300033618", � "type": "Type", � "_label": "painting"� }]

}

31

IIIF and Semantic�Metadata

32 of 53

Sidebar on Vocabulary

We typically use Getty’s AAT – Art & Architecture Thesaurus

​

Terms fall into three categories:

  • Required: You must use the term to be considered valid.�Example: “Primary Name”
  • Recommended: You should use the term unless there’s a reason not to. Example: “Painting”
  • Listed: You can use the term if you want, no pressure.�Example: “Village”

​

32

IIIF and Semantic�Metadata

33 of 53

Identifiers

{

"id": "https://rijks.nl/o/sk-c-5",� "type": "HumanMadeObject",� "_label": "Nightwatch",

"identified_by":[{

"type": "Identifier", � "_label": "NW’s objno",

"content": "SK-C-5",

"classified_as": [{� "id": "https://…/300404621",� "type": "Type",� "_label": "repository number"� }]

}]�}

33

IIIF and Semantic�Metadata

34 of 53

Names

{

"id": "https://rijks.nl/o/sk-c-5",� "type": "HumanMadeObject",� "identified_by":[{� "type": "Name", � "content": "The Night Watch",

"language": [{� "id": "https://…/300388277",� "type": "Language",� "_label": ”English"}]

},{

"type": "Name", � "content": ”De Nachtwacht",

"language": [{� "id": "https://…/300388256",� "type": "Language",

"_label": "Dutch"}]� }]}

34

IIIF and Semantic�Metadata

35 of 53

Statements

{

"id": "https://rijks.nl/o/sk-c-5",

"type": "HumanMadeObject",

"_label": "Nightwatch",

"referred_to_by":[{� "type": ”LinguisticObject", � "_label": "NW materials",� "content": "oil on canvas",

"language": […],

"classified_as": [{� "id": "https://…/300435429",� "type": "Type",� "_label": "materials"� }]

}]�}

35

IIIF and Semantic�Metadata

36 of 53

Activities

{

"id": "https://rijks.nl/o/sk-c-5",

"produced_by":{� "type": "Production", � "carried_out_by": [{� "id": "https://…/500011051",� "type": "Person"}],� "took_place_at": [{� "id": "https://…/7006952",� "type": "Place"}],

"technique": [{� "id": "https://…/300053343",� "type": "Type"}],

"timespan": {…}

}�}

36

IIIF and Semantic�Metadata

37 of 53

TimeSpans

{

"id": "https://rijks.nl/o/sk-c-5",

"produced_by":{� "type": "Production", � "timespan": {� "type": "TimeSpan",� "begin_of_the_begin":”1642-01-01”, � "end_of_the_begin":”1642-06-01”,� "begin_of_the_end":”1642-06-01”,� "end_of_the_end": ”1642-12-31",� "identified_by":[{� "type": "Name",� "content": ”1642",� "classified_as": [{…}]� }

}�}

37

IIIF and Semantic�Metadata

38 of 53

Beginnings & Ends of Existence

  • HumanMadeObject produced_by Production� removed_by PartRemoval
  • DigitalObject created_by Creation
  • VisualItem, LinguisticObject created_by Creation
  • Concepts, Set created_by Creation
  • Person born Birth
  • Group formed_by Formation
  • HumanMadeObject destroyed_by Destruction
  • Person died Death
  • Group dissolved_by Dissolution�

​

38

IIIF and Semantic�Metadata

39 of 53

Partitioning Everywhere

To be more specific about an aspect of some entity, we need to describe the individual part with that aspect:

    • A frame is part of a Painting (physical things)
    • A digital image is part of a digital document (digital things)
    • A chapter is part of the full text (works)
    • A city is part of a county (place)
    • An agent’s role is part of an activity (temporal)
    • A concept has a broader concept (types)
    • An entity is a member of a collection (set/group membership)

39

IIIF and Semantic�Metadata

40 of 53

Partitioning

{

"id": "https://rijks.nl/o/sk-c-5b", � "type": ”HumanMadeObject",� "part_of": [{� "id": "https://rijks.nl/o/sk-c-5",� }],� "produced_by":{� "type": "Production", � "part": [{

"type": "Production",� "technique": [{� "id": "https://…/300053343”}] � "carried_out_by": [{� "id": "https://…/500099999”}] � }]� }�}

40

IIIF and Semantic�Metadata

41 of 53

Equivalence

{

"id": "https://…/ulan/500011051", � "type": "Person",� "_label": "Rembrandt",� "equivalent": [{� "id": "https://wikidata…/Q5598",� "type": "Person", � "_label": "Rembrandt"� }]�}

41

IIIF and Semantic�Metadata

42 of 53

Dimensions

{

"id": "https://rijks.nl/o/sk-c-5",� "type": ”HumanMadeObject", � "dimension": [{� "type": "Dimension",� "value": 379.5,� "unit":[{� "id": "https://…/300379098",� "type": "MeasurementUnit", � "_label": "cm"� }],� "classified_as":[{� "id": "https://…/300055644",� "type": "Type", � "_label": "height" � }]� }]�}

42

IIIF and Semantic�Metadata

43 of 53

Related Digital Images

{

"id": "https://rijks.nl/o/sk-c-5",� "type": "HumanMadeObject", � "representation": [{� "type": "VisualItem",� "digitally_shown_by":[{� "type": "DigitalObject", � "format": "image/jpeg",

"classified_as": [{…}],

"access_point": [{

"id": "https://…/skc5.jpg",� "type": "DigitalObject"}]� }]� }]�}

43

IIIF and Semantic�Metadata

44 of 53

Related Digital Images with IIIF

{

"id": "https://rijks.nl/o/sk-c-5",� "representation": [{� "digitally_shown_by":[{� "type": "DigitalObject", � "access_point": [{...}]

"digitally_available_via": [{

"type": "DigitalService",� "access_point": [{

"id": "https://…/iiif/…",� "type": "DigitalObject"}]� "conforms_to": [{� "id": "https://iiif.io/…",� "type": "DigitalObject"}],� "format": "application/ld+json" �}]}]}]}

44

IIIF and Semantic�Metadata

45 of 53

Related Digital Content

{

"id": "https://rijks.nl/o/sk-c-5",� "type": "HumanMadeObject", � "subject_of": [{� "type": "LinguisticObject",� "language": [{…}],� "digitally_carried_by":[{� "type": "DigitalObject", � "format": "text/html",

"classified_as": [{…}],

"access_point": [{

"id": "https://…/skc5.html",� "type": ”DigitalObject"}]� }]� }]�}

45

IIIF and Semantic�Metadata

46 of 53

Relationships Between Records

  • Can now describe anything by itself
  • Core relationships: classification, partitioning, events
  • Other relationships between records
    • carries / shows
    • current_owner / custodian / location
    • made_of
    • about / represents
    • residence�

46

IIIF and Semantic�Metadata

47 of 53

Linked Art APIs ❤️ IIIF APIs

Lessons learned from IIIF:

    • JSON-LD (semantics are more important!)
    • Simplify property names to be usable
    • Structure is consistent
    • Only GET, easily cacheable
    • Sensible partitioning of responses based on usage
    • No URI structure requirements (à la Prezi)
    • Non-Semantic APIs are Activity Stream based format � (à la Discovery, Search and Annotation Pages)

​

47

IIIF and Semantic�Metadata

48 of 53

Finding Back Links?

Problem: We chose for object refers to the artist, but the artist doesn’t refer to their objects, when looking at the artist record, how do you know which objects they produced?

Naïve Answer: Search!

​

Problem: Standardizing search is impractical (cough sparql)

Answer: Hypertext Application Language link sets!�

Problem: uhhh… hyper what now?

48

IIIF and Semantic�Metadata

49 of 53

Finding Back Links with HAL

Separate the links needed for the API from the semantic data

​

  • Uses IETF standard (forthcoming, draft recently updated)
  • Tooling including validation exists already
  • Add _links to the top level JSON object, that includes
    • Namespace declaration
    • Named links

​

​

​

49

IIIF and Semantic�Metadata

50 of 53

HAL Links: Example

{

"id": "https://rijks.nl/p/rembrandt",� "type": "Person", � "_links": {

"curies": [{

"name": "la",� "href": "https://linked.art/api/1.0/rels/{rel}",� "templated": true� }],

"la:agentProducedObject": {� "href": "https://rijks.nl/p/rembrandt/producedObjects/1"

}

}�}

https://datatracker.ietf.org/doc/html/draft-kelly-json-hal-10

50

IIIF and Semantic�Metadata

51 of 53

Response Format

https://w3.org/TR/activitystreams-core

https://iiif.io/api/discovery/1.0/ �https://iiif.io/api/search/2.0/

{

"@context": "https://linked.art/ns/v1/linked-art.json", � "id": "https://rijks.nl/p/rembrandt/producedObjects/1",� "type": "OrderedCollectionPage", � "partOf": {"id": "…", "type": "OrderedCollection”}, � "next" : {"id": "…", "type": "OrderedCollectionPage”},� "orderedItems": [� {� "id": "https://rijks.nl/o/sk-c-5",� "type": "HumanMadeObject"

}, {…}

]

}

51

IIIF and Semantic�Metadata

52 of 53

Discussion Provocation

Linked Art is the solution to the �“discovery challenge” of IIIF

​

It can describe and connect resources across all cultural and natural heritage domains, is abstract enough to accommodate future domains, and is usable enough to be widely implemented: IIIF should adopt Linked Art as a new TSG and Specification

52

IIIF and Semantic�Metadata

53 of 53

​

Thank You!

Discuss!

​

https://linked.art/

53

IIIF and Semantic�Metadata