Setting up Fedora as a component of the national repository for archaeology

David Novák

Institute of Archaeology CAS, Prague

Fedora Virtual Showcase 2024

• AIS CR (https://www.aiscr.cz/) = Archaeological Information System of the Czech Republic
• large research infrastructure listed in the national roadmap (https://www.vyzkumne-infrastruktury.cz/) approved by government
• consortium of the two Institutes of Archaeology of the Czech Academy of Sciences
• centralised team, data, services, hardware facilities, software applications, know-how, collaborations, standards and methodologies

2

What is AIS CR?

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

• Provides digital workflow to handle legal responsibilities in the field of archaeology
• developers (announcing projects)
• archaeologists (conducting fieldwork & reporting)
• heritage managers (oversight and curation)
• general public (data access and collaboration)
• Archives data on archaeological fieldwork
• Develops related SW and run needed HW
• Supports and represents the community
• Develops and implements standards
• Ensures long term strategy and sustainability
• Integrates disparate resources

3

What AIS CR does?

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

• 200k projects as announced by developers since 2008
• 212k fieldwork events (what, where, when data) recorded retrospectively since 19th century
• 10k systematically described sites (deserted villages, castles, hillforts, barrow cemeteries etc.)
• 9k individual finds recorded by amateur collaborators (AMCR-PAS system)
• 200k documents (reports, photographs, maps…) on fieldwork events

(about the same number digitised but still not published)

• 10k bibliographic resources related to fieldwork events
• 40+ standardised controlled vocabularies
• 2.6k C14 dates with detailed metadata

​

(and more – 3D models, specialised datasets, etc.)

4

What data we handle?

https://digiarchiv.aiscr.cz/

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

• 1919 – Institutes of Archaeology entrusted with the administration of archaeological research and finds => archives in Prague and Brno
• 1990s - a series of discrete databases and applications
• 2000s – digitization of the documentation
• 2012–2015 – Archaeological Map of the Czech Republic (AMCR) developed (public launch in 2017)
• 2016 – listed in the national Roadmap of Large Research Infrastructures as Archaeological information System of the Czech Republic (AIS CR)
• 2020 – a complete reimplementation of the AMCR launched based on the experience from the first years of operation
• 2022 – decision to implement Fedora 6 as middleware for data storage
• 2024 – innovated AMCR put into operation in production environment (data migration in July; public run in August)

5

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

A book summarising development up to 2015 (Kuna et al. 2015)

https://zenodo.org/doi/10.5281/zenodo.11220286

AIS CR / AMCR history

6

AIS CR architecture

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

7

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

What are the services?

https://amcr-info.aiscr.cz/

https://api.aiscr.cz/

8

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

https://ocfl.io/

https://aiscr-webamcr.readthedocs.io/

https://github.com/ARUP-CAS/aiscr-webamcr/blob/dev/webclient/xml_generator/definitions/amcr.xsd

https://github.com/ARUP-CAS/aiscr-webamcr/blob/dev/webclient/xml_generator/definitions/metadata.xml

9

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

AMCR application (full stack)

10

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Entity-relation model of the AMCR database

11

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

SQL to XML transformation

12

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Fedora repository structure

13

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Fedora OCFL storage

14

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Handling of deleted records

15

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

https://fedora-test.aiscr.cz/rest/fcr:search?condition=fedora_id%3DAMCR%2Frecord%2F*&condition=modified%3E%3D2023-12-15T00%3A00%3A00.000Z

16

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

oai_amcr

oai_dc

• 914 000 record containers (archival groups)
• XML metadata
• files (datastreams) attached to records
• 919 000 indirect containers used for record colletions
• 15 493 471 files in the OCFL storage (very demanding)
• 3 full weeks of migrations
• XML metadata file for each record
• 336k data files
• creation of indirect containers
• generating and storing thumbnails

17

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Migration process and Fedora contents

Pros

• based in widely accepted standards
• high modularity, extendibility and independency of system components
• Fedora 6 does what it should do – nothing less, nothing more
• covers both the end-user services and LTP related workflows
• fully versioned data with options for validation (XSD, SHA-512)
• no vendor lock-in
• Fedora community and Lyrasis support

Cons

• high demands for keeping consistency between PSQL DB and XML metadata (using Django signals and Fedora transactions)
• difficult application testing and debugging
• partly outdated documentation, minor bugs and missing features in Fedora 6
• increased needs for system components orchestration and monitoring
• complex and performance-intensive backups of the OCFL storage

18

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Review of the outcomes

19

THANK YOU!

Fedora Virtual Showcase 2024

Setting up Fedora as a component of the national repository for archaeology

Follow AIS CR community at Zenodo: https://zenodo.org/communities/aiscr