Introducing Fedora Repositories
Learning Outcomes
Understand the purpose of a Fedora repository
Learn what Fedora can do for you
Understand the key capabilities of the software
Introduction to Fedora 4
What is a Fedora Repository?
Secure software that stores, preserves, and provides access to digital materials
Supports complex semantic relationships between objects inside and outside the repository
Supports millions of objects, both large and small
Capable of interoperating with other applications and services
Exposing and Connecting Content
Flexible, extensible content modeling
Atomic resources with semantic connections using standard ontologies
RDF-based metadata using Linked Data
RESTful API with native RDF response format
Basic Concepts
New Vocabulary
Fedora 3 | Fedora 4 Linked Data Platform (LDP) |
Objects and Datastreams | Resources LDP-Resource |
Objects | Containers LDP-Basic Container LDP-Direct Container LDP-Indirect Container |
Datastreams | Binaries Non-RDF Source |
Resources
Both containers and binaries are resources
Container resources can have both containers and binaries as children
The tree structure allows for inheritance of things like security policies
Properties
Resources have a number of properties, expressed as RDF triples
Name-value pairs; translated to RDF on REST-API responses
Properties can be RDF literals or URIs
Any number of RDF namespaces can be defined and used
Core Features
Component Stack
Standards
Focus on existing standards
Fewer customizations to maintain
Opportunities to participate in related communities
Core Features and Standards
1. Create/Read/Update/Delete - Linked Data Platform
2. Versioning - Memento*
3. Authorization - WebAC
4. Transactions
5. Fixity
Versioning
Versions can be created on resources with an API call
A previous version can be restored via the REST-API
Authorization
The authorization framework provides a plug-in point within the repository that calls out to an optional authorization enforcement module
Currently, four authorization implementations exist: No-op, WebAC, Role-based and XACML
WebAccessControl
W3C approach for managing authorization using linked data
Interoperable with other applications that implement the same approach
Implemented in Fedora 4 by community stakeholders
Transactions
Multiple actions can be bundled together into a single repository event (transaction)
Transactions can be rolled back or committed
Can be used to maintain consistency
Fixity
Over time, digital objects can become corrupt
Fixity checks help preserve digital objects by verifying their integrity
On ingest, Fedora can verify a user-provided checksum against the calculated value
A checksum can be recalculated and compared at any time via a REST-API request
Fedora Vagrant Components
Solr
Triplestore
(Fuseki, Sesame)
Fedora 4
LDP / WebAC / Memento??
Apache Camel
Non-core Features
Two Feature Types
1) Optional, pluggable components
Separate projects that can interact with Fedora 4 using a common pattern
2) External components
Consume and act off repository messages
Component Architecture
External Component Integrations
Leverages the well-supported Apache Camel project
Camel is middleware for integration with external systems
Can handle any asynchronous, event-driven workflow
External - Indexing
Index repository content for search
Indexing is configurable - could be based on rdf:type property "Indexable" to filter from non-indexable content
Solr has been tested
External - Triplestore
An external triplestore can be used to index the RDF triples of Fedora resources
Any triplestore that supports SPARQL-update can be plugged in
Fuseki, Sesame and BlazeGraph have been tested
External/Pluggable - Audit Service
Maintains a history of events for each repository resource
Both internal repository events and events from external sources can be recorded
Uses the existing event system and an external triplestore
Events can be persisted back to Fedora
Pluggable - OAI Provider
fcrepo4-oaiprovider implements Open Archives Protocol Version 2.0 using Fedora 4 as the backend
Exposes an endpoint which accepts OAI conforming HTTP requests
Supports oai_dc out if the box, but users are able to add their own metadata format definitions to oai.xml
Pluggable - SWORD Server
SWORD is a lightweight protocol for depositing content from one location to another
fcrepo4-swordserver implements 2.0 AtomPub Profile, using Fedora 4 as the backend
SWORD v2 includes AtomPub CRUD operations
Transform
Service to transform resources using a stored program
E.g. transform resource properties to Solr fields
Accessible via /fcr:transform endpoint
Performance and Scalability
Metrics
A number of scalability tests have been run:
Uploaded a 1 TB file via REST API
16 million objects via federation
10 million objects via REST API
Transaction Performance
Multiple actions can be bundled together into a single repository event (transaction)
Transactions offer performance benefits by cutting down on the number of times data is written to the repository filesystem (which tends to be the slowest action)
Clustering
Two or more Fedora instances can be configured to work together in a cluster
Fedora 4 currently supports clustering for high-availability use cases
A load balancer can be setup in front of two or more Fedora instances to evenly distribute read requests across each instance
Useful Resources