Technical Webinar on �RO-Crate

29 August 2024 | 11:00-12:00 CEST

FAIR-EASE has received funding from the European Union's Horizon Europe Framework Programme (HORIZON) - under grant agreement No. 101058785.

Welcome and Housekeeping

Rob Carrillo�Trust-IT Services

FAIR-EASE has received funding from the European Union's Horizon Europe Framework Programme (HORIZON) - under grant agreement No. 101058785.

Welcome & Housekeeping

• Write on the chat to say hello and where you are connecting from.

​

• Please mute your microphone and switch off your camera and do not speak unless provided permission by the host. You may ask this via chat.

​

• This is a recorded webinar. Slides and the recording will be published online on the event page within 1 week of the broadcast.

​

• Please keep your questions until the end in case your question is addressed in later slides.

​

• Do share your impressions and screenshots of the webinar. Tag us on our channels:

​

• X : @FAIR_EASE
• LinkedIn: /company/fair-ease

Getting Started with RO-Crate

Stian Soiland-Reyes,�The University of Manchester, UK

FAIR-EASE has received funding from the European Union's Horizon Europe Framework Programme (HORIZON) - under grant agreement No. 101058785.

Aims of FAIR Research Objects

Describe and package data collections, datasets, software etc. �→ data moves with its metadata

Platform-independent object exchange between repositories and services

Support reproducibility and analysis: link data with codes and workflows

Transfer of sensitive/large distributed datasets with persistent identifiers

Aggregate citations and persistent identifiers

Propagate provenance and existing metadata

Publish and archive mixed objects and references

Reuse existing standards, but hide their complexity

Realizing FAIR Digital Objects with RO-Crate

7

Reference existing repositories

Re-use Web standards (JSON-LD, schema.org)�Persistent identifiers w/FAIR Signposting

Add context: people, projects, etc.

https://www.researchobject.org/ro-crate/specification.html

https://www.npmjs.com/package/ro-crate-html �https://w3id.org/ro/doi/10.5281/zenodo.6913045

Guidance by examples

9

https://www.researchobject.org/ro-crate/specification

RO-Crate in practice

RO-Crate is used by multiple international projects

​

Applied across research domains –

from life sciences to cultural heritage

​

​

​

https://www.researchobject.org/ro-crate/use_cases

Collecting corpora for a �Language Data Commons

https://www.researchobject.org/ro-crate/ldaca

https://www.researchobject.org/ro-crate/paradisec

https://youtu.be/p-GZbe-Kzww

{

"@id": "#place-Kiriwina",

"@type": "Place",

"geo": {

"@id": "#geo-151.0,-8.6-151.2,-8.4"

},

"name": "Kiriwina"

},

{

"@id": "#language-kij",

"@type": "Language",

"code": "kij",

"location": {

"@id": "#geo-150.294,-8.879-151.2,-8.32"

},

"name": "Kilivila"

},

{

"@id": "#geo-150.294,-8.879-151.2,-8.32",

"@type": "Geometry",

"asWKT": "POLYGON((150.294 -8.32, 151.2 -8.32, 151.2 -8.879, 150.294 -8.879, 150.294 -8.32))"

},

​

​

ROHub: Earth observation data

The EOSC project RELIANCE used RO-Crate to package data cubes of earth observation data, along with documentation, images and workflows

→ FAIR2ADAPT project

​

Connects to related infrastructures for interactive execution/analysis.

​

Metadata includes temporal coverage, spatial coverage and vertical coverage.

​

ROHub publishes the archived RO-Crates to general-purpose repositories (Zenodo, B2Share) for longevity and PIDs.

12

https://w3id.org/ro-id/6fa27870-c1a4-4386-8d51-855d5ac932e2

https://reliance.rohub.org/

https://www.researchobject.org/ro-crate/in-use/rohub.html

Climate-Adapt4EOSC

New HORIZON-INFRA-2024-EOSC-01-01 projects starting ~ Jan 2025: FAIR2ADAPT, Climate-Adapt4EOSC

ClimateAdapt will use ontologies, knowledge graphs and DDI-CDI to map diverse set of Earth science data spaces, sources and APIs.

RO-Crate captures provenance of FAIRification transformations, carries onwards licenses and attributes.�→ RO-Crate as metadata vessel

Legal & Policy engine reasons on acceptable use of combined data sources. �→ RO-Crate as evidence store.

Logo to be decided..

Building an �EOSC ecosystem of�FAIR Workflows

14

• EOSC projects BY-COVID, EOSC-Life, EuroScienceGateway, BioDT exchange rich �Workflow RO-Crates within an emerging �EOSC ecosystem of workflow services
• Workflow Crates transfer
• identifiers, authors, license, workflow system
• executable workflows in their native format (e.g. Galaxy)
• interoperable CWL description of the workflow
• software citations (e.g. tools used)
• required data sources
• test suites
• workflow execution provenance

https://workflowhub.eu/

https://w3id.org/workflowhub/workflow-ro-crate/

https://w3id.org/ro/wfrun/

RO-Crate Profiles: �Community defined Content Checklists

Profiles give a set of conventions, types and properties that minimally require and expect to be present in that subset of RO-Crates.

• Duck typing for creation, consumption, rendering
• Classification for finding and comprehension.
• Profile Crate for further defining RO-Crates profile resources
• Key to extensibility and diversity

Recording provenance of workflow runs with RO-Crate, https://arxiv.org/abs/2312.07852

https://www.researchobject.org/ro-crate/profiles.html

​

RO-Crate tutorials

https://www.researchobject.org/ro-crate/tutorials

Sandbox (experimental): https://ro-crate.ldaca.edu.au/

Python: https://pypi.org/project/rocrate/

General Introduction on Interoperability: �Work done in FAIR-Impact

Esteban González Guardia,

Universidad Politécnica de Madrid

FAIR-EASE has received funding from the European Union's Horizon Europe Framework Programme (HORIZON) - under grant agreement No. 101058785.

What is interoperability?

• Interoperability - a characteristic of an IT system, whose interfaces are completely understood, to work with other IT systems, at present or in the future, in either implementation or access, without any restrictions or with a controlled access [Source: Wikipedia]

​

• Interoperability is focused on making sure that the data can be integrated with other data, and can be used with applications or workflows for analysis, storage, and processing. Furthermore, the following principles are identified (for data and its corresponding metadata) [Source: FAIR Data Principles: https://doi.org/10.1038/sdata.2016.18]:
• I1. (Meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation
• I2. (Meta)data use vocabularies that follow FAIR principles
• I3. (Meta)data include qualified references to other (meta)data

Source: Semantic challenges of Open Science, the EOSC perspective. Oscar Corcho (UPM). 2022

​

EOSC Interoperability Framework

Source: European Commission: Directorate-General for Research and Innovation, Corcho, O., Eriksson, M., Kurowski, K., Ojsteršek, M. et al., EOSC interoperability framework – Report from the EOSC Executive Board Working Groups FAIR and Architecture, Publications Office, 2021, https://data.europa.eu/doi/10.2777/620649

​

Successful, effective, homogeneous and sustainable “data” sharing inside and across research communities

​

This architecture will change in the future with the adoption of the EOSC Nodes

Layers on interoperability in EOSC IF

Source: Semantic challenges of Open Science, the EOSC perspective. Oscar Corcho (UPM). 2022

​

The European Interoperability Framework four levels of interoperability

Licenses

Data protection

GDPR

Copyright

Policy

Governance

Community standards

Metadata & Ontologies

Interpretation of meaning & structure

Frameworks

Services

Infrastructure providers

Rules of participation

Bringing communities together

Interoperability & Research Objects

What is the role of Research Objects & RO-Crate?

Interoperability & Research Objects

• In the context of EOSC IF, they use the term Digital Object to refer to the kind of objects that allow binding all critical information about any entity.
• It includes research data, software, scientific workflows, hardware designs, protocols, provenance logs, publications, presentations, etc., as well as all their metadata (for the complete object and for its constituents)
• Research Objects, plus RO-Crate, are examples of Digital Objects.
• It is a good option for sharing information.
• Adaptable to resource’s nature.
• Metadata defined by semantic artefacts

Source: European Commission: Directorate-General for Research and Innovation, Corcho, O., Eriksson, M., Kurowski, K., Ojsteršek, M. et al., EOSC interoperability framework – Report from the EOSC Executive Board Working Groups FAIR and Architecture, Publications Office, 2021, https://data.europa.eu/doi/10.2777/620649

​

Challenges

​

How to assess the FAIRness of a Research Object

​

​

Can we use them for legal interoperability?

FAIR Assessment Approach

Source: Wolniewicz, M. FAIR Research Object Assessment: A landscape analysis.

• Aggregate the FAIRness of each component and the RO itself
• Each component of the RO has its own FAIRness score

​

• We use the tool FAIROs to assess the FAIRness of the Research Objects and other external tools for each resource.
• Datasets: F-UJI
• Ontologies -> FOOPS
• Software -> Custom tool

​

• How can we aggregate all obtained results?
• Score system: Using aggregation metrics (we have applied two)
• Is it a score system enough?
• Provenance needed!

Landscape analysis on Rohub

Source: FAIR IMPACT

Interoperable

​

We analyzed ROs from ROHub.

ROHub uses JSON-LD, which represents knowledge using the RDF standard.

​

RO-Crate relies on Schema.org, a popular vocabulary to describe resources on the Web

​

​

​

Source: Wolniewicz, M. FAIR Research Object Assessment: A landscape analysis.

Landscape analysis on Rohub: Conclusion

Source: FAIR IMPACT

Using an specification like RO-Crate to describe Research Objects gives a more realistic picture of their FAIRness.

Legal Interoperability

Source: Rouchon, O., Kraaikamp, E., Gonzalez, E., Fink Kjeldgaard, A. S., Pedersen Tenderup, N., Davidson, J., Hodson, S., Rettberg, N., & Scharnhorst, A. (2024). D6.2 - Core metadata schema for legal interoperability (Versión v1). Zenodo. https://doi.org/10.5281/zenodo.11104269

We study the impact of the GDPR on legal interoperability in practice.

​

We created a landscape analysis to study the foundation for a legal interoperability framework, by evaluating the relevance of existing widely adopted metadata schemas and controlled vocabularies used in data repositories for the description of legal constraints.

Legal Interoperability

• The RO-Crate specification does not include a specific element designed explicitly for describing access policies.
• There is only one attribute to refer to the entity of the property rights, copyrightHolder
• The RO-Crate specification does not include specific elements to describe data protection and privacy.

Source: Rouchon, O., Kraaikamp, E., Gonzalez, E., Fink Kjeldgaard, A. S., Pedersen Tenderup, N., Davidson, J., Hodson, S., Rettberg, N., & Scharnhorst, A. (2024). D6.2 - Core metadata schema for legal interoperability (Versión v1). Zenodo. https://doi.org/10.5281/zenodo.11104269

General Introduction on Interoperability:

Work done in FAIR-Impact

Source: Oscar Corcho, Universidad Politécnica de Madrid

What is interoperability in the EOSC context?

Practical Use-cases of RO-Crate

FAIR-EASE has received funding from the European Union's Horizon Europe Framework Programme (HORIZON) - under grant agreement No. 101058785.

Use Case 1:�Workflows in Galaxy

Marie Josse, CNRS, FR

Galaxy an open source platform

https://earth-system.usegalaxy.eu/

• Sustainability of Galaxy’s workflows

​

• Reproducibility and interoperability from one platform to another of Galaxy’s workflows

​

• Implemented within Galaxy’s core

Advantages of RO-Crate in Galaxy

Workflows in Galaxy

Exporting a workflow invocation in

RO-Crate

​

• Run the workflow

​

• Galaxy saves all the analytical steps of the workflow in your history

​

​

• Run the workflow

​

• Galaxy saves all the analytical steps of the workflow in your history

​

• Generate the RO-Crate from this history

Exporting a workflow invocation in

RO-Crate

Workflowhub registry

https://workflowhub.eu/

Workflowhub registry

Share and version my workflow

Run my workflow from Workflowhub to Galaxy

A concrete example of a RO-Crate

A concrete example of a RO-Crate

Initial dataset uploaded by the user

A concrete example of a RO-Crate

Data consumed and produced by the workflow

Creation time of the RO-Crate 12th August 2024 at 07:36am GMT

By convention, in RO-Crate the @id value of ./ means that this document describes the directory of content in which the RO-Crate metadata is located. This means that where the ro-crate-metadata.json here is the RO-Crate root

Data produced by the 1st tool and used as input by the following tools

Data produced by the 2nd tool and used as input by the following tools

Data produced by the 2nd bis tool and used as input by the following tools

Data produced by the 3rd tool and used as input by the following tools

Data produced by the last tool and used as input by the following tools

A concrete example of a RO-Crate

Tree structure of the folders and files next to this file the ro-crate-metadata.json

License of the workflow

A concrete example of a RO-Crate

Date of when the workflow was runned

Start of the workflow with some metadata on the initial dataset used for the analysis

1st tool used: Prodigal

2nd tool used: Regex Find And Replace

Reuse a RO-Crate

Readable format by galaxy to import the workflow

All the datasets needed to re-run the exact same workflow

Expand to full workflow form allows you to change all the different parameter of each step before running the workflow

• Ameliorate the RO-Crate profile

​

• Add details on the resources used (CPU, GPU, run time,...)

​

• Documentation

​

• Add the metadata of the tools

​

• Add possibility to directly import a RO-Crate in Galaxy

​

​

Contribution opportunities

A start of contribution to improve the documentation

With FAIR-EASE we wish to build up knowledge on RO-Crate (we’re starting from scratch) to contribute to ameliorate the RO-Crate functionalities of Galaxy

THANK YOU !

​

This a work in collaboration with :

Use Case 2:�Data management

Marc Portier, VLIZ, BE

Use Case 2:

Data management

data.emobon.embrc.eu

combining git + rocrates

roprofiles

achieving LOD publication of data through gh actions and

fair-signposting, linked open data

large file support tech detail

support tech harvesting

human consumption

browsing the web approach

analysis based on linked open data

EMBRC.eu/emo-bon

• European Marine Omics �Biodiversity Observation Network

= A long-term omics observatory �of marine biodiversity

• Shared procedures & datamanagement
• @FAIR-EASE aka pilot 5 �(5.3.1 Marine Omics Observatory)
• The distributed data management �for embrc/emo-bon is
• based on github & RO-Crate
• targeting a Linked-Open-Data publication (5٭ semantic web) of the data

data.emobon.embrc.eu/** (LOD)

The full picture :

Agenda / Ambition - explain this image

• The model / flow behind this BON
• Where and how data is managed
• How the FAIR / LOD publication of this data is achieved
• How that enables open usage and consumption

Note:

• Focus on Plan & Vision
• Implementation at ~60%

The model & Flow

1. Sampling (field & biobank)
2. Sequencing & ENA link
3. Analysing (MetaGoFlow)

Data Management & Storage

• Sensible Units-of-Work (people / flow)
• Distributed & Tracking Changes
• Metadata included (at the source)
• With automated workflows
• DVC.org + S3 extension for large objects

FAIR & Linked Open Data Publication

• gh-actions & workflow cater for automated�downloads, syncs, QC, triple generation, ..
• gh-pages + ro-crate-preview for human exploration
• embedded fair-signposting support discovery
• minor extra "webcontent" for overall "space"

Open & Interoperable Data Reuse

• browsing & analysis
• dedicated lake / index
• # aggregators

«RO-Profiles»

• RO-Profiles
• easy plugable external rules to RO-Crates
• simple identifier → of additional expectations
• Hook for tools:
• assisted entry & templating (inside authoring tool)
• Processing
• validating, QC reports, …
• Step towards Machine-Actionable DMPs
• Allowing to govern some cross-dataset “conformity”

ro-crate-preview.html (demo)

Github-Actions and Github Pages

• https://github.com/vliz-be-opsci/rocrate-to-pages «github action»
• Converts
• From «git repo» (an authored basic ro-crate)
• Into «mini website» (a published ro-crate github.io)

​

• More GH-Actions at emo-bon adding:
• Google Docs import
• ENA Synchronisation and Linking of metadata
• QC Reports & Issues
• Semantic Uplifting (triple generation) of some data
• TO-BE:
• provenance
• LD Fragments: change feeds (LDES) and/or indexes

Can you express your knowledge ?

• Living under "Central Repository" Control
• often first place for "metadata"
• too late → often only at publication time, not continuously during work
• too limited → least common denominator
• lagging on innovation → revisit old records when model is extended

vs.

• «Get it while it is hot» & «Fix it at the Source» (once)
• "specimen by John" - syndrome
• lower the cost of capturing valuable side-wise info (e.g. provenance)
• allow the expert to express and extend
• repository can still harvest and decide on what to search (index)
• reverse control → repositories to adapt under growing innovation

Semantic Public Datasets

RO-Crates as FAIR Digital Objects (FDO)

• (meta)data included with semantic annotation

​

​

• Self-contained «package»
• all parts (files, any format)
• all remote parts (online references)
• all descriptions (semantics)

​

• Addressable on the web
• available as online resources (if needed protected)
• (self) published like mini-web-sites

«Data Entities»

«Contextual Entities»

./ro-crate-metadata.json(ld)

Stepping up Simple → Complex

Distribution 🤝 Uniformisation

html | url | http

one simple block

one individual website

the global knowledge graph

whatever you imagined

one data crate

a cloud of interlinked datasets

On having an «open» Style

• Known Hallmarks of Good (Standards) Design
• Make Simple Things Easy, Make Hard Things Possible
• Have a lots of Spaces Within
• Atomic (do one thing well)
• Collaborative (play along with others)
• e.g. web-standards → HTML, URL, HTTP, RDF
• RO-Crate design principles…
• conform to these best practices
• healthy community that reflects carefully on these aspects

Open Q&A

Moderators:�Rob Carrillo & Gael Lymer

FAIR-EASE has received funding from the European Union's Horizon Europe Framework Programme (HORIZON) - under grant agreement No. 101058785.

Thank you!