International Commons

Benchmarks

Working Group Session

Karen Payne, ISC World Data System

Mark Leggott, Research Data Canada�Andrew Treloar, Australian Research Data Commons

June 17, 13:00 UTC

A WG within the GORC IG

gorc_benchmarking@rda-groups.org

International Commons

Benchmarks

Working Group Session

Karen Payne, ISC World Data System

Mark Leggott, Research Data Canada

June 17, 13:00 UTC

A WG within the GORC IG

gorc_benchmarking@rda-groups.org

Agenda Items

1. Welcome
2. Comments in Council about the Case Statement
3. Call for co-chairs outside the Global North
4. Recap from P17
5. WG collaborative working spaces
6. Database and Submission forms
7. Next steps
1. Recurring meetings

​

​

WG Rolling Notes

https://bit.ly/3xljZ03

​

Comments from Council on Case Statement (1 of 4)

RDA Council acknowledges the value of the Global Open Research Commons International Benchmarking WG and would like recognise and endorse the Group subject to specific revisions.

​

• Include a clear definition of benchmark and strongly consider identifying an alternative term, e.g. criterion, interoperable resources, etc. and focus the group’s effort on documenting the attributes of a successful research commons rather than providing judgements on the quality of different research commons.

​

​

Define Benchmarks

“...this WG will establish benchmarks to compare features across commons”

“...features across science clouds”

“...what we would expect to find in a mature research commons”

“The WG will collectively decide what constitutes a benchmark.”

“...the existence of a feature or service and is comparable across organizations”

“ ...the most important features and the minimal elements...“

​

​

• A well defined decision making process
• A consistent and openly available data privacy policy
• Federated Authentication and Authorization infrastructure
• Community supported and well documented metadata standard(s)
• A workflow for adding and maintaining PIDs for managed assets
• A mechanism for utilizing vocabulary services
• A process to inventory research artefacts and services
• An Open Catalogue of these artefacts and services
• A proven workflow to connect multiple different research artefact types (e.g. data and publications; data and electronic laboratory notebooks; data and related datasets)
• A mechanism to capture provenance for research artefacts
• Mechanisms for community engagement and input; an element or scale for inclusion

Proposed definition for Benchmark

​

• A standard, feature or point of reference from which information can be documented, or measurements or comparison may be made
• A property of a system that indicates a class, type or level of functionality or performance

​

Comments from Council on Case Statement (2,3 of 4)

• Clarify the proposed review process and explain how the group intends to describe areas of RCs where global interoperability can be considered and be inclusive of any infrastructure that defines themselves in the “Research Commons” context to avoid impressions of any potential disenfranchisement.​
• Take into consideration the TAB feedback on ambition and geographical scope.​

​

​

Target List of Commons

Pan National:

• European Open Science Cloud
• African Open Science Platform
• Nordic e-Infrastructure Collaboration
• the Arab States Research and Education Network, ASREN

National:

• German National Research Data Infrastructure (NFDI)
• DANS
• GAIA-X
• UK JISC Open Research Framework
• China Science and Technology Cloud (CSTCloud);
• Australian Research Data Commons
• NDRIO (Canada)
• NII Research Data Cloud (Japan)
• KISTI (South Korea)

We propose to reduce scope: remove ~20 domain commons except IVOA and GEO

Representation in Europe, Africa, ME, Asia-Pacific

​

No explicit representation: Americas (but captured in domain commons)

​

Comments from Council on Case Statement (4 of 4)

• Clarify the recommendation to be produced - the case statement points to “Supporting Outputs”; RDA working groups are expected to produce endorsable and adoptable “Recommendations”.​

CS: The Adoption Plan will include the following:

• Integration of the benchmarks into the Typology and larger GORC roadmap being created by the parent IG.
• Integration/intersections with the CODATA GOSC work, including use cases.
• Add “The WG will create an organizational structure and framework that can capture observations of benchmarks, KPIs and implementations currently in use or expected.”
• Database of observations + contributions to IG Roadmap

Call for Chairs

Call for Chairs outside the Global North

​

​

Next Steps

1. Set up recurring meetings
2. Submit response to Council’s Feedback to Case Statement; update with new chairs
3. Session proposal for Plenary 18
4. Review properties to be gathered
5. Mock up and test intake form and database
6. Seed database
7. Conduct lit review