1 of 31

WP3 Status

​

Horizon 2020 Research and Innovation Programme

Grant Agreement # 857647

2 of 31

WP3

  • T3.1
    • Some issues open in github to be addressed
  • T3.2
    • Hackathon provided a good experience on scaling and limitations and we are addressing the identified issues
    • Service QA now being fulfilled by several TSs SCIPION (IM), SAPS (EC3), MSWSS (IM)
    • Service QA deployment with Kubernetes is now supported
    • Working on the missing bits of FAIR assessment integration
  • T3.3 last deliverable D3.5 from T3.3 ready for review
    • Missing the feedback from FAIR tools integration in the pipelines
    • Also missing issuance of badges for FAIR

1

www.eosc-synergy.eu - RIA 857647

3 of 31

WP3

  • Exploitation
    • Other communities
    • Our own software

​

  • WP6 bridge
    • Training / MOOC content

2

www.eosc-synergy.eu - RIA 857647

4 of 31

​

T3.1 status

​

EOSC-Synergy AHM @Ibergrid 2022

Mario David

EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647

5 of 31

Vision and high level architecture

The vertical view represent the high level architecture of the Quality Assurance developed in EOSC-Synergy.

4

  1. Quality Assurance baseline documents: Detail a set abstract quality metrics.

​

​

​

  • SQA as a Service (SQAaaS): Implement Quality assessment based on the baselines documents.

​

​

​

  • Issue badges: as a proof that a given Software or Service has passed the Quality Assurance criteria.

www.eosc-synergy.eu - RIA 857647

6 of 31

Plans and discussion until the end of the project - Software QA baseline

  • One more release - v4.1
  • 11 open issues: https://github.com/indigo-dc/sqa-baseline/issues - 7 Tagged “question”
    • (for After Synergy) Release, management and support - not automated
      • [ENHANCEMENT] Release conventions #17
      • [ENHANCEMENT] How to deal with unsupported software #18
    • (for After Synergy) New annex on recommendations to achieve SQA
      • [ENHANCEMENT] Cookiecutter-like for creating new repositories #15
      • [ENHANCEMENT] Inclusion of Sustainability Criteria #38
    • Discuss if we add it
      • [ENHANCEMENT] add LicenceURL #68
        • Wilko => Yes, make it a MUST, I think: FAIRness, FAIR Signposting, assessment tools, all use it. This will make it machine actionable. Example: SignPosting: relationtype: license => The target of the link is the URI of a license that applies to the resource that is the origin of the link.
      • [ENHANCEMENT] Should metadata be a MUST? #87
        • Wilko => Yes, I think so. However, what format? That should that also be indicated?: codemeta?
      • [ENHANCEMENT] Criterion for document linting #90
        • Wilko => File under the Q.Doc, I think? For Linting is more than just Style only?
    • 3 issues solved in branch “devel” including:
      • [ENHANCEMENT] Re-organize the current criteria breakdown #91

5

www.eosc-synergy.eu - RIA 857647

7 of 31

Software QA baseline v4.1 - Quality model

6

www.eosc-synergy.eu - RIA 857647

8 of 31

Plans and discussion until the end of the project - Service QA baseline

  • One more release - v2.1
  • 8 open issues: https://github.com/EOSC-synergy/service-qa-baseline/issues
    • (for After Synergy) 3 Tagged “question”
      • [ENHANCEMENT] Accessibility for a service #22
        • Discussion: if this can be automated, after checking the thread and references, it seems it can’t be automated samuelbernardo.mail@gmail.com
      • [ENHANCEMENT] Check https://12factor.net/ 12 factor app #24
      • [ENHANCEMENT] Potential Tool for SvcQC.Api02 (Security Testing of APIs) #37
        • Discussion: this is a tool, that maybe used in SQAaaS, we will include this in an Annex if/when it’s in the tools for the jenkins pipelines
    • 3 already solved in branch “dev”
    • TO BE DIscussed
      • “Add annex with implementation and tools” similar to the SQA baseline annexes:
      • Discuss the contents of this annex since we should not repeat what is in the SQA baseline already #49
        • Wilko => Perhaps mention somewhere that some tools from the SQAaaS can be applied to these sections as well?
        • Wilko => Not all sections can de provided with a tool I suppose?
      • [ENHANCEMENT] Consider IaC criteria to avoid vendor lockin #51

​

7

www.eosc-synergy.eu - RIA 857647

9 of 31

Service QA baseline v2.1 - Quality model

8

www.eosc-synergy.eu - RIA 857647

10 of 31

Integrating Software QA and Service QA:

With open-source code

9

9

SW source code

static tests

SW source code

unit tests

Build artifacts

Deployment

OR

auto deployment

Running service

dynamic tests

Auto delivery

(Publish artifacts)

Auto deployment

Production service

Badge issuing

SQA

ServiceQA

Badges

Intermediate

www.eosc-synergy.eu - RIA 857647

11 of 31

Integrating Software QA and Service QA:

With Commercial/Closed source SW

10

Deployment

OR

auto deployment

Running service

dynamic tests

Production service

Badge issuing

ServiceQA

Badges

Intermediate

www.eosc-synergy.eu - RIA 857647

12 of 31

​

T3.2 status

​

EOSC-Synergy AHM @Ibergrid 2022

Pablo Orviz

EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647

13 of 31

Service validation

  • Already done:
    • TSs fulfilling SvcQC.Dep
      • SCIPION (IM), SAPS (EC3), MSWSS (IM)
  • Finish k8s support on the SQAaaS platform
    • Successfully tested with JePL (JePL-k8s-test, JePL-helm-test)
    • Missing bits:
      • Template for k8s script in SQAaaS API (version 2.11.0)
      • Helm tool support in SQAaaS tooling (O3AS TS) (version 1.8.0)
  • Support functional testing (SvcQC.Fun)
    • SCIPION & WORSICA volunteered
    • Change current logic in SvcQC.Dep (currently cleans up the deployment, need to move this step to the latest step in the pipeline)

www.eosc-synergy.eu

14 of 31

FAIR assessment

  • Integrate FAIR tools (FAIR-EVA, F-UJI) into SQAaaS’ QAA
    • Already there:
      • SQAaaS web (staging) provides the input forms
      • SQAaaS API spec already contains the generic properties for the FAIR assessment (PID, service metadata protocol & endpoint, matchmaking rules, ..)
    • Missing bits: implement SQAaaS API spec in API server
      • Adding FAIR properties to POST /pipeline/assessment
      • Logic to compose the pipeline for FAIR assessment
      • Test integration of FAIR-EVA & F-UJI output validator plugins
      • Support for external matchmaking policies

www.eosc-synergy.eu

15 of 31

SQAaaS platform

  • Stabilizing/strengthen the service at sqaaas.eosc-synergy.eu
    • Several issues emerged from the Hackathon experience
      • Move away from platforms that impose limits (GitHub, Docker Hub, ..)
    • Key to improve usability & disseminate our work (prospective hackathons, hands-on, tutorials)
  • Improve documentation at docs.sqaaas.eosc-synergy.eu, including:
    • CI/CD tools already supported for each programming language
    • How to customize each one to e.g. add exceptions to rules (default rules imposed by linters should not be demanded)
  • Finish the k8s recipes for deploying your own SQAaaS platform
    • Only beneficial for use cases that want to implement their own quality approach (incl. custom criteria & badges)
  • Journal paper
    • Including the experience of reference TSs in WP4 (incl. survey)

www.eosc-synergy.eu

16 of 31

JePL

  • Improve documentation to align with latest features
  • Carry out a next release including:
    • [sw criteria] Maven support (incl. validation with SAPS)
    • [sw criteria] Pull images before launching the pipeline
    • [srv criteria] Support for IM & EC3
    • [srv criteria] Support for k8s (kubectl, helm)

www.eosc-synergy.eu

17 of 31

​

T3.3 status

​

EOSC-Synergy AHM @Ibergrid 2022

Fernando Aguilar

EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647

18 of 31

Technical FAIR Framework: Implementation

  • Development of FAIR assessment Tools:
    • FAIR EVA (developed under EOSC-Synergy umbrella)
    • F-UJI
  • Automatic tests to check FAIR indicators.
    • Defined by FAIRsFAIR & RDA
  • Connected to repositories and thematic services

17

www.eosc-synergy.eu

19 of 31

Technical FAIR Framework: FAIR EVA

18

www.eosc-synergy.eu

20 of 31

Technical FAIR Framework: Validation

  • Analysis of FAIR (automatic) assessment tools
    • FAIR Evaluation Services, FAIR enough, FAIR checker.
  • Contextualize the FAIR environment
    • FAIR metrics EOSC Task Force, RDA, Signposting, FAIR Data Point
  • Adopting JePL: Docker, OpenAPI, Automatic testing, deployment.

19

DSpace 7 deployment with data ingestion and FAIR assesment

www.eosc-synergy.eu

21 of 31

Technical FAIR Framework: Monitoring

  • FAIR quality criteria to provide (bronze, silver, gold) badges
  • Technical proposal. Communities to configure the list of criteria
  • Difficulties to compare among FAIR tools:
    • Semantics to link metrics - indicators - principles
    • Simple Knowledge Organization System (SKOS)
  • ​

20

www.eosc-synergy.eu

22 of 31

T3.3: Missing points

  • Badge providing
    • Some tests finished with plugins
    • Complete the web interface and connection with badgr
  • Final status of thematic services in terms of FAIR
  • Test complete pipeline with WORSICA - Dataverse - FAIR assessment

21

www.eosc-synergy.eu

23 of 31

T3.3: Missing points

  • Badge providing
    • Some tests finished with plugins
    • Complete the web interface and connection with badgr
  • Final status of thematic services in terms of FAIR
  • Test complete pipeline with WORSICA - Dataverse - FAIR assessment
  • Engage some external community in using the SQAaaS assessment capabilities

22

www.eosc-synergy.eu

24 of 31

www.eosc-synergy.eu

25 of 31

​

WP3 perspectives for future projects

​

EOSC-Synergy AHM @Ibergrid 2022

EOSC-SYNERGY receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857647

26 of 31

T3.1 Perspectives for future projects

  • Check what can be taken (makes sense) from the EOSC Task Force:
    • Infrastructure for quality research software
    • Summary of Quality Attributes all papers that were reviewed:

​

  • From the new projects (DT-Geo, AI4EOSC, eventually others)
    • Criteria on “Workflows”, “Metadata”, “Provenance”
      • May depend on user communities
      • May be similar to data FAIR assessment

25

www.eosc-synergy.eu - RIA 857647

27 of 31

T3.2 Perspectives

  • QAA module (badges, quality reports, ..)
    • Integration in the EOSC ecosystem
      • Not just as a service in the portal
      • Information for EOSC service management
      • Support for EU projects
    • Social coding platforms
      • Trigger QAA from GitHub actions, GitLab CI, ..
      • Share Badgr’s badges in code repositories (much like shields.io)
  • Pipeline as a Service module
    • Reach out to more research communities/infrastructures ⇝ jumpstart in CI/CD without any previous knowledge
  • JePL: major release v3
    • Including built-in support for service deployment

www.eosc-synergy.eu

28 of 31

T3.2 Perspectives

  • Support for further programming languages
    • Currently Python and Java
  • Improved support for security assessments
    • Support more security tools
  • Implement further quality criteria from the baselines
    • Both from software and services
    • Further levels of badges Platinum, Titanium etc
  • Integration with EOSC
    • EOSC portal etc
  • Support further deployment tools/methods in Service QA
    • Beyond Kubernetes, IM, EC3 etc

www.eosc-synergy.eu

29 of 31

T3.3 Perspectives for future projects

  • From the new projects (DT-Geo, AI4EOSC, eventually others)
    • Improvements of FAIR EVA (scalability, performance)
    • Extend the use of semantics to integrate tools
    • Semantics for Badge - Transparency
  • Extend the use of FAIR EVA to check data in detail
    • GBIF Apparent Quality Index
    • Others
  • Involving in (complex) FAIR context
  • Integration of further tools on SQAaaS

28

www.eosc-synergy.eu - RIA 857647

30 of 31

Transversal

  • Support further user requirements in a moving field
  • Documentation
  • Training both MOOC and conventional
  • Hackathons

29

www.eosc-synergy.eu - RIA 857647

31 of 31

www.eosc-synergy.eu