1 of 11

Workflow description of EOSC future SP9�

Benjamin Beuster

Sikt - Norwegian Agency for Shared Services in Education and Research

​

RDA 20th Plenary, Gothenburg - Research Data Alliance 10th Anniversary Plenary meeting, March 21-23, 2023

2 of 11

Goals and steps

​

  • Describe the workflow of the project
  • Use a metadata standard to make the data FAIR (DDI-CDI + DDI-L)
  • Build a tool that describes all important steps of the data integration workflow
  • Researchers from the different domains can evaluate the process, and reuse the process, steps ot content for their own research
  • Integrate the tool with the ESS Data Portal

​

3 of 11

Workflow description of the project

European Social Survey

Repository

Metadata

(Header)

API

(Data)

NetCDF

ERA5 Climate Data

Polygons

(Shapes)

NUTS Geogrpahy

Population

Estimates

(Grid)

Global Human

Settlements

4 of 11

5 of 11

6 of 11

Step

Sub-Activity

Activity

Activity

Sub Activity

Step

Step

Sub Activity

Step

7 of 11

Step

Sub-Activity

Activity

Integrate climate data and air quality data with the ESS

Ingest and prepare data from

ERA5

Call API to get a list of NUTS polygons for relevant regions in NetCDF

Read NetCDF into Pandas dataframes

Ingest and prepare data from EEA Air Quality

Call API to get a list of stations with GPS coordinates in CSV

8 of 11

Substep or Parameter

Substep or Parameter

Step

Step

Substep

Parameter

In

Parameter

Out

Parameter

In

9 of 11

Parameter

Substep or Parameter

Step

Write Integrated data

Merge data from EEA, ERA5 and ESS (Substep)

​

ERA5 and EEA data

(In)

​

ESS8, ESS9, ESS10 data (In)

​

Integrated data

(Out)

10 of 11

11 of 11

ESS Platform

architecture

Process

CDI-XML