25/08/2022
Tech Talk: Making AuScope Portal Play FAIR
Vincent Fazio
AuScope AVRE�Senior Engineer
CSIRO
vincent.fazio@csiro.au
ARDC is enabled by NCRIS.
2
Guiding principles that aid in knowledge discovery
�Helping users of scientific data, both humans and machines (e.g. workflows, algorithms)
Improving discovery, access, integration and analysis
Four facets of FAIR:
2. Why be FAIR?
3
It is very abstract, but necessary in a practical sense, answering questions such as:
Accurate academic citation is vital for most researchers and must be supported by infrastructure
Project sponsors also need to be recognised
3. AuScope Discovery Portal
4
AuScope – Australia’s premier provider of research tools, data, analytics and support to Australia’s geoscience community
AuScope Discovery Portal is part of AuScope’s AVRE (Australian Virtual Research Environment)
Explore, display and download Australian geological data
Data sourced from:
Types of data:
4. Is AuScope Portal FAIR Enough?
5
Challenge
FINDABLE:
ACCESSIBLE:
INTEROPERABLE?
REUSABLE:
5. AuScope Portal Metadata Architecture
6
NCI Geonetwork4
Geoscience Australia Geonetwork4
AuScope Portal API
server
AuScope portal in browser
AuScope
Geonetwork4
iso19139
Public datasets
Harvested via Geonetwork OGCWxS Harvester (OGC WMS GetCapabilities)
Builds a cache of catalogue dataset records (OCG CSW GetRecords)
Display geospatial
datasets and metadata
Metadata catalogues
OCG WMS server*
Custom datasets via OCG WMS GetCapabilities
* - Any of: Geoserver3, ESRI GIS Server2, NCI GSKY1
OCG WMS/WFS server*
OCG WMS/WFS server*
OCG WMS/WFS server*
OCG WMS/WFS server*
6. AuScope Portal modifications
7
Response
Requires:
* - Any of: Geoserver3, ESRI GIS Server2, NCI GSKY1
7. Geonetwork: pros of metadata harvesting
8
Response
Harvesting metadata via Geonetwork OCGWxS harvester has some advantages:
8. Geonetwork: cons of metadata harvesting
9
Response
Harvesting has a couple of drawbacks:
9. Deficiencies to be rectified
10
Challenge
This means that our metadata may be missing things such as:�
?
10. One solution - Templates
11
Response
During harvesting Geonetwork can use templates to insert metadata
Some elements are copied directly from “GetCapabilities” response
TIP: Many other elements require a stub to be present in the template to trigger copying
e.g. WMS URLs
Template Loading
https://www.geonetwork-opensource.org/manuals/trunk/en/install-guide/loading-samples.html
ISO19139 template
GetCapabilities
New ISO19139 record
Merge
11. How to use Geonetwork’s templates
12
Response
Procedure:
1. Load template
2. Edit template, add licensing details, stubs etc.
3. Select template in OCGWxS harvester configuration
12. Another problem
13
ISO19139 is an older standard from 2007
Challenge
Being older there are many things it cannot easily represent
e.g. project sponsor, end-user license, distributor license …
By default Geonetwork creates harvested records in ISO19139 standard
13. Solution: upgrade harvesting to ISO19115-3:2018
14
Response
Creation of harvested records in ISO19115-3 XML format is supported by Geonetwork’s ISO19115-3 plugin
�
https://geonetwork-opensource.org/manuals/3.10.x/en/annexes/standards/iso19115-3.2018.html
XSLT
https://en.wikipedia.org/wiki/XSLT
ISO19115-3 plugin
http://www.cnig.gouv.fr/wp-content/uploads/2015/06/titellus_fx_prunayre_iso19115-3_inspire2015.pdf
If the template is ISO19115-3 then the created records will be ISO19115-3
ISO19115-3 template
GetCapabilities
New ISO19115-3 record
Merge
14. How to upgrade to ISO19115-3:2018?
15
Response
Create and configure new ISO19115-3 template before harvesting
Select new template in OCGWxS harvest config
https://geonetwork-opensource.org/manuals/3.10.x/en/annexes/standards/iso19115-3.2018.html
ISO19115-3 plugin
http://www.cnig.gouv.fr/wp-content/uploads/2015/06/titellus_fx_prunayre_iso19115-3_inspire2015.pdf
ISO19115-3 plugin is built into Geonetwork v3.8+
15. Dealing with hand curated records – STEP 1
16
Challenge
How to upgrade hand curated ISO19139 records to ISO19115-3?
STEP1: Export files as XML/ZIP to local filesystem and then import them back again using XSLT
16. Upgrading hand curated records – STEP 2
17
Response
How to upgrade hand curated ISO19139 records to ISO19115-3?
STEP2: Re-import XML/ZIP file, use provided XSLT to convert to ISO19115-3
TIP: For Geonetwork v3.12.5/v4.2.0 and earlier the record will still have “ISO19139” in the metadata field�Update this metadata field to “ISO19115-3:2018”
17. Lessons Learned.
18
Response
In terms of knowing a record’s metadata standard the Geonetwork UI can be deceptive.
Check the raw XML or use the tick boxes in the editor board to confirm your record’s standard.
If you have metadata-poor sources harvesting will not allow you to completely bypass the burdens of curation.
You will still need to maintain templates.
Geonetwork is complex. It requires experimentation to find the most suitable configuration.
Questions from the Audience?
19
References.
20
OGC Server Implementations�
Thank you!
Acknowledgements:�
* AuScope Ltd
* AuScope AVRE MAINTAIN Team
ARDC is enabled by NCRIS.