STAC and OS Ecosystem

ESIP Winter Meeting 2020

​

Matthew Hanson

​

@GeoSkeptic

SpatioTemporal Asset Catalog specification and software ecosystem

Why?

What is it?

• Defined metadata for geospatial catalogs and assets
• Focus on search and discovery
• Simple and extensible

​

• Static catalogs
• Crawlable so can be indexed
• Dynamic API
• Searchable on arbitrary fields
• OGC Features API with STAC specific extensions

What is it not?

• Full-fledged metadata standard
• Replacement for data provider’s internal metadata
• Single source of truth
• Covering all kind of datasets
• Replacement for ISO 19115, OGC CSW, …

When?

stacspec.org

Who?

stacspec.org

The state of STAC

• Getting wide adoption
• Version 0.9.0-rc1 just released
• Version 1.0 beta in coming months
• 1 billion records
• Split versioning of spec and API

• Catalogs
• There’s always a root catalog
• Contains other catalogs and collections
• Collections
• Can contain other catalogs (called sub-catalogs) to group Items further
• e.g., For Landsat-8 there might be sub-catalog by Landsat Path and Row
• Ultimately contains Items which are similar in some way(s)
• e.g., same instrument, same region containing data from different instruments, etc.
• Items
• A single scene or set of datafiles for a specific location at a specific date and time (when and where)

The specification

​

Catalog Fields

Links

Hierarchical Links

Collection Fields

Item Fields

Item Properties

Item Assets

Content Extensions

EO Extension

EO Band Object

EO Band Common Names

Linking Assets to Bands

STAC Ecosystem

https://github.com/radiantearth/stac-spec/blob/master/implementations.md

• Validate any STAC entity (Catalog, Collection, Item)
• https://github.com/sparkgeo/stac-validator
• Any STAC version
• Service hosted online: https://staclint.com/

STAC Validator

• Browser for navigating static STAC catalogs:
• https://github.com/radiantearth/stac-browser�
• Navigate through Catalogs and Collections, down to Items.

STAC Browser

QGIS Plugin

• Search for Items in any publicly available STAC catalog
• https://github.com/kbgg/qgis-stac-browser
• Show footprints of found Items
• Downloading assets of found Items
• Display assets

​

https://github.com/stac-utils/stac-api

• STAC dynamic API reference implementation
• Node library for a (STAC) API
• Serverless deploy to AWS
• Crawl existing static STAC catalogs to index
• Keep up to date by subscribing to STAC SNS topics

stac-api

​

• sat-api-pg - https://github.com/developmentseed/sat-api-pg
• Postgres backend
• Staccato - https://github.com/planetlabs/staccato
• Java
• Elasticsearch backend
• Pygeoapi - ?
• ​

​

Other API Implementations

​

https://github.com/sat-utils/sat-stac

• Python library for creating or working with STAC catalogs, collections, and items
• Used for creating existing catalogs, such as
• https://landsat.stac.cloud
• https://sentinel.stac.cloud
• http://cbers.stac.cloud/
• See the Jupyter Notebook tutorials

sat-stac

https://github.com/sat-utils/sat-stac-landsat

• Indexes Landsat-8 scenes on AWS
• Transform Landsat metadata to STAC (MTL -> STAC)
• Deployed Lambda function for real-time publishing of new Landsat scenes
• SNS message with entire STAC Item
• See repo for SNS Topic ARN

​

sat-stac-landsat

sat-stac-sentinel

​

https://github.com/sat-utils/sat-stac-sentinel

• Sentinel-2(a/b), Sentinel-1 coming
• Index Sentinel scenes on AWS
• Transform Sentinel metadata to STAC (tileInfo.json -> STAC)
• Deployed Lambda function for real-time publishing new Sentinel scenes
• SNS message with entire STAC Item
• See repo for SNS Topic ARN

https://github.com/azavea/pystac

• Python library for creating or working with STAC catalogs, collections, and items
• More features than sat-stac
• Defines STAC classes more explicitly

PySTAC

https://github.com/sat-utils/sat-search

• Python library for searching STAC complaint endpoints
• Command Line Interface (CLI) as a general purpose tool
• Save search results to be loaded and used later
• Download specific assets from all items
• Specify specific band colors to download those bands
• e.g., Download all “red” and “nir” bands from these Landsat and Sentinel Items

​

​

sat-search

​

sat-fetch

​

https://github.com/sat-utils/sat-fetch

• Works just like sat-search, plus...
• Option to fetch imagery for just AOIs, rather than the entire asset
• Requires GDAL, or use publicly available Docker image
• Warning: sat-fetch is currently out of date

​

Ngorongoro crater: landsat-8 and sentinel-2

Ngorongoro crater: landsat-8 scenes

• ​

https://github.com/sat-utils/sat-api-browser

• Initial prototype
• Web-based UI for searching and saving results from STAC APIs
• Works with any STAC compliant API

​

sat-api-browser

​

https://github.com/pangeo-data/intake-stac

​

• Plugin to Intake
• Turn STAC Item Assets into Dask arrays
• Stack together multiple Assets
• Load STAC search results as Intake catalogs

intake-stac

• NASA Common Metadata Repository
• All NASA data from all DAACs in CMR … (?)
• Element 84 currently working on adding STAC support to CMR
• By October
• Implemented as a proxy to CMR
• Separate STAC API endpoint
• no ingestion
• Targeting STAC 0.8.0 (to be released by end of month)

NASA CMR STAC

E84 Earth Search

Public Search and Discovery API for Earth on AWS Datasets

• Central search catalog and standardized metadata
• Single API that allows searching of STAC metadata for all Earth on AWS datasets
• Powered by sat-api (https://github.com/sat-utils/sat-api)
• Backed by an ElasticSearch index
• Proposed initial datasets
• Sentinel-1
• Sentinel-2
• CBERS-4
• Landsat-8
• NAIP

Landsat-8�Landsat is perhaps one of the most oft-cited and used EO datasets because of its historical archive (Landsat-8 was launched in 2013, but the entire Landsat archive goes back to 1982, with some caveats).

https://earth-search.aws.element84.com/stac

• Underlying data representation is critical to vision of STAC use cases
• Record versioning
• Static catalog maintenance (e.g. removal, updates, etc.)
• Currently dealing with this as using a DB or ElasticSearch and then periodically exporting the static catalog.
• Dealing with different types of remote sensing data
• Currently dealing with this through extensions (e.g. EO, SAR, Drones, Labels)
• Problem is there is overlap in these extensions - don’t have a good way to share fields and provide validation.

STAC challenges

​

• Use the tools - provide feedback!
• Use STAC metadata - provide feedback!

​

• If you’re an expert in a data type:
• Provide input on the extensions
• If we don’t have an extension for, help write it

How can you be part of this?

For updates, releases, news:

​

@GeoSkeptic

@element84