Fusion of satellite SAR and passive microwave radiometer observations for automatic sea ice mapping using convolutional neural networks

Tore Wulf, Jørgen Buus-Hinkler, Suman Singha, Matilde Brandt Kreiner

​

Introduction

Synthetic Aperture Radar (SAR) is very useful for sea ice monitoring, because it:

• is unimpeded by clouds
• is independent of solar illumination
• has a high spatial resolution (<100m, Sentinel-1)

​

but SAR imagery is also complex, ambiguous and difficult to interpret.

​

ML-based interpretation of SAR imagery often requires sophisticated computer vision algorithms, e.g. Convolutional Neural Networks (CNNs).

SAR Imagery

Sea Ice Map

CNN

Sentinel-1 HH, August 21st, 2018

Sentinel-1 HH, October 12th, 2019

Sentinel-1 HH, May 16th, 2021

Ice Charts as label data

In an ice chart, each polygon is assigned:

• the total sea ice concentration
• partial concentrations of stages of development (ice type)
• partial concentrations of forms (floe sizes)

​

​

Advantages of using ice charts as label data:

• available at scale
• covers large geographical areas, different sea ice regimes and a diverse variety of sea ice conditions

​

Disadvantages of using ice charts as label data:

• the polygon format is not consistent with reality
• ice charts are based on subjective interpretation
• there are no associated uncertainties

Example Ice Chart from the Greenland Ice Service (DMI)

Tasiilaq, Central East Greenland, March 4th, 2023

Training dataset

1202 unique matches of Sentinel-1 EW imagery and manually produced ice charts from 2018 up to and including 2021, covering Greenland waters (DMI ice charts) and the Canadian Arctic (CIS ice charts).

​

Auxiliary parameters:

• AMSR-2 brightness temperatures for frequencies {6.9, 7.3, 10.7, 18.7, 23.8, 36.5, 89.0} GHz and polarisations {H, V}

Sentinel-1 HH

Ice chart polygon map

Ice chart look-up table

Example scene, May 16th, 2021

Sentinel-1 HV

AMSR-2 brightness temperatures

…

…

6.9 GHz, H-pol

18.7 GHz, H-pol

89.0 GHz, H-pol

Fusion of Sentinel-1 and AMSR-2

Sentinel-1 SAR imagery has a high spatial resolution (<100m), but the backscatter signatures can be ambiguous.

​

AMSR-2 brightness temperatures have a coarse spatial resolution (tens of kilometers), but generally reliable discrimination between sea ice and open water.

​

AMSR-2 observations are complementary to Sentinel-1 and help resolve ambiguities in the backscatter signatures.

Example scene, February 20th, 2021

Sentinel-1 HH

Sentinel-1 HV

Ice Chart - Sea Ice Concentration

AMSR-2 36.5 GHz, H-pol

DMI-ASIP CNN Overview

Overview:

The CNN follows a U-Net-like encoder-decoder structure. The encoder network (~22m params) consists of 6 stages, each comprised of multiple inverted residual blocks, for multi-scale feature extraction.

​

The decoder network (~1m params) mirrors the general structure of the encoder. The CNN has three decoders; One for each sea ice parameter - SIC, SoD and FLOE.

​

The decoders output pseudo-probabilistic distributions over the predefined set of classes for each sea ice parameter.

​

Implementation details:

Preprocessing:

• Data resampling (to 80m grid spacing), data scaling, patch extraction, etc.

Regularization:

• Common data augmentation
• Stochastic depth
• Label smoothing

Optimization strategy:

• Loss: cross-entropy with unitary weighting of the per-objective losses
• Optimizer: AdamW (initial learning rate 3e-4)
• Multi-step learning rate scheduling
• Mixed precision training

Trained from scratch on a single NVIDIA A100.

Sentinel-1

+

AMSR-2

SIC

SoD

FLOE

Sea Ice Concentration - Results

Ice Chart SIC vs. CNN SIC

20 carefully selected examples from the dataset have been set aside for DMI-ASIP performance evaluation.

​

DMI-ASIP achieves R²-scores of 94% (freezing season), 92% (melting season) and 93% (overall) when evaluated against the test dataset at pixel level.

Example scene, Jan 24th, 2020

Sentinel-1 HH

Sentinel-1 HV

Ice Chart - Sea Ice Concentration

DMI-ASIP - Sea Ice Concentration

Sea Ice Concentration - Uncertainty

Modern neural networks tend to be poorly calibrated, i.e. the confidence of the output is not aligned with the accuracy of the output.

​

We mitigate this issue by using label smoothing (during training) and temperature scaling (post-processing).

Sentinel-1 swath, March 7th, 2023

DMI-ASIP - Sea Ice Concentration

Sentinel-1 HH

CNN - Confidence

Reliability Diagram

Sea Ice Concentration - Pan-Arctic

Mosaics based on Sentinel-1 HH EW and IW scenes from March 1st to March 11th, 2023

Sentinel-1 HH

DMI-ASIP - Sea Ice Concentration

Sea Ice Concentration - Svalbard example

Sentinel-1 HH mosaic (March 1st to March 11th, 2023)

Sentinel-1 HH

Sentinel-1 HV

DMI-ASIP - Sea Ice Concentration

06:48 UTC, March 11th, 2023

OSI SAF - Sea Ice Concentration

12:00 UTC, March 11th, 2023

Sea Ice Concentration - Generalization to Antarctica

How well does the DMI-ASIP model generalize beyond the training dataset?

​

When evaluated against a small, limited test dataset from Antarctica, the DMI-ASIP model achieves an R²-score of 89%, compared to 93% on the Arctic test dataset. Also, the calibration suffers as the DMI-ASIP model becomes more over-confident.

Reliability Diagram

Ice Chart SIC vs. CNN SIC

Example scene, January 13th, 2019

Sentinel-1 HH

Sentinel-1 HV

Ice Chart - Sea Ice Concentration

DMI-ASIP - Sea Ice Concentration

Stage of Development -

Preliminary results

Sentinel-1 HH

Ice Chart - Stage of Development

DMI-ASIP - Stage of Development

November 12th, 2020

February 20th, 2021

February 3rd, 2021

Stage of Development - Pan-Arctic (preliminary results)

Mosaics based on Sentinel-1 EW and IW scenes from March 1st to March 11th, 2023

Sentinel-1 HH

DMI-ASIP - Stage of Development (Ice Type)

Links & DOIs

AI4Arctic Sea Ice Dataset version 2:

https://doi.org/10.11583/DTU.13011134.v3

​

AI4Arctic Sea Ice Challenge Dataset:

https://doi.org/10.11583/DTU.c.6244065.v2

​

DMI-ASIP Greenland

https://doi.org/10.48670/moi-00122

​

DMI-ASIP Antarctica

• Coming soon!

​

​

​

Contact: twu@dmi.dk