Satellite Embedding Deep Dive

Ujaval Gandhi

ujaval@spatialthoughts.com

Hands-on with the Satellite Embedding dataset in GEE

CC BY 4.0

Satellite Embedding Deep Dive

This workshop is designed to help you get started with the Satellite Embedding Dataset in GEE and use it for a variety of applications.

• Visualizing Embeddings
• Unsupervised Classification
• Crop Type Classification and Surface Water Mapping
• Supervised Classification
• Mapping Mangroves and Urban Tree Cover
• Object Detection with Similarity Search
• Mapping Brick Kilns and Solar Farms

bit.ly/satellite-embedding-workshop

This slide-deck

Get Access to the Workshop Video

Get Access to the Code Repository

• Get the workshop repository → Click this link to open Google Earth Engine code editor and add the repository to your account.
• If successful, you will have a new repository named users/ujavalgandhi/GEE-Satellite-Embedding in the Scripts tab in the Reader section.
• If you do not see the repository in the Reader section, click Refresh repository cache button in your Scripts tab.

​

Introduction

• Background in GIS and Remote Sensing
• Intern at Indian Institute of Remote Sensing (IIRS), Dehradun, India
• MS in Geospatial Information Engineering from University of Wisconsin - Madison, USA
• 15 years of professional experience
• One of the early employees at Google Inc.
• Moved to India in 2006 and established the maps team
• Migrated internal teams to QGIS
• Led the GIS and Aerial Imagery team in India from 2007-2019
• Developed expertise in Google Earth Engine and trained 1000+ scientists and researchers across India
• Presently
• Left Google in 2020 to work on startup Spatial Thoughts.
• Building open-source content and helping people master advanced geospatial skills.

Ujaval Gandhi

Introduction

• Masters in Remote Sensing
• Interest in Automating Workflows
• Training Associate
• Class logistics
• Project support

​

Vigna Purohit

Introduction to the

Satellite Embedding Dataset

Satellite Embeddings Dataset

• New analysis-ready global dataset produced by Google DeepMind’s AlphaEarth Foundations model.
• The model ingests time-series of many different datasets to produce a unified digital representation of each pixel - called ‘embeddings’.
• Produced for each year from 2017-2024 at 10m resolution.

Image © Google

Training Sources

• The ‘embeddings’ at each pixel are created from
• All sentinel-2, sentinel-1, ScanSAR, Landsat-8 images for the entire year
• + climate variables from ERA5 for the entire year
• + elevation data, gravity fields, text information etc.
• Input data stream of thousands of dimensions are compressed into 64-dimensional output.

​

Image © Google

Understanding Embeddings

Embeddings are compressed representations of higher dimensional inputs.

A good mental model to understand embeddings is Principal Components Analysis (PCA).

Image © Google

Understanding Embeddings

Embeddings are “unit-length” (i.e., they have a magnitude of 1 (scaled from -1 to 1)

They can be thought to represent a coordinate on a 64-dimensional “unit-sphere”

Image © Google

Understanding Embeddings

Since embeddings are continuous and stored in an image format, you can treat each axes of the embedding space just like spectral bands for visualization and analysis.

But remember that Embeddings are learned features and cannot be interpreted in terms of physical measurements.

Every axes can include contributions from space, time, and measurements.

Image © Google

Understanding Embeddings

Embeddings encode the spatial context around each pixel.

Each pixel uses features in a 1.28 km x 1.28 km window

Image © Google

Understanding Embeddings

Embeddings encode the temporal context for each pixel

Each pixel uses the full calendar year’s data (per sensor)

Understanding Embeddings

Embeddings are designed to be linearly composable

Pyramiding works as expected in Earth Engine. i.e. a 20m pixel is created by averaging four 10m pixels and represent the properties of the pixel at that resolution.

Image © Google

Satellite Embeddings Dataset

• The result is provided as an analysis-ready dataset in Earth Engine.
• Includes temporal context for the entire year.
• Include spatial context of surrounding pixels.
• No need for cloud-masking, spectral transformation or other corrections.
• 1 image per year with 64-bands of information.
• The images can be used in a variety of downstream applications.
• Replace image inputs or engineered features into classification or regression workflows
• Automatic clustering
• Change detection
• Search similar objects or regions

Visualizing Embeddings

Unsupervised Classification

Unsupervised Classification

• Embeddings are very well suited for unsupervised learning.
• Clustering algorithms group pixels with similar embedding vectors into the same cluster.
• You can choose small number of clusters for more general classes and higher number of clusters for more fine-grained separation.
• Apply local knowledge, expert rules or field samples to automatically label clusters.

Image © Google

Crop Type Mapping

• One of the harder problems in Remote Sensing
• Requires lots of data pre-processing
• Need to add temporal context
• Need to add spatial context
• Need to do sensor fusion to incorporate radar and optical data
• Need to incorporate climate variables
• Requires high-quality labels
• Need to obtain high-quality samples covering all crops in the region
• Need to have samples for specific time periods

​

Using Satellite Embedding

Use embeddings directly for crop classification tasks

Need to add temporal context

✅ Includes annual trajectories

Traditional Approach

Embeddings

Need to add spatial context

✅ Encodes spatial context around each pixel

Need to do sensor fusion to incorporate radar and optical data

Need to model response to climate variables

✅ Includes Sentinel-2, Landsat-8 (Optical) + Sentinel-1 (Radar) + ScanSAR (Radar)

✅ Includes ERA5-Land climate variables

Select a region and apply a crop mask

Extract random samples for clustering

Perform unsupervised clustering

Labeling Clusters

• Clustering algorithm will group pixels of same crops into the same cluster.
• Clusters can be labeled using field labels or aggregate statistics
• Using field labels
• A small number of field samples can be used to identify crops.
• Overlay the samples on the clusters and select cluster(s) for belonging to each crop.
• Using aggregate statistics
• Most countries/regions regularly publish aggregate crop statistics.
• Clusters can be identified by comparing cropped areas with cluster areas.
• Effective technique in regions with a few distinct crops.

Detected crop map by comparing with aggregate statistics

(left) crop map from satellite embeddings (right) crop map from CDL

Surface Water Mapping

• Apply unsupervised clustering methods on embeddings image.
• Calculate Sentinel-2 based MNDWI for each cluster.
• Automatically label the cluster with highest MNDWI as water.
• Fully automated unsupervised approach to surface water mapping.
• Code in Supplement > Unsupervised_Classification_Surface_Water

Supervised Classification

Supervised Classification

• You can substitute embeddings for other spatiotemporal features (e.g., composites, harmonics) in existing workflows .
• Embeddings are optimized for low-shot classification.
• Performs well with a small number of samples
• k-Nearest Neighbors (kNN) classifier is particularly suited to classify embedding features.
• Training samples are used to “partition” or cluster the embedding space, assigning a label for each pixel based on the sample of its closest neighbor in the embedding space.
• Tree-based classifiers also work well for regression workflows.

Supervised Classification

• Less number of high-quality samples perform better than large number of training data.
• Start with small number of samples and add counter-examples in areas of misclassification
• Helps the kNN classifier define a tighter cluster more specific to the class of interest
• Embeddings unlock a range of new classes that can be mapped
• Features requiring spatial context (dirt roads vs. bare soil)
• Vegetation with a specific pattern (forest plantation, golf courses)
• Features with temporal context (evergreen vs. deciduous forest)
• …

Mapping Mangroves

We will collect training samples by visual inspection of image.

​

Mapping Urban Tree Cover

• Fine-grained landcover classification can be achieved using embeddings as features.
• Create classifications with legends that previously may not have been possible.
• Use mean embeddings to use polygons for training samples.
• Code in Supplement > Supervised_Classification_Urban_Tree_Cover

Object Detection with Similarity Search

Object Detection

• Object detection is a hard problem for pixel-based methods.
• Requires custom models
• Need to train custom models for each object of interest
• Need to use deep learning libraries
• Requires a different computation environment
• Need GPUs for training and inference
• Expensive to scale
• Need high-resolution imagery. CNNs cannot ‘see’ the objects in medium-resolution imagery

​

Using Satellite Embedding

Use similarity search with embeddings for object detection

Need to train custom models for each object of interest

✅ Embeddings can be generalized across a wide variety of objects

Traditional Approach

Embeddings

Need to use deep learning libraries

✅ Uses simple similarity search

Need GPUs for training and inference

Need expensive high-resolution imagery

✅ Can be run in familiar computing environments like Code Editor

✅ Works on pixel-based embeddings created from open-source imagery

Similarity Search

• By comparing the embedding vector for a sampled location with every other location, we can measure their similarity
• Cosine similarity is the most commonly used measure to find similarity between vectors.
• Similar to ‘Spectral Angle’ metric used in Remote Sensing
• The result is always between -1 and 1
• 1 → vectors point in the same direction (perfectly similar)
• 0 → vectors are orthogonal (no similarity)
• -1 → vectors point in opposite directions

Image © Google

Mapping Brick Kilns

• Many urban areas in South Asia have facilities to make clay-fired bricks for construction.
• Often unregulated and unlicensed
• Associated with severe air pollution, soil degradation and human-rights violations.
• Mapping brick kilns is of high interest for regulatory enforcement. [Learn more]
• The unique geometric and spectral features of brick kilns allow satellite embeddings to separate them from other features easily.

Select a region of interest

Add Reference Location(s)

Calculate Similarity (brighter areas are more similar)

Apply a threshold and visualize the matches

Mapping Grain Silos

See our tutorial on Similarity Search with Satellite Embedding Dataset

​

Resources

• Introduction to the Satellite Embedding Dataset
• Unsupervised Classification with Satellite Embedding Dataset
• Supervised Classification with Satellite Embedding Dataset
• Regression with Satellite Embedding Dataset
• Similarity Search with Satellite Embedding Dataset

References (Satellite Embeddings)

• Brown, C. F., Kazmierski, M. R., Pasquarella, V J., Rucklidge, W. J., Samsikova, M., Zhang, C., Shelhamer, E., Lahera, E., Wiles, O., Ilyushchenko, S., Gorelick, N., Zhang, L. L., Alj, S., Schechter, E., Askay, S., Guinan, O., Moore, R., Boukouvalas, A., & Kohli, P.(2025). AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data. arXiv preprint arXiv.2507.22291. https://arxiv.org/abs/2507.22291
• Houriez, L., Pilarski, S., Vahedi, B., Ahmadalipour, A., Scully, T. H., Aflitto, N., Andre, D., Jaffe, C., Wedner, M., Mazzola, R., Jeffery, J., Messinger, B., McGinley-Smith, S., & Russell, S. (2025, August). Scalable geospatial data generation using AlphaEarth foundations model (arXiv:2508.11739). arXiv. https://arxiv.org/abs/2508.11739

​

References (Unsupervised Classification)

• Sherrie Wang, George Azzari, David B. Lobell, Crop type mapping without field-level labels: Random forest transfer and unsupervised clustering techniques, Remote Sensing of Environment, Volume 222, 2019, Pages 303-317, ISSN 0034-4257, https://doi.org/10.1016/j.rse.2018.12.026
• Cordeiro, M. C. R.; Martinez, J.-M.; Peña-Luque, S. Automatic Water Detection from Multidimensional Hierarchical Clustering for Sentinel-2 Images and a Comparison with Level 2A Processors.Remote Sensing of Environment 2021, 253, 112209. https://doi.org/10.1016/j.rse.2020.112209.

​

​