1 of 71

What is Google Earth Engine?

https://goo.gl/pR5tkF

Nick Clinton, nclinton@google.com �Earth Engine Developer Relations

2 of 71

Developer Advocates - Earth Engine

Tyler Erickson, PhD

Nick Clinton, PhD�

3 of 71

Background

4 of 71

5 of 71

Goodchild et al. (2012):

“The supply of geographic information from satellite-based and ground-based sensors has expanded rapidly, encouraging belief in a new, fourth, or “big data,” paradigm of science that emphasizes international collaboration, data-intensive analysis, huge computing resources, and high-end visualization.”

6 of 71

Deriving Information from Earth Observation Data

7 of 71

Source: NASA

8 of 71

Google Mission Statement

"To organize the world's information and make it universally accessible and useful."

https://www.google.com/about/company/

9 of 71

-Jim Gray (1944-2007)

“Often it turns out to be more efficient to move the questions than to move the data.”

10 of 71

11 of 71

12 of 71

Before Earth Engine

13 of 71

14 of 71

15 of 71

16 of 71

17 of 71

https://earthengine.google.com/timelapse/

18 of 71

19 of 71

Data Catalog

20 of 71

> 200 public datasets

MODIS

Daily, NBAR, LST, ...

Terrain

SRTM, GTOPO, NED, ...

Atmospheric

NOAA NCEP, OMI, ...

Land Cover

GlobCover, NLCD, ...

The Earth Engine Public Data Catalog

> 4000 new images every day

> 5 million images

> 5 petabytes of data

Landsat and Sentinel

Raw, TOA, SR, ...

... and many more, updating daily!

Landsat data. Whatever USGS has, Earth Engine has. Landsats 1-8. Raw, TOA reflectance, surface reflectance. Sentinels 1-2. TOA reflectance. SAR data from Sentinel-1 processed to backscattering coefficients. Most MODIS terrestrial composites. Global DEMS at a variety of resolutions. Multiple land cover datasets. Atmosphere and climate data. And growing daily. There is a one or two day latency between scene acquisition and when it is ingested to the catalog.

To discover things, search the catalog from the Code Editor (details coming).

You can request data on the externally facing dataset request list, where every Earth Engine user can vote for datasets that will be useful for them.

You can upload your own PRIVATE data and nobody (not even Google) can see them. But we hope you share the data and make them public ;)

21 of 71

BIG data: 40+ years of Landsat imagery

22 of 71

Daily MODIS composites

23 of 71

Terrain

24 of 71

Atmosphere

25 of 71

Sentinel

Sentinel-1

26 of 71

API

27 of 71

Data Types and Geospatial Processing Functions

Image - band math, clip, convolution, neighborhood, selection ...
Image Collection - map, aggregate, filter, mosaic, sort ...
Feature - buffer, centroid, intersection, union, transform ...
Feature Collection - aggregate, filter, flatten, merge, sort …
Filter - by bounds, within distance, date, day-of-year, metadata ...
Reducer - mean, linearRegression, percentile, histogram ….
Join - simple, inner, outer, inverted ...
Kernel - square, circle, gaussian, sobel, kirsch …
Machine Learning - CART, random forests, bayes, SVM, kmeans, cobweb …
Projection - transform, translate, scale …

over 1000 data types and operators, and growing!

Image is the fundamental raster data structure in Earth Engine. Bands. Pixels. Metadata.

ImaeCollection is stack or time series of images.

Feature is the fundamental vector structure in Earth Engine. Geometry and metadata.

FeatureCollection is a collection of Features (surprise).

Filter is how to limit the scope of analysis temporally, spatially or by metadata.

Reducer is the way to aggregate data. Statistics such as min, max, mean, SD, variance, covariance, linear regression, etc.

Joins are the way to combine different data sets.

Kernel facilitates image processing operations, for example convolution.

Machine Learning algorithms support supervised and unsupervised classification.

Projection is the way to control scale and appearance of data outputs.

And growing in response to user requests.

28 of 71

What can you do with Earth Engine?

Get an Image

Pick your: Projection, resolution, bands, bounding-box, visualization

29 of 71

What can you do with Earth Engine?

Get an Image

Apply an algorithm to an image

Library functions or script your own.

30 of 71

What can you do with Earth Engine?

Get an Image

Apply an algorithm to an image

Filter a collection

Time, Space & Metadata search

31 of 71

What can you do with Earth Engine?

Get an Image

Apply an algorithm to an image

Filter a collection

Map an algorithm over a collection

N → N

32 of 71

What can you do with Earth Engine?

Get an Image

Apply an algorithm to an image

Filter a collection

Map an algorithm over a collection

Reduce a collection

N → 1 or N → M

33 of 71

What can you do with Earth Engine?

Get an Image

Apply an algorithm to an image

Filter a collection

Map an algorithm over a collection

Reduce a collection

Compute aggregate statistics

34 of 71

35 of 71

// Make a median composite from one year of Landsat 5.

// Load the image collection.

var collection = ee.ImageCollection('LANDSAT/LT5_L1T_TOA');

// Filter it down to 2001.

var filtered = collection.filterDate('2001', '2002');

// For each pixel, for each band, calculate the median and make an image� // of the result. The median tends to remove clouds, shadows, data gaps.

var medianImage = filtered.median();

// Add the image to a map.

Map.addLayer(medianImage, {bands:['B3', 'B2', 'B1'], min:0, max:0.25});

36 of 71

Earth Engine Demo

37 of 71

Geospatial�Datasets

Algorithmic�Primitives

add

focal_min

filter

reduce

join

distance

mosaic

convolve

Results

Storage and Compute

Requests

38 of 71

Publications

39 of 71

https://earthenginepartners.appspot.com/science-2013-global-forest

40 of 71

Photo: Dibyendu Ash, www.goingwild.com

"This study was made possible by free availability of satellite imagery, cloud computing services, and interactive web tools. We were able to analyze 14 years of high-resolution global forest loss data across 76 landscapes that span 13 countries."

41 of 71

https://code.earthengine.google.com/d418ce95c8cca2aff7e456a8ca9c9d3a

42 of 71

http://www.sciencedirect.com/science/article/pii/S0034425715001637

43 of 71

44 of 71

http://www.tandfonline.com/doi/abs/10.1080/01431161.2016.1145366

https://github.com/nasa/CrisisMappingToolkit

45 of 71

http://www.sciencedirect.com/science/article/pii/S003442571630044X

46 of 71

47 of 71

48 of 71

49 of 71

Applications

50 of 71

IaaS

Infrastructure

as a Service

PaaS

Platform

as a Service

SaaS

Software

as a Service

Cloud Computing Service Models

High level�of built-in

functionality

High level�of�customizability

51 of 71

52 of 71

http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002415

http://www.earthenv.org/cloud

53 of 71

http://eeflux-training.appspot.com/

54 of 71

http://clim-engine.appspot.com/

55 of 71

http://mol.org/

56 of 71

Application: http://aqua-monitor.appspot.com/

Code: https://github.com/gena/aqua-monitor

57 of 71

58 of 71

59 of 71

Surface Water Monitor Links

Application: http://surface-water.appspot.com/

Source: https://github.com/Servir-Mekong/SurfaceWaterTool

Paper: http://www.mdpi.com/2072-4292/8/5/386/htm

60 of 71

61 of 71

62 of 71

Ecological Dashboard Links

Application: http://ecodash-beta.appspot.com/

Source: https://github.com/Servir-Mekong/ecodash

63 of 71

Mobile to Cloud pipeline

https://opendatakit.org/

https://rs-collective.appspot.com

Fusion Table

https://goo.gl/BQNCvZ

Earth Engine

https://code.earthengine.google.com/

App Engine

ODK

Collect

http://gnomapix.appspot.com/

ODK Aggregate

This is a demonstration project to show how mobile devices can be connected to the cloud to aggregate responses, with Earth Engine using the data to make a crowd-sourced land cover map.

http://www.google.com/earth/outreach/tutorials/odk_collect.html

To collect data, first install ODK Collect on your phone. Open ODK Collect, then go to the menu button > General Settings > Configure platform settings > URL and enter https://rs-collective.appspot.com. From ODK Collect, Get Blank Form and select the IGBP form (I think that's the only one on there, but I may have created one matching the classification of the Gong et al. paper). Collect points by going INTO the center of homogeneous land cover polygons that are at least four 30 meter pixels large. The points will be uploaded to rs-collective.appspot.com for anyone to use.

IGBP Land cover classes:

Evergreen needleleaf forests: Lands dominated by needleleaf woody vegetation with a percent cover >60% and height exceeding 2 m. Almost all trees remain green all year. Canopy is never without green foliage.

Evergreen broadleaf forests: Lands dominated by broadleaf woody vegetation with a percent cover >60% and height exceeding 2 m. Almost all trees and shrubs remain green year round. Canopy is never without green foliage.

Deciduous needleleaf forests: Lands dominated by woody vegetation with a percent cover >60% and height exceeding 2 m. Consists of seasonal needleleaf tree communities with an annual cycle of leaf-on and leaf-off periods.

Deciduous broadleaf forests: Lands dominated by woody vegetation with a percent cover >60% and height exceeding 2 m. Consists of broadleaf tree communities with an annual cycle of leaf-on and leaf-off periods.

Mixed forests: Lands dominated by trees with a percent cover >60% and height exceeding 2 m. Consists of tree communities with interspersed mixtures or mosaics of the other four forest types. None of the forest types exceeds 60% of landscape.

Closed shrublands: Lands with woody vegetation less than 2 m tall and with shrub canopy cover >60%. The shrub foliage can be either evergreen or deciduous.

Open shrublands: Lands with woody vegetation less than 2 m tall and with shrub canopy cover between 10% and 60%. The shrub foliage can be either evergreen or deciduous.

Woody savannas: Lands with herbaceous and other understory systems, and with forest canopy cover between 30% and 60%. The forest cover height exceeds 2 m.

Savannas: Lands with herbaceous and other understory systems, and with forest canopy cover between 10% and 30%. The forest cover height exceeds 2 m.

Grasslands: Lands with herbaceous types of cover. Tree and shrub cover is less than 10%.

Permanent wetlands: Lands with a permanent mixture of water and herbaceous or woody vegetation. The vegetation can be present either in salt, brackish, or fresh water.

Croplands: Lands covered with temporary crops followed by harvest and a bare soil period (e.g., single and multiple cropping systems). Note that perennial woody crops will be classified as the appropriate forest or shrub land cover type.

Urban and built-up lands: Land covered by buildings and other man-made structures.

Cropland/natural vegetation mosaics: Lands with a mosaic of croplands, forests, shrubland, and grasslands in which no one component comprises more than 60% of the landscape.

Snow and ice: Lands under snow/ice cover throughout the year.

Barren: Lands with exposed soil, sand, rocks, or snow and never have more than 10% vegetated cover during any time of the year.

Water bodies: Oceans, seas, lakes, reservoirs, and rivers. Can be either fresh or salt– water bodies.

64 of 71

In preparation...

65 of 71

Training Data

66 of 71

Surface water occurrence - Pekel et al., JRC

南昌

67 of 71

River morphology

Bryk et al. UC Berkeley

68 of 71

Malaria risk mapping

Sturrock et al. UCSF

69 of 71

Who uses Earth Engine?

70 of 71

Come to a workshop at AGU, December 12-13, 2016!

https://events.withgoogle.com/google-agu-2016/earth-engine-workshops/

71 of 71

earthengine.google.com/signup