Earth Engine + TensorFlow

Chris Brown, Software Engineer

cfb@google.com

​

https://goo.gl/Xtr9rk

​

Who am I?

• Software Engineer on Google Earth Engine
• Satellites + Machine Learning
• Penn State ‘13
• Villanova ‘15

What to expect

1. Broad overview of machine learning in the context of remote sensing.
2. Hands-on exercise applying ML to classify regions of the earth’s surface.
3. More advanced use cases.
4. Whats next?

Machine Learning (specifically supervised learning)

An algorithm design technique that supplants the “how” with examples of the “what.”

ALL THE ALGORITHMS

• Logistic Regression
• Bayesian Classification
• Random Forest
• K-NN
• Like 400 more…

Onion???

200x400 pixels = 80000D Input Space!

ARTIFICIAL NEURAL NETS

Neural Nets

• When built up of more than ~2 layers, referred to as a “Deep Neural Net” (or DNN) and training these is sometimes referred to as “Deep Learning”
• A network of activation functions very apt at transforming complex input manifolds.
• We call values at the input “Features,” and values at the output “Labels”.
• Incredible expressive power, and the ML algorithm around which TensorFlow is build.
• I highly recommend https://developers.google.com/machine-learning/crash-course/ to learn more and build intuition.

"Deep learning is a particular kind of machine learning that achieves great power and flexibility by representing the world as a nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones."

Ian Goodfellow and Yoshua Bengio and Aaron Courville, "Deep Learning"

Machine Learning in Remote Sensing

Machine Learning in Remote Sensing

• Predicting climate
• Classifying land-cover
• Classifying land-use
• Removing clouds from images
• Predicting crop yield
• Use your imagination!

ML Input Features for Remote Sensing

• Per-pixel spectra.
• Time-series.
• Statistics
• Any combination of the above!

About TensorFlow

TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.

Useful docs

https://www.tensorflow.org/get_started/

​

https://github.com/tensorflow/tensorflow

​

https://cloud.google.com/ml-engine/docs/tensorflow/getting-started-training-prediction

Train/Test data (Export.table)

.TFRecord

Image data (Export.Image)

.TFRecord

.train()

.predict()

Predictions

.TFRecord

upload

1

2

3

4

5

https://colab.research.google.com/notebooks/welcome.ipynb

's

Search

1. Get these slides
2. Make a copy of this notebook.
1. (File > Save a copy in Drive)

Let’s get to work!

Spatial Contexts

Spatial Context

• Our exercise explored classification of land-cover. This can be thought of as detecting material: “This surface is impervious, this surface is vegetation, etc…”
• When we want to detect land-use, we need to know the material, and how that material is arranged in space.
• At higher resolutions, detecting objects is nigh-impossible without spatial context

ROAD

ROAD???

Convolutional Neural Nets

• Vanilla CNN (whole image labeling)
• R-CNN (bounding box detection)
• FCNN (semantic segmentation), a label for every pixel in the image

CNN Intuition

Convolution

Fully Convolutional Neural Nets

Detecting Objects

• Detect cars in 15cm aerial imagery
• Use RGB as our input channels (bands)
• Get the location of each car in the image.

Car Detector

Generate Training Data

Annotate examples

Export training data

Train Model

Export Inference Data

Predict

Extract Features

Case Study

Settlement Detection

• Where do humans live?
• Important for detecting:
• Flood/disease risk.
• Carbon footprint
• Population
• Habitat impact
• Etc..

​

Settlement Detection

• Problem concerns land-use, so spatial context is relevant.
• We’ll opt to use a FCNN like architecture for our modeling.
• We have a dataset of ~84k settlement/non-settlement point samples collected in partnership with the German Aerospace Center.

Blindspot

• XCeption-like U-Net.
• Lightweight, designed for low-resolution imagery.
• Able to learn complex spatial relationships with only point-samples.
• Really an ensemble of Blindspots for coping with noisiness of input data.
• Trained using CloudML Engine.

Demo

Where do I go from here?

• Want to learn more about machine learning? https://developers.google.com/machine-learning/crash-course/
• Need Earth Engine help? https://developers.google.com/earth-engine/help

​

Thank You!