1 of 46

​

​

Computer Vision meets Remote Sensing

Sayantan Das | Research Intern at ISRO

ucalyptus.github.io

2 of 46

Too many memes coming

3 of 46

Agenda

  • A brief intro to Satellite Imagery and Remote Sensing
  • Landsat multispectral imagery
  • Sentinel Microwave Data Acquisition / SAR Images are the future !
  • Enter GEE , the devmanus
  • LandCover using TF and GEE and GCS
  • Despeckling of SAR images (Non ML versus ML)
  • Geo Apps and Deployment Tools
  • Future of GeoAI Deployment Pipeline
  • Praising TFRecords!
  • Model Deployment ( Thanks Sayak Da )

​

​

4 of 46

Introduction to Satellite Imagery

Satellite imagery or spaceborne imagery are images of Earth or other planets taken by Imaging satellites operated by government and businesses around the world.

​

​

​

​

​

ucalyptus.github.io

5 of 46

How are Satellite Images Captured

6 of 46

Different Satellite Imaging Products

​

​

​

​

​

​

​

Landsat 8 data product Landsat 7 data product

​

​

​

7 of 46

SENTINEL 2

SENTINEL 1

8 of 46

9 of 46

10 of 46

11 of 46

12 of 46

Landcover Classification

https://github.com/ucalyptus/EarthEngine-Deep-Learning

​

13 of 46

14 of 46

15 of 46

Despeckling of SAR / Microwave Images

16 of 46

Way Forward with Webapps and Deployment Tools

17 of 46

Forest Loss Earth Engine App

18 of 46

Streamlit - Fastest way to build custom ML tools

https://github.com/streamlit/streamlit

19 of 46

Microsoft SandDance

https://github.com/microsoft/SandDance

20 of 46

Typical Geo AI Pipeline

21 of 46

Benefits of TFRecord and TFRecordDataset (TF2.0)

  • Serialized Data becomes easy to read efficiently .
  • Data stored in a set of files (100-200 MB each) to ease reading linearly.
  • Uses Protobuf for serialization of structured data.
  • tf.Example Documentation

​

22 of 46

Me praising Google Earth Engine because it’s free despite the image resolution

23 of 46

More details about this project

​

​

ucalyptus/EarthEngine-Deep-Learning

24 of 46

For all things 2.x

​

​

github.com/sayakpaul/TF-2.0-Hacks

25 of 46

Model deployment

​

​

Courtesy: Sayak Paul

26 of 46

Model deployment

​

  • Serializing the final model in SavedModel format
  • Uploading our model artifacts to Google Cloud Storage

​

​

​

sayak.dev

27 of 46

Model deployment

​

  • Serializing the final model in SavedModel format
  • Uploading our model artifacts to GCS
  • Kickstart an AI Platform model submission job via the AI Platform GUI
    • Go to Models and click on NEW MODEL

sayak.dev

28 of 46

Model deployment

​

  • Kickstart an AI Platform model submission job via the AI Platform GUI
    • Go to Models and click on NEW MODEL
    • Create the model

sayak.dev

29 of 46

Model deployment

​

  • Kickstart an AI Platform model submission job via the AI Platform GUI
    • Go to Models and click on NEW MODEL
    • Create the model
    • Create a version of the model

sayak.dev

30 of 46

Model deployment

​

  • Kickstart an AI Platform model submission job via the AI Platform GUI
    • Go to Models and click on NEW MODEL
    • Create the model
    • Create a version of the model
    • Specify the runtime and upload the artifacts

sayak.dev

31 of 46

Model deployment

​

  • Kickstart an AI Platform model submission job via the AI Platform GUI
    • Go to Models and click on NEW MODEL
    • Create the model
    • Create a version of the model
    • Specify the runtime and upload the artifacts

And voila!

**In my case (LandCover) my artifacts include the mixer.json file

sayak.dev

32 of 46

https://aihub.cloud.google.com/

33 of 46

References

​

  1. Machine Learning BootCamp Launchpad by Sayak Paul
  2. code.earthengine.google.com/
  3. Protobuf Python Basics

​

​

34 of 46

Thank You

​

sayantandas30011998@gmail.com

www.ucalyptus.github.io

​

Slides : http://bit.ly/sessionzero-geo

35 of 46

EXTRAS

36 of 46

Cloud Masking with Quality Assessment Band

37 of 46

38 of 46

39 of 46

40 of 46

41 of 46

Nitti gritties about TFRecord from LandCover Classification Example

Demystifying the transformation of data from TFRecord / into TFRecord format and beyond:

  • Read a file in Cloud Storage into a tf.data.Dataset
  • For parsing the exported TFRecord files, we map featurenames into float32 tf.io.FixLenFeature objects. It helps Tensorflow read data in TFRecord files into tensor.Specifically, all numeric data exported from Earth Engine is exported as float32
  • Parse the TFRecord. Read a serialized example into the structure defined by featuresDict.

A tuple of the predictors dictionary and the label, cast to an `int32`

​

42 of 46

43 of 46

TFRecord business during Inference phase

​

Now it's time to classify the image that was exported from Earth Engine.

​

If the exported image is large, it will be split into multiple TFRecord files in its destination folder.

​

There will also be a JSON sidecar file called "the mixer" that describes the format and georeferencing of the image.

​

Here we will find the image files and the mixer file, getting some info out of the mixer that will be useful during model inference.

44 of 46

  1. GCS provides us the ImageFileList
  2. We create a tf.data.Dataset from JSON mixer file and ImageFileList called ImageDataset
  3. We use model.predict on ImageDataset
  4. Write our predictions using TFRecordWriter

​

Next Steps

45 of 46

46 of 46