Deep Learning for Remote Sensing: Challenges and Opportunities

Anthony Ortiz

October 23th, 2018

1

What is Remote Sensing?

Science of obtaining information about an object, area or phenomenon through an analysis of data acquired without direct contact with the area, object or phenomenon under investigation.

2

Who here has ever done Remote Sensing?

3

Things get a little more complicated

when the scale is increased

4

Things get a little more complicated

when the scale is increased

5

Components Remote Sensing System

6

Principles Behind Remote Sensing (RS)

• Electromagnetic energy reaching the earth’s surface is reflected, transmitted, or absorbed.

​

​

7

Principles Behind Remote Sensing (RS)

• Specific targets have an individual and characteristic manner of interacting with incident radiation that is described by the spectral response of the target.
• Electromagnetic Radiation make characteristic patterns as they travel through space. Each wave has a certain shape and length. The distance between peaks is called wavelength.

​

8

Platforms and Sensors

Platforms are used to house the sensors which obtain the data.

Platforms can be:

• Ground-based
• Airborne, ex. Drones, Aircraft
• Space borne, ex. Satellite

Types of sensor:

• Active, ex. Optical satellite sensors
• Passive, ex. LiDAR, RADAR

9

Resolutions

• Spatial: Smallest unit-area measured
• Spectral: Wavelength bands
• Temporal: Revisit time
• Radiometric: Energy

10

Classification Based on Spectral Resolution

Panchromatic: A single band image generally displayed as shades of gray.

Multispectral: A multispectral image consists of a few image layers, each layer represents an image acquired at a particular wavelength band.

Hyperspectral: A hyperspectral image consists of about a hundred or more contiguous spectral bands.

11

Examples of Remote Sensing Applications

12

Climate Change

Earth System Science is interdisciplinary scientific endeavor, treating Earth as a complex but integral system.

13

Satellites & Sensors Example

NASA’s Terra & Aqua satellite:

• Terra: launched on 18th Dec 1999.
• Aqua: launched on 4th May 2002.

Both have MODIS sensor: Moderate Resolution Imaging Spectroradiometer

• 1-2 days to complete earth view
• 36 spectral bands

Resolution: 1000 - 250 meters

14

Remote Sensing of Arctic Sea Ice

15

Pollution Monitoring

16

Deforestation

17

Deforestation

18

Situational Awareness

19

Flooding

Wildfire

Louisiana, August 2016. NOAA

Digitalglobe

Ecology and Conservation

20

UN Global Goals for Sustainable Development

21

UN Global Goals for Sustainable Development

22

Opportunities

23

Deep Learning for RS Catalyst and Opportunities

​

• Sensing Revolution and Space 2.0
• Relaxation on Sensing Constraints
• The Amount of Data Being Collected is Accelerating Rapidly
• High Spatial Resolution Satellite Imagery
• High Temporal Resolution

24

Challenges and Open Research Questions

25

Challenges

• Intra class variation

​

26

NWPU-RESISC45

Challenges

• Object size variability

​

​

27

Challenges

• Generalization

​

​

28

Scale

Challenges

• Generalization

​

​

29

Scale

Challenges

• Generalization

​

​

30

Seasonality

Challenges

• Generalization
• High dimensionality issues

​

31

Other Challenges

Limited labeled training data.

Many RS application depend on very complicated highly nonlinear physical/atmospheric models.

What architectural extensions will DL systems require in order to tackle complicated RS problems?

Multi-Sensor and multi-temporal processing

How can DL in RS successfully utilize transfer learning?

32

Success Test Cases where RS Community is Already Applying Deep Learning

33

Land Cover Mapping and Conservation

​

​

​

​

​

​

​

AI Land Cover, MSFT AI

34

Combining satellite imagery and machine learning to predict poverty (Jean et al., Stanford)

35

Combining Remote Sensing Data and Machine Learning to Predict Crop Yield

36

UN Global Goals for Sustainable Development

37

Thank you

38