1 of 25

Autoencoder 

2 of 25

Unsupervised Learning

  • Definition
    • Unsupervised learning refers to most attempts to extract information from a distribution that do not require human labor to annotate example
    • Main task is to find the ‘best’ representation of the data

​

  • Dimension Reduction
    • Attempt to compress as much information as possible in a smaller representation
    • Preserve as much information as possible while obeying some constraint aimed at keeping the representation simpler
    • This modeling consists of finding “meaningful degrees of freedom” that describe the signal, and are of lesser dimension.

2

3 of 25

Autoencoders

  •  

3

4 of 25

Autoencoder

  • Dimension reduction
  • Recover the input data

​

​

4

5 of 25

Autoencoder

  • Dimension reduction
  • Recover the input data
    • Learns an encoding of the inputs so as to recover the original input from the encodings as well as possible

​

​

5

6 of 25

Autoencoder

  •  

6

Source: Dr. Francois Fleuret at EPEL

7 of 25

Key Features of Autoencoders

  • Dimensionality Reduction
  • Feature Extraction
  • Preventing Overfitting
  • Noise Reduction (Denoising Autoencoders)
  • Efficient Encoding for Data Compression

7

8 of 25

Autoencoder with MNIST

8

9 of 25

Autoencoder with TensorFlow

  • MNIST example
  • Use only (1, 5, 6) digits to visualize in 2D

​

9

10 of 25

Test or Evaluation

  • It is remarkable that a latent space with only two neurons can capture sufficient information to reconstruct an MNIST image, originally represented in a 784-dimensional space.

10

11 of 25

Distribution in Latent Space

  • Make a projection of 784-dim image onto 2-dim latent space

​

​

​

​

​

​

​

​

  • Clustering or classification can be performed in the 2D latent space, rather than in the original 784-dimensional input space
    • Dim. reduction is performed before applying further machine learning algorithms or analyses
  • Note:
    • The latent space may change if the model is re-trained.
    • Each latent variable (or dimension) does not have an explicit physical meaning.

​

11

12 of 25

Autoencoder as Generative Model

12

13 of 25

Generative Capabilities

  •  

13

Source: Dr. Francois Fleuret at EPEL

14 of 25

MNIST Example

14

15 of 25

Generative Models

  •  

15

16 of 25

Latent Representation

  •  

16

Source: Dr. Francois Fleuret at EPEL

17 of 25

Latent Representation

  •  

17

Source: Dr. Francois Fleuret at EPEL

18 of 25

Walk in the Latent Space

18

19 of 25

Vector Arithmetic in Latent Space

19

20 of 25

Autoencoder for Pendulum Dynamics

  •  Dynamics of a simple pendulum
    • 500 images

20

 

128

128

21 of 25

Reconstruction and Latent Representations

  •  

21

22 of 25

Walk in the Latent Space

22

23 of 25

Revisit the Problem: Flow Around a Circular Cylinder

  • We previously studied fluid flow past a circular cylinder at low Reynolds number in the context of dimension reduction
  • Now that we've introduced the autoencoder to dimension reduction

​

23

24 of 25

Autoencoder

  • Autoencoder Models with 2 Latent Dimensions

24

25 of 25

Autoencoder

  • Autoencoder Models with 2 Latent Dimensions
    • Reconstruction and
    • Periodic pattern in latent space

25