1 of 19

Autoencoder 

2 of 19

Unsupervised Learning

  • Definition
    • Unsupervised learning refers to most attempts to extract information from a distribution that do not require human labor to annotate example
    • Main task is to find the ‘best’ representation of the data

​

  • Dimension Reduction
    • Attempt to compress as much information as possible in a smaller representation
    • Preserve as much information as possible while obeying some constraint aimed at keeping the representation simpler
    • This modeling consists of finding “meaningful degrees of freedom” that describe the signal, and are of lesser dimension.

2

3 of 19

Autoencoders

  •  

3

4 of 19

Autoencoder

  • Dimension reduction
  • Recover the input data

​

​

4

5 of 19

Autoencoder

  • Dimension reduction
  • Recover the input data
    • Learns an encoding of the inputs so as to recover the original input from the encodings as well as possible

​

​

5

Original space

Latent space

6 of 19

Autoencoder

  •  

6

Source: Dr. Francois Fleuret at EPEL

7 of 19

Autoencoder with MNIST

7

8 of 19

Autoencoder with TensorFlow

  • MNIST example
  • Use only (1, 5, 6) digits to visualize in 2-D

​

8

9 of 19

Test or Evaluation

9

10 of 19

Distribution in Latent Space

  • Make a projection of 784-dim image onto 2-dim latent space

​

10

11 of 19

Autoencoder as Generative Model

11

12 of 19

Generative Capabilities

  •  

12

Source: Dr. Francois Fleuret at EPEL

13 of 19

MNIST Example

13

14 of 19

Latent Representation

  •  

14

Source: Dr. Francois Fleuret at EPEL

15 of 19

Latent Representation

  •  

15

Source: Dr. Francois Fleuret at EPEL

16 of 19

Interpolation in High Dimension

16

17 of 19

Interpolation in Manifold

17

18 of 19

MNIST Example: Walk in the Latent Space

18

19 of 19

Generative Models

  •  

19