1 of 82

Amazing GANs

By iRonhead

2 of 82

3 of 82

Create an Account on Kaggle

4 of 82

Join the Game

https://goo.gl/bfBC0b

or

https://kaggle.com/join/kkboxdatagame1706

5 of 82

Create Your Team

6 of 82

Find KKStream�on Facebook Now!

7 of 82

Routine

8 of 82

imironhead

9 of 82

Because

Everybody is making GANs

  • 2014/06, ml on Coursera
  • CS231n
  • Andrew: read papers, replicate results
  • Yann LeCun's cake
  • Cool

10 of 82

Prerequisite

I have only 35 minutes

  • A little bit of Python, of course
  • A little bit of deep learning
  • A little bit of Tensorflow
  • A lot of coffee

11 of 82

Neural Networks

12 of 82

Vector

<22, 2, 8, 0>

13 of 82

Matrix

Wx + b

14 of 82

Activation, ReLU

<0, 1, 2, -1, -2>

<0, 1, 2, 0, 0>

15 of 82

Feedforward

W11, b11

W21, b21

16 of 82

Backpropagation

W12, b12

W22, b22

W11 => W12

b11 => b12

W21 => W22

b21 => b22

17 of 82

Stochastic Gradient Descent

W

18 of 82

Learning Rate

W

learning rate

19 of 82

Trained Neural Network

W11, b11

W21, b21

G(

) =>

20 of 82

Convolutional Neural Networks

21 of 82

Convolution

1

2

3

4

2

1

2

3

3

2

1

2

4

3

2

1

0

1

0

1

2

1

0

1

0

12

12

10

input

kernel / filter

output

10

22 of 82

Sobel Filter

*

=

1

0

-1

2

0

-2

1

0

-1

23 of 82

Gaussian Filter

*

=

24 of 82

Convolution with Multiple Kernels

input

kernel / filter�can be trained

output

25 of 82

ReLU

26 of 82

Deeper

27 of 82

Flatten

28 of 82

Trained Convolutional Neural Network

G( ) =>

29 of 82

Generative Adversarial Networks

30 of 82

The Universe of Vision

31 of 82

Collect Images

32 of 82

Generator Neural Network

?

33 of 82

Generator Neural Network

?

34 of 82

Discriminator Neural Network

  • All generated images should be fake.
  • All collected images should be real.

35 of 82

Train the NN

Train Discriminator

Train Generator

Train Discriminator

Train Generator

36 of 82

Deep Convolutional

Generative Adversarial Networks

37 of 82

38 of 82

Generator

100 z

128

32

32

256

16

16

512

8

8

1024

4

4

3

64

64

G(z)

39 of 82

Discriminator

1

3

64

64

128

32

32

256

16

16

512

8

8

1024

4

4

40 of 82

LSUN Bedroom

  1. Large-Scale Scene Understanding Challenge
  2. 10 categories
  3. w x 256 / 256 x h
  4. 3,033,042 bedroom images

41 of 82

Results

  • 130,000 steps, 2.74 episodes
  • Learning rate 0.00002
  • About to collapse

42 of 82

Energy Based�Generative Adversarial Networks

43 of 82

44 of 82

Generator for MNIST

128 z

256

64

32

16

32

32

16

16

8

8

4

4

G(z)

1

32

32

45 of 82

Auto-Encoder

16

16

2

8

8

4

8

8

4

16

16

2

1

32

32

x

1

32

32

Dec(Enc(x))

4

4

8

Enc(x)

bottleneck

46 of 82

Discriminator, an Auto-Encoder

16

16

2

8

8

4

8

8

4

16

16

2

1

32

32

x

1

32

32

Dec(Enc(x))

4

4

8

Enc(x)

bottleneck

47 of 82

Margin - Weak Generator

48 of 82

Margin - Strong Generator

49 of 82

Pulling Away

16

16

2

8

8

4

8

8

4

16

16

2

1

32

32

1

32

32

4

4

8

Enc(x)

50 of 82

MNIST

  • Yann LeCun, Corinna Cortes, Christopher J.C. Burges
  • Modified NIST
  • 28 x 28 grayscale images
  • Training Set: 60,000 examples
  • Test Set: 10,000 examples

51 of 82

Results

  • Margin = 8.0
  • 159,000 steps

52 of 82

Boundary Equilibrium

Generative Adversarial Networks

53 of 82

54 of 82

Generator, a Decoder

55 of 82

Discriminator, an Auto-Encoder

56 of 82

Boundary Equilibrium

57 of 82

CelebA

  • Large-scale CelebFaces Attributes
  • 10,177 identities
  • 202,599 face images
  • 5 landmark locations
  • 40 binary attributes

58 of 82

(´Ａ｀。)

59 of 82

Random Generated Faces

60 of 82

Generated Linear Interpolated Faces

61 of 82

Generated Bilinear interpolated Faces

62 of 82

Experiment on GANs

63 of 82

NAN

64 of 82

Logarithm Epsilon

65 of 82

Vanished Gradients

66 of 82

Where are the gradients?

67 of 82

Sigmoid

1

3

64

64

256

16

16

1024

4

4

68 of 82

Collapsed

69 of 82

Oops!

This is PyCon.�I have to Python.

70 of 82

Graph can be Partially Restored

71 of 82

Use tf.image

  • Use ore if you want to be my guinea pig.

72 of 82

Image Summaries

  • tf.summary.image need a 4D tensor [batch_size, h, w, c]

73 of 82

Reshape

1

2

3

4

batch

1

2

3

4

reshape

1

2

3

4

split

1

2

3

4

concat

uint8

74 of 82

Use tf.gfile

  • Compatible with Google Cloud Machine Learning Engine and Cloud Storage.

75 of 82

Advices

Make your GAN life easier!

  • Get a GPU
  • Try MNIST first

76 of 82

Why GANs

77 of 82

Ok!

GAN is Cool!

Then?

78 of 82

Gaze Estimation

79 of 82

Challenges

$$

  • ML needs lots of data
  • Labeling is expensive
  • Challenging even for humans

80 of 82

Make Real Labeled Eyes

81 of 82

SimGAN

82 of 82

Thank You