1 of 40

Going Deeper with GoogLeNet and CaffeJS

“Deep Learning in the Browser”

Christoph Körner

1

Slides available on bit.ly/2cwhdyG

2 of 40

About me

2

3 of 40

Agenda

  • Deep Learning Refresher
  • GoogLeNet
  • CaffeJS - Deep Learning in the browser
  • Whats next

3

4 of 40

Deep Learning Refresher

4

5 of 40

Ingredients for Deep Classif.

  • Convolutions + Nonlinearities
  • Pooling
  • Fully Connected layers
  • Softmax
  • Many layers and parameters

5

6 of 40

Convolutions

6

7 of 40

Pooling

7

8 of 40

Parameters in Dense Layers

8

x*y*depth

neurons

9 of 40

Parameters in Convolutions

9

depth

filters

sx

sy

10 of 40

Deep Learning Architectures

10

11 of 40

Top-1 accuracy

11

12 of 40

Top-1 accuracy density

12

13 of 40

AlexNet: 62,378,344 params (250MB)

13

14 of 40

VGG (~400MB) and ResNet (~230MB)

14

15 of 40

GoogLeNet: 6,998,552 params (28MB)

15

16 of 40

GoogLeNet

“moving from fully connected

to sparsely connected architectures”

16

17 of 40

GoogLeNet

  • ~10 times less parameter than AlexNet
  • ~15% higher classification accuracy
  • ~30MB weights in memory
  • ~1.5 billion MAC operations (FP)

17

18 of 40

Inception Module (Network in a Network)

18

19 of 40

Ingredients for Inception Module

  • Hebbian Principle
  • Bottleneck Convolutions

19

20 of 40

Ingredient 1: Hebbian Principle

“Cells that fire together, wire together”

20

Source: Siegrid Löwel about Hebb’s Principle

21 of 40

Idea of Local Neighborhood

21

22 of 40

Idea of Local Neighborhood

22

23 of 40

Idea of Local Neighborhood

23

24 of 40

Idea of Local Neighborhood

24

25 of 40

A Naive Approach (does not scale)

25

26 of 40

Ingredient 2: Bottleneck Convolutions

  • Input features are correlated
  • Remove redundancy by Aggregation
  • Compute Convolutions
  • Expand features afterwards (if needed)

26

27 of 40

Inception Module

27

28 of 40

Extensions to v1

  • Batch Norm 2015
  • Inception v3 2105

28

29 of 40

CaffeJS - DL in the Browser

29

Source: CaffeJS

30 of 40

CaffeJS

  • Started as Proof of Concept
  • Run trained DL models in the browser�→ especially GoogLeNet

  • Based on ConvNetJS by Andrej Karpathy

30

31 of 40

Why CaffeJS

  • Client-side model evaluation
    • Fraud detection
    • Image upscaling
    • Sentiment detection
    • Gender + age detection (webcam)
    • Image filters

31

32 of 40

Why CaffeJS

  • Teaching & Learning
    • No requirements (but a browser)
    • Understand & analyze Deep Nets
    • Debugging of Deep Nets (FF and BP)
    • Visualize the filters, layers, activations, etc.
    • Feed webcam stream into Deep Nets

32

33 of 40

Using pre-trained Models

  • Offline training (many GPUs, long time)
  • Export weights
  • Perform only forward pass (no BP needed)

33

34 of 40

CaffeJS - Caffe Models in the Browser

  • Work in progress…
  • Prases *.prototxt models
  • Transforms *.caffemodel weights into bin files (one file per layer) - Thanks to #1669 in Keras
  • Updates weights in ConvNetJS models
  • Forward and backward pass

34

35 of 40

CaffeJS vs. ConvNetJS

  • Graph structure for layers + layer iterator
  • Gradient correction for wide models
  • Fixed Local Response Normalization
  • Abstractions for visualizations
  • New Layers (Concat, AVG Pool, etc.)

35

36 of 40

Demos with CaffeJS

36

37 of 40

Problems of CaffeJS

  • Forward pass still slow
    • 6s for GoogLeNet
    • Most time spend in early convolutions
  • Too much overhead for weights (fc6 and fc7)
  • Memory issues (above 200MB)

37

38 of 40

Whats next

38

39 of 40

Some more useful resources

39

40 of 40

Thank you.

40