First step Deep Learning�with Tensorflow

​

Jiqiong QIU

​

​

1

About me

​

2

• Master of engineer degree at INSA de Lyon�
• PhD in Color Formulation by Statistical Learning at UTC and BASF Coatings

(EP 2887275A1: Method and system for determining a color formula)�

• Data scientist at Sfeir

First step Deep Learning with Tensorflow

​

1. What is Deep Learning?
2. Why Tensorflow?
3. Digital Recognition using CNN
4. Resources

3

What is Deep Learning?

4

1

1 What is Deep Learning

1.1 From AI to Deep Learning

1.2 Neural Network�1.3 Deep Learning

​

​

​

​

​

5

1.1 From AI to Deep Learning

6

Artificial Intelligence

Machine�Learning

Logistic,�Regression, SVM,�Neural Network

Deep Learning

CNN, LSTM,�Neural Turing�Macines

1.2 Neural Network

7

Artificial Neuron

Neuron

1.2 Neural Network

8

MLP: Multilayer Perceptron

Hidden layer

Output layer

Input layer

Input #1

​

Input #3

​

Input #4

​

Input #5

​

Output

​

1.3 Deep Learning

9

• More layer
• Different Neuron types

Output layer

ALEXNET

Why Tensorflow?

10

2

2 Why Tensorflow?

11

• Open source by Google
• Python API
• Board
• Android (SDK) Mobile application

And more: http://deeplearning.net/software_links/

​

Digital Recognition�using CNN

12

3

3 Digital Recognition using CNN

3.1 Digital Recognition and Data Set

3.2 CNN

​

​

​

​

​

​

13

3.1 Digital Recognition and Data Set

14

• Images 28x28 pixels
• Training set: 60 000
• Test set: 10 000

MNIST database: Mixed National Institute�of Standards and Technology database

​

3.2 CNN : Convolutionnal Neural Network

​

15

3.2.1 Convolution layer

3.2.2 Pooling layer

3.2.3 MLP and more

3.2.4 Training and Results

3.2 Digital Recognition using CNN

16

Convolutionnal Neural Network

​

3.2.1 Convolution layer

​

17

Natural Images have the property of being “stationary”

​

3.2.1 Convolution layer

​

18

Kernel Matrix

3.2.1 Convolution layer

​

19

3D Neuron MATRIX

Input Depth = 3 (rgb)

Output Depth = 400

3.2.1 Convolution layer

​

20

3.2.1 Convolution layer: Tensorflow

​

​

21

3.2.2 Pooling layer

22

Pooling (or Subsampling)

​

​

• 28x28 pixels
• 400 features (depth) over 3x3 inputs
• 2 strides
• 13x13x400 = 169x400= 67600

3.2.2 Pooling layer: Tensorflow

​

​

23

3.2.3 MLP and More

​

24

MLP

Convolution + Pooling

3.2.3 MLP and More

​

25

MLP

Convolution + Pooling

Reshape (3D to 1D)

3.2.3 MLP and More

26

Dropout

Hidden layer

Output layer

Input layer

Input #1

​

Input #3

​

Input #4

​

Input #5

​

Output

​

3.2.3 MLP and More

27

• Multiple Classification:�[0,1,..,8,9]�
• Softmax:

3.2.3 MLP and More: Tensorflow

​

​

28

• Reshape��
• Dropout��
• Output of CNN

3.2.4 Training and Results

29

• Cost Function:��
• Cost Function is defined by:
• Input Values (images, noted as x)
• Weight matrix (w)

3.2.4 Training and Results

​

30

Training process

​

3.2.4 Training and Results: Tensorflow

​

​

​

31

• Cost Function��
• Training process

3.2.4 Training and Results

​

​

​

32

3.2.4 Training and Results

​

​

​

33

3.2.4 Training and Results

​

​

​

34

Resources

35

4

4 Resources

​

36

• For this tutorial:�https://github.com/Sfeir/demo-tenserflow

https://docs.google.com/presentation/d/1ch4YiKD83wERmmEFRvFIQ98Mtz65aGsLP3uWiTElY2I/edit?usp=sharing

• MOOC:�CS229: Machine Learning

UD730: Deep Learning - Taking machine learning to the next level

CS231n: Convolutional Neural Networks for Visual Recognition

CS224d:Deep Learning for Natural Language Processing

• Summer school:

IPAM 2012

Montreal 2015

• Website

http://deeplearning.net/ https://www.topcoder.com/ https://www.kaggle.com/

Thank you.

37

by Jiqiong QIU�SFEIR - Copyright ©2016

38