1 of 37

TensorFlow in Context

​

Jiqiong QIU

​

​

1

2 of 37

About me

​

2

  • Master of engineer degree in Bioinformatic and modelization at INSA de Lyon�
  • PhD in Color Formulation by Statistical Learning at UTC and BASF Coatings

(EP 2887275A1: Method and system for determining a color formula)�

  • Data scientist at Sfeir

3 of 37

TensorFlow in Context

​

  1. Deep Learning
  2. TensorFlow
  3. TensorFlow in Context

​

3

4 of 37

Deep Learning

4

1

5 of 37

1 Deep Learning

1.1 What is Deep Learning?

1.2 Difference between academic research and industry application�

​

​

​

​

​

5

6 of 37

1.1 What is deep learning

6

Artificial Intelligence

Machine�Learning

Logistic,�Regression, SVM,�Neural Network

Deep Learning

CNN, LSTM,�Neural Turing�Machines

7 of 37

1.1 What is deep learning

7

8 of 37

1.2 Difference between academic research and industry application

8

9 of 37

1.2 Difference between academic research and industry application

9

​

Academic Research

Industry Application

Key Point

Research

Application

Time Investment

​

Long term

Short term

Development Environment

Stand alone

IDE, Compilation tools, Teamwork etc

​

Goal

Interest/ publication

​

Problem solving

10 of 37

TensorFlow

10

2

11 of 37

2 TensorFlow

2.1 Key features

2.2 Comparison with others deep learning libraries

​

​

​

​

​

​

11

12 of 37

2.1 Key features

12

  • Open source by Google
  • Python API
  • Board
  • Android (SDK) Mobile application

https://github.com/tensorflow/tensorflow

13 of 37

2.1 Key features

However, TensorFlow is very slow...

13

https://github.com/soumith/convnet-benchmarks

https://www.reddit.com/r/MachineLearning/comments/48gfop/tensorflow_speed_questions/

14 of 37

2.2 Comparison with others deep learning libraries

14

http://deeplearning.net/software_links/

15 of 37

2.2 Comparison with others deep learning libraries

15

Name

Language

OS

GPU

Related Library

Theano

Python

Win, Lin, Mac

CUDA,Opencl

Lasagne, Keras

Torch

Lua, C

Lin, IOS, Android

CUDA

​

Caffe

C++, Python, Matlab

Lin, Win, Mac

CUDA, Opencl

​

TensorFlow

Python

Lin, Mac, Android

CUDA

Keras, Skflow

mxnet

Python, R, Julia

Lin, Windows, Mac

CUDA

​

https://github.com/zer0n/deepframeworks

16 of 37

2.2 Comparison with others deep learning libraries

​

Why TensorFlow?

16

17 of 37

TensorFlow in Context

17

3

18 of 37

3 TensroFlow in Context

​

3.1 What is unique about TensorFlow?

3.2 TensorFlow with Data Science Tools

3.3 TensorFlow for Big Data

​

18

19 of 37

3.1 What is unique about TensorFlow

That would be crazy if it weren't Google

​

19

20 of 37

3.1 What is unique about TensorFlow

The author list of TensorFlow:

  • Jeff Dean: father of MapReduce
  • Ian Goodfellow: main contributor of Theano/PyLearn2
  • Yangqing Jia: main contributor of Caffe
  • and other great Google researchers and engineers.

20

21 of 37

3.2 TensorFlow with Data Science Tools

Why we need deep learning in Industry application besides playing Go?

21

22 of 37

3.2 Tensorflow with Data Science Tools

Avoid hand-crafted features

22

23 of 37

3.2 Tensorflow with Data Science Tools

No free lunch:

Deep learning applications are generally applied to massive unstructured data.

23

MNIST 60k

ImageNet 50 million

Yelp Restaurant Photo Classification

230 k

24 of 37

3.2 Tensorflow with Data Science Tools

Most used data science languages:

​

​

​

​

​

TensorFlow has an API in Python

​

24

​

Python

R

Data Manipulation

Pandas

dplyr, data.table

Data Visualization

Matplotlib, Seaborn

​

ggplot2, ggvis

Machine Learning

scikit-learn

caret

25 of 37

3.2 Tensorflow with Data Science Tools

Deep Learning is hard:

25

26 of 37

3.2 Tensorflow with Data Science Tools

Deep learning library like keras, Skflow (based on TensorFlow) were developed with a focus on enabling fast experimentation.

26

27 of 37

3.3 Tensorflow for Big Data

​

No free lunch:

Deep learning applications are generally applied to massive unstructured data.

27

MNIST 60k

ImageNet 50 million

Yelp Restaurant Photo Classification

230 k

GPU makes the deep learning training possible

28 of 37

3.3 Tensorflow for Big Data

​

28

CPU vs GPU

29 of 37

3.3 Tensorflow for Big Data

​

Training on Multiple-GPU:

  • A single GTX 580 GPU has only 3GB of memory
  • GPU memory limits the maximum size of the networks that can be trained
  • Training examples may be too big to fit on on GPU

29

30 of 37

3.3 Tensorflow for Big Data

​

30

1 GPU vs multiple-GPU

31 of 37

3.3 Tensorflow for Big Data

​

In TensorFlow, the supported device types are CPU and GPU. They are represented as strings. For example:

  • "/cpu:0": The CPU of your machine.
  • "/gpu:0": The GPU of your machine, if you have one.
  • "/gpu:1": The second GPU of your machine, etc.

Much earier than others libraries

31

https://www.tensorflow.org/versions/r0.7/how_tos/using_gpu/index.html

32 of 37

3.3 Tensorflow for Big Data

​

32

GPU is not enough

33 of 37

3.3 Tensorflow for Big Data

​

33

Speed

Ease of use

Generality

Runs Everywhere

​

34 of 37

3.3 Tensorflow for Big Data

​

34

https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html

​

Distributed Tensor Flow on Spark is published on early 2016

​

35 of 37

Tensorflow in Context

35

Name

Language

OS

GPU

Related Library

Theano

Python

Win, Lin, Mac

CUDA,Opencl

Lasagne, Keras

Torch

Lua, C

Lin, IOS, Android

CUDA

​

Caffe

C++, Python, Matlab

Lin, Win, Mac

CUDA, Opencl

​

Tensorflow

Python

Lin, Mac, Android

CUDA

Keras, Skflow

mxnet

Python, R, Julia

Lin, Windows, Mac

CUDA

​

36 of 37

Thank you.

36

37 of 37

by Jiqiong QIU�SFEIR - Copyright ©2016

37