1 of 26

Danbury AI

2 of 26

IS Sponsored By:

https://bitly.com/AS-SW

3 of 26

Machine Learning With TensorFlow

TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google's Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well.

Presented By: Andrew & Mike

4 of 26

Overview

  • Introduction
  • Deep Learning Framework Space
  • TensorFlow Basics
  • Installing TensorFlow with Virtualenv.
  • Graph Visualization with TensorBoard
  • A Single Neuron
    • Interactive Sessions and Ipython
  • MNIST Walkthrough
  • TF Learn
  • TensorFlow GPU Processing
  • Sliding window and MINST walkthrough.

5 of 26

Deep Learning Framework Space

  • Caffe (2014)

(+) Great Model Zoo

  • TensorFlow ( 2015 )

(+) Good deployment infrastructure.

(+) Support for heterogeneous distributed systems

(+) Mobile Friendly

(+) Python and C++ Interfaces

​

  • Theano ( 2007 )

(+) Deep learning frameworks built on top ( Keras, Lasagne and Blocks )

(+) Long academic history

(+) Python + Numpy

​

  • Torch (2012)

(+) Abundance of pretrained models

(+/-) Lua

(+) Used by Facebook and Twitter

​

​

6 of 26

What is TensorFlow?

  • TensorFlow is an open source software library for numerical computation using data flow graphs.
  • Predecessor of Google’s Deepmind DistBelief.
  • Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them.
  • Lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code.
  • TensorBoard, a data visualization toolkit.

7 of 26

TensorFlow Features

  • Auto-Differentiation
    • You define the computational architecture of your predictive model, combine that with your objective function, and just add data -- TensorFlow handles computing the derivatives for you.
  • Research and Production Environment
    • At Google, research scientists experiment with new algorithms in TensorFlow, and product teams use TensorFlow to train and serve models live to real customers.
  • Portability
    • Runs on CPUs or GPUs, and on desktop, server, or mobile computing platforms.
  • Flexibility
    • TensorFlow isn't a rigid neural networks library. If you can express your computation as a data flow graph, you can use TensorFlow. Defining handy new compositions of operators is as easy as writing a Python function and costs you nothing in performance. And if you don't see the low-level data operator you need, write a bit of C++ to add a new one.
  • Language Options
    • Comes with an easy to use Python interface and a no-nonsense C++

interface to build and execute your computational graphs. SWIG extensions.

8 of 26

The Data Flow Graph

  • A TensorFlow computation is described by a directed graph, which is composed of a set of nodes.
    • Each node has zero or more inputs and zero or more outputs and represents the instantiation of an operation.
      • Some kinds of nodes maintain and update persistent state for branching and looping control. (i.e. variables)
    • An operation has a name and represents some abstract computation ( e.g. matrix multiply, add ).
    • A kernel is a particular implementation of an operation that can be run on a particular device ( e.g. CPU, GPU ).
    • A TensorFlow binary defines the sets of operations and kernels available via a registration mechanism and can be extended.
  • Tensors are arbitrary dimensionality arrays that flow along normal edges in the graph (from outputs to inputs). z
  • Control dependencies are special edges that indicate the source node for the control dependence must finish executing before the destination node.

9 of 26

Data Flow Graph Efficiency

  • To do efficient numerical computing in Python, we typically use libraries like NumPy that do expensive operations such as matrix multiplication outside Python, using highly efficient code implemented in another language. Unfortunately, there can still be a lot of overhead from switching back to Python every operation. This overhead is especially bad if you want to run computations on GPUs or in a distributed manner, where there can be a high cost to transferring data.

​

  • TensorFlow also does its heavy lifting outside Python, but it takes things a step further to avoid this overhead. Instead of running a single expensive operation independently from Python, TensorFlow lets us describe a graph of interacting operations that run entirely outside Python. This approach is similar to that used in Theano or Torch.

​

  • The role of the Python code is therefore to build this external computation graph, and to

dictate which parts of the computation graph should be run.

10 of 26

Graph Example

11 of 26

Installing TensorFlow ( Python 2 )

  • Install pip and Virtualenv.

​

# Ubuntu/Linux 64-bit�$ sudo apt-get install python-pip python-dev python-virtualenv��# Mac OS X�$ sudo easy_install pip�$ sudo pip install --upgrade virtualenv

​

  • Create a Virtualenv environment.

​

$ virtualenv --system-site-packages ~/tensorflow

​

12 of 26

Installing TensorFlow ( Python 2 )

​

  • Activate the Virtualenv environment and install TensorFlow in it.

​

$ source ~/tensorflow/bin/activate # If using bash�$ source ~/tensorflow/bin/activate.csh # If using csh�(tensorflow)$ # Your prompt should change��# Ubuntu/Linux 64-bit, CPU only:�(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl��# Ubuntu/Linux 64-bit, GPU enabled. Requires CUDA toolkit 7.5 and CuDNN v4. For�# other versions, see "Install from sources" below.�(tensorflow)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl��# Mac OS X, CPU only:�(tensorflow)$ pip install --upgrade

https://storage.googleapis.com/tensorflow/mac/tensorflow-0.8.0-py2-none-any.whl�

​

​

​

13 of 26

Installing TensorFlow ( Python 2 )

​

  • After the install you will activate the Virtualenv environment each time you want to use TensorFlow.

​

$ source ~/tensorflow/bin/activate # If using bash.�$ source ~/tensorflow/bin/activate.csh # If using csh.�(tensorflow)$ # Your prompt should change.�# Run Python programs that use TensorFlow.�...�# When you are done using TensorFlow, deactivate the environment.�(tensorflow)$ deactivate

​

14 of 26

TensorBoard

  • Interactive visualization tool baked into TensorFlow.
  • TensorBoard operates by reading TensorFlow events files, which contain summary data that you can generate when running TensorFlow.
  • Denote nodes with summary operations for output to event files.
  • Great example:

https://github.com/DanburyAI/DanburyAITensorFlowPresentationAug2016/blob/master/mnist_with_summaries.py

Live demo: https://www.tensorflow.org/tensorboard/index.html

15 of 26

A Single Neuron

DEMO

​

See “Beginner MINST.ipynb” on repo

​

https://github.com/DanburyAI/DanburyAITensorFlowPresentationAug2016/blob/master/Beginner%20MNIST.ipynb

16 of 26

MNIST Walkthrough

DEMO

​

See “Beginner MINST.ipynb” on repo

​

https://github.com/DanburyAI/DanburyAITensorFlowPresentationAug2016/blob/master/Beginner%20MNIST.ipynb

17 of 26

TF Learn

  • To smooth the transition from the Scikit Learn world of one-liner machine learning into the more open world of building different shapes of ML models. You can start by using fit/predict and slide into TensorFlow APIs as you are getting comfortable.
  • http://tflearn.org/installation/�

​

18 of 26

TensorFlow GPU Processing

GPU > CPU

  • Matrix multiplication, convolution, and large element-wise operations.
  • GPU RAM should be ‘saturated’ otherwise the overhead required to transfer data to the GPU will cancel out computation speedup

​

GPU = CPU

  • Indexing, dimension-shuffling and constant-time reshaping

​

GPU < CPU

  • Summation over rows/columns

19 of 26

Sliding Window MNIST

DEMO

​

​

20 of 26

Resources and Sources

21 of 26

Resources & Sources

​

22 of 26

Resources & Sources

23 of 26

QUESTIONS& COMMMENTS

​

Presented By: Andrew Ribeiro

And Michael Rogowski

FIND THIS PRESENTATION ONLINE:

​

https://github.com/DanburyAI/DanburyAITensorFlowPresentationAug2016

24 of 26

UNUSED SLIDES

25 of 26

TensorBoard: class tf.train.SummaryWriter

  • The SummaryWriter class provides a mechanism to create an event file in a given directory and add summaries and events to it. The class updates the file contents asynchronously. This allows a training program to call methods to add data to the file directly from the training loop, without slowing down training.
  • ​

26 of 26

Softmax Regression Model

  • Softmax regression (or multinomial logistic regression) is a generalization of logistic regression to the case where we want to handle multiple classes. In logistic regression we assumed that the labels were binary: y(i)∈{0,1}y(i)∈{0,1}. We used such a classifier to distinguish between two kinds of hand-written digits. Softmax regression allows us to handle y(i)∈{1,…,K}y(i)∈{1,…,K} where K is the number of classes.
  • When K = 2, Softmax == Logistic Regression.
  • ​