Tensorflow Intro
B03901048 EE3 Ching-Lun Tai
Outline
1) Building neural networks 2) Visualization
- About Tensorflow - Result visualization
- Session - Tensorboard
- Variables & Scope 3) Appendix
- Placeholder - Activation functions & Optimizers
- Define add_layer - Reference
PART I
Building NN
Find toolkit? Tensorflow!
Q: What is Tensorflow?
A: A python-based toolkit for neural networks developed by Google.
Q: Advantages?
A: Open source, visualizable, and many metaframeworks for use!
Processing structure
-data-flow graph based
(method of visualization will be discussed later)
-node: mathematical operation
-line: data between nodes, represented by tensors.
0-dim: scalars
1-dim: vectors
2↑-dim: matrices
General Principle
To build a neural network with Tensorflow, you can follow the steps below:
1) Import the modules you need.
2) Define an add_layer function to construct layers.
3) Define the variables and initialize them.
4) Create a session to perform the operation.
5) Build the NN, and send data with placeholders and feed_dict.
6) Train and test the NN, and observe the results with Tensorboard.
Importing module
-Basic
-result visualization (will be discussed later)
Session (1)
To control the operation of our program, we have to create a session.
There are two ways to create a session.
Ex: A simple matrix multiplication
Session (2)
-method 1: create a variable for a session
-method 2: use with to create a session
→Remember to capitalize the S!
→Remember to close the session if using this method!
→If you choose to use with, you don’t have to close the session explicitly.
Variables & Scope (1)
There are two kinds of scopes and two methods of creating variables.
I. tf.name_scope()
Note: Tensorflow adopts the data type float32 only! (neglect few exceptions)
Remember to define the data type!
Variables & Scope (2)
Result of I.
tf.get_variable→no effect!!
tf.Variable→names will change!!
Variables & Scope (3)
II. tf.variable_scope()→to reuse the variables
If you want to reuse some variables, remember to add the following line:
scope.reuse_variables()
Variables & Scope (4)
Result of II.
tf.get_variable→can be reused!!
tf.Variable→names will change!!
Variables & Scope (5)
1) If you define any variable, remember to initialize with the following lines:
init = tf.global_variables_initializer()
init = tf.global_variables_initializer()
init = tf.global_variables_initializer()
So important that we repeat 3 times !!
2) Activate the initialization with session using the following line:
sess.run(init)
Placeholder (1)
If we want to send the “outside” data into our neural networks, we have to use placeholder as a container.
1) Define the data type of the placeholder
Note: Tensorflow adopts the data type float32 only! (neglect few exceptions)
Placeholder (2)
2) Use session to perform the “sending”, and we employ feed_dict={} to designate the variables that will be sent in.
Note: It method 2 of creating a session is used, we don’t need to close it explicitly!
Define add_layer
We can define a function add_layer for further additions of layers in our neural networks.
Note: The value of biases are suggested not to be zero and can be adjusted.
Example: building a simple neural network
After we introduce the basic use of Tensorflow, you can try building your own neural network!
If you are a newcomer to Tensorflow, you can run the following example found on the Internet:
https://github.com/MorvanZhou/tutorials/blob/master/tensorflowTUT/tensorflow11_build_network.py
It will definitely help you grasp the concept of building a neural network with Tensorflow!
PART II
Visualization
Result visualization (1)
For intuitive visulaization of our results, we can generate some plots.
1) Import the module
2) plt.figure()→add_subplot()→plt.show()
ax.scatter→scattering plot
ax.line→line
Result visualization (2)
3) For continuous display, add the following line:
plt.ion()
Ex:
Tensorboard (1)
Tensorflow features Tensorboard, which can help us analyze our neural networks.
Note: It is better to use Google Chrome to browse Tensorboard.
Tensorboard (2)
Steps:
1) Define the names (inputs, weights ,biases, loss, ...)
Ex: Inputs
Tensorboard (3)
2) Create a session and use tf.summary.FileWriter() to restore the graph
3) Go back to your terminal and enter the following command line:
tensorboard --logdir='logs/'
4) Copy the website address displayed on your terminal to the web browser and you can see the Tensorboard!
Tensorboard (4)
There are two kinds of summary diagrams.
I. tf.scalar_summary()→loss
II. tf.histogram_summary()→others(weights, biases, outputs, ...)
Note:
(1) These two kinds of summary diagrams must be used together, otherwise there may be some bugs!
(2) The diagrams will dynamically change when the program is running!
PART III
Appendix
Activation functions & Optimizers
I. Activation functions
https://www.tensorflow.org/versions/r0.10/api_docs/python/nn/activation_functions_
II. Optimizers
https://www.tensorflow.org/api_guides/python/train
Basic: GradientDescentOptimizer
Advanced: MomentumOptimizer & AdamOptimizer
Alphago (a.k.a. master): RMSPropOptimizer
Reference
I. Morvan Zhou’s youtube channel & website
https://www.youtube.com/playlist?list=PLXO45tsB95cKI5AIlf5TxxFPzb-0zeVZ8
https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/
II. Tensorflow
III. CS224d: Tensorflow Tutorial
https://cs224d.stanford.edu/lectures/CS224d-Lecture7.pdf
http://www.planwallpaper.com/static/images/thank-you-clothesline-752x483.jpg