Deep Learning Adventures TensorFlow In Practice - Presentation 5
A quick overview of Courseraโs Tensorflow in Practice specialization course
Robert Kraig, David Patton, George Zoto
In the beginning...
Get to know our community
Hello, my name is ___ and I am a new/existing/regular member of this meetup.
I enjoy ___ and I am interested in learning more about ___
Training set ๐
Hello, my name is George and I am a regular member of this meetup. I enjoy applying deep learning to solve interesting problems and I am interested in learning more about time series and forecasting.
Not a typical Meetupโฆ Get ready for a fun game on 7/3 ๐๐
Attribution to Coursera and deeplearning.ai
Chapter 1 - TensorFlow in Practice Specialization
Chapter 1 - TensorFlow in Practice Specialization
Setup
Colaboratory is a free Jupyter notebook environment that requires no setup and runs entirely in the cloud. You can write and execute code, save and share your analyses, and access powerful computing resources, all for free from your browser.
Course 4: Sequences, Time Series and Prediction
Week 1: Sequences and Prediction
Week 2: Deep Neural Networks for Time Series
Week 3: Recurrent Neural Networks for Time Series
Week 4: Real-world time series data
Sequences and Prediction
Time series: An ordered sequence of values that are usually equally spaced over time e.g. every day in the weather forecast
Single value at each time step: univariate time series
Multiple values at each time step: multivariate time series
Sequences and Prediction
Source: https://www.coursera.org/learn/tensorflow-sequences-time-series-and-prediction
https://www.britannica.com/science/longitude
Movement of a body can also be plotted as a series of univariates or as a combined multivariate
Latitude
Longitude
Coordinates
Sequences and Prediction
Sequences and Prediction
Anomaly detection
Patterns / time series analysis
Imputation
Forecasting
Imputation
Sequences and Prediction
Anomaly detection
Sequences and Prediction
Anomaly detection
Stationary Time Series / Process: Its unconditional joint probability distribution does not change when shifted in time. Consequently, parameters such as mean and variance also do not change over time.
Sequences and Prediction
Anomaly detection
Sequences and Prediction
Anomaly detection
Sequences and Prediction
Anomaly detection
Sequences and Prediction
Anomaly detection
Sequences and Prediction
Anomaly detection
Sequences and Prediction
Anomaly detection
Course 4: Sequences, Time Series and Prediction
Week 1: Sequences and Prediction
Week 2: Deep Neural Networks for Time Series
Week 3: Recurrent Neural Networks for Time Series
Week 4: Real-world time series data
Deep Neural Networks for Time Series
Anomaly detection
Deep Neural Networks for Time Series
Deep Neural Networks for Time Series
Deep Neural Networks for Time Series
Deep Neural Networks for Time Series
Deep Neural Networks for Time Series
Deep Neural Networks for Time Series
Deep Neural Networks for Time Series
Lawrenceโs LR optimization?
For consistent results:
import random
seed = 51
tf.random.set_seed(seed)
random.seed = seed
Set all runs to epochs=500
Restart runtime and run all each test.
Lawrenceโs LR optimization results (seed=51)
mae 4.500836
mae 4.8579106
mae 5.3804765
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
mae 4.490887
Lawrenceโs LR optimization?
For consistent results:
import random
seed = 52
tf.random.set_seed(seed)
random.seed = seed
Set all runs to epochs=500
Restart runtime and run all each test.
Lawrenceโs LR optimization results (seed=52)
mae 4.577156
mae 4.5869045
mae 4.632053
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
mae 4.528657
Letโs continue our Time Series adventure ๐
Course 4: Sequences, Time Series and Prediction
Week 1: Sequences and Prediction
Week 2: Deep Neural Networks for Time Series
Week 3: Recurrent Neural Networks for Time Series
Week 4: Real-world time series data
Recurrent Neural Networks for Time Series
Sources: https://www.coursera.org/learn/tensorflow-sequences-time-series-and-prediction
http://karpathy.github.io/2015/05/21/rnn-effectiveness
https://iust-deep-learning.github.io/972/static_files/assignments/assignment_05_preview.html
return_sequences=True
Shape = [batch_size, #time_steps, #dims]
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Source: https://www.coursera.org/learn/tensorflow-sequences-time-series-and-prediction
https://en.wikipedia.org/wiki/Huber_loss
Loss Function: Huber
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
LR: 5e-5
Recurrent Neural Networks for Time Series
Readjust LR: 3e-6
Recurrent Neural Networks for Time Series
LR: 3e-6
LR: 5e-5
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Recurrent Neural Networks for Time Series
Readjust LR: 3e-6
tanh
linear
ReLU
Leaky ReLU
Validation MAE = 5.67
Validation MAE = 5.44
Validation MAE = 5.71
Validation MAE = 5.41
Recurrent Neural Networks for Time Series
Readjust LR: 3e-6
tanh
linear
ReLU
Leaky ReLU
Letโs continue our Time Series adventure ๐
Course 4: Sequences, Time Series and Prediction
Week 1: Sequences and Prediction
Week 2: Deep Neural Networks for Time Series
Week 3: Recurrent Neural Networks for Time Series
Week 4: Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Real-world time series data
Course 4: Sequences, Time Series and Prediction
Week 1: Sequences and Prediction
Week 2: Deep Neural Networks for Time Series
Week 3: Recurrent Neural Networks for Time Series
Week 4: Real-world time series data
Letโs continue our Time Series adventure ๐
Check out these resources
Check out these resources
Check out these AI for Healthcare resources
Check out this certification and books
Letโs continue our Time Series adventure ๐
Time for a fun game ๐๐
Practice here or use Flashcards: https://quizizz.com/join/quiz/5eefd13d3db4b8001bb6d72b/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336
Time for a fun game ๐๐
Practice here or use Flashcards: https://quizizz.com/join/quiz/5eefd13d3db4b8001bb6d72b/start?from=soloLinkShare&referrer=5d921444d0fa99001a135336
Questions
Discussion