Training Neural Nets: a Hacker’s Perspective
Agenda
The motivation
The motivation
“Deep learning neural networks have become easy to define and fit, but are still hard to configure.”
Training a neural network
Training a neural network
Wrong shuffling!
Training a neural network
Training a neural network
Feature interpretation
Training a neural network
Training a neural network
Training a neural network
No zero outs!
Training a neural network
Training a neural network
Training a neural network
Training a neural network
Training a neural network
Training a neural network
Training a neural network
Maintaining a healthy prototyping process
Maintaining a healthy prototyping process
Maintaining a healthy prototyping process
Maintaining a healthy prototyping process
Overfitting a single batch of data
Overfitting a single batch of data
Overfitting a single batch of data
Overfitting a single batch of data
Model complexity as a f(order)
Deciding on a model architecture:
Model complexity as a f(order)
Deciding on a model architecture:
Model complexity as a f(order)
Some points to remember here:
Model complexity as a f(order)
Visualize performance
Visualize performance
Deep learning is a shout in the void and the oblivion is inevitable!
Model complexity as a f(order)
Some points to remember here:
Model complexity as a f(order)
Some points to remember here:
Model complexity as a f(order)
Chasing the hyperparameters
Chasing the hyperparameters
Chasing the hyperparameters
Going beyond ...
Going beyond ...
Going beyond ...
Going beyond ...
Summary
References
References
Slides are available here: http://bit.ly/DFGoa19