ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
222EIA001: Deep Learning
2
3
Lecture No.:Topic to be Covered
4
5
1Introduction - What is Deep Learning? – Machine Learning Vs. Deep Learning, representation Learning
6
2Width Vs. Depth of Neural Networks
7
3Activation Functions: RELU, LRELU, ERELU],
8
4Boltzmann Machines, autoencoders
9
5Optimization Techniques, Gradient Descent, Batch Optimization
10
6Optimization Techniques, Gradient Descent, Batch Optimization
11
7Back Propagation - Calculus of Back Propagation,
12
8Back Propagation - Calculus of Back Propagation,
13
9Bayesian Learning, Decision Surfaces Linear
Classifiers, Machines with Hinge Loss
14
10Bayesian Learning, Decision Surfaces Linear
Classifiers, Machines with Hinge Loss
15
11Unsupervised Training of Neural Networks, Restricted Boltzmann Machines, Auto Encoders
16
12Unsupervised Training of Neural Networks, Restricted Boltzmann Machines, Auto Encoders
17
13Perceptron and Multi-layer Perceptron – Hebbian
Learning - Neural net as an Approximator
18
14Perceptron and Multi-layer Perceptron – Hebbian
Learning - Neural net as an Approximator
19
15Training a neural network - Perceptron learning rule -
Empirical Risk Minimization - Optimization by gradient descent
20
16Training a neural network - Perceptron learning rule -
Empirical Risk Minimization - Optimization by gradient descent
21
17Convergence in Neural networks - Rates of Convergence –
Loss Surfaces – Learning rate and Data normalization
22
18Convergence in Neural networks - Rates of Convergence –
Loss Surfaces – Learning rate and Data normalization
23
19RMSProp, Adagrad and Momentum , Stochatic
Gradient Descent
24
20RMSProp, Adagrad and Momentum , Stochatic
Gradient Descent
25
21Acceleration – Overfitting and Regularization
26
22Acceleration – Overfitting and Regularization
27
23Choosing a Divergence Loss Function – Dropout –
Batch Normalization
28
24Convolutional Neural Networks (CNN) - Weights as Templates – Translation Invariance
29
25Training with shared parameters – Arriving at the
convolutional model
30
26Mathematical details of CNN
31
27Alexnet – Inception – VGG - Transfer Learning
32
28Alexnet – Inception – VGG - Transfer Learning
33
29Recurrent Neural Networks (RNNs)
34
30Recurrent Neural Networks (RNNs)
35
31Modeling sequences - Back propagation through time
- Bidirectional RNNs
36
32Modeling sequences - Back propagation through time
- Bidirectional RNNs
37
33Modeling sequences - Back propagation through time
- Bidirectional RNNs
38
34Exploding/vanishing gradients - Long Short-Term
Memory Units (LSTMs)
39
35Exploding/vanishing gradients - Long Short-Term
Memory Units (LSTMs)
40
36Exploding/vanishing gradients - Long Short-Term
Memory Units (LSTMs)
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100