Basic Introduction to �Deep Learning

Dr. Patrick McClure

National Institute of Mental Health

Part 1: Linear Models

Key Components of Machine Learning

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​

Data

Regression

​

Classification

Model

Regression

Linear Regression Model

​

Classification

Logistic Regression Model

​

Objective Function

Regression

Linear Regression Model

​

Classification

Logistic Regression Model

​

Learning Algorithm

Gradient Descent

Learning Algorithm

Gradient Descent

Learning Algorithm

Gradient Descent

Chain Rule for Linear Regression

Learning Algorithm

Gradient Descent

Chain Rule for Logistic Regression

Learning Algorithm

Gradient Descent

Gradient Descent with Momentum

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​

Basic Introduction to �Deep Learning

Dr. Patrick McClure

National Institute of Mental Health

Part 2: Neural Networks

Data: Non-linear

Regression

​

Classification

Model: Non-linear

Regression

​

Classification

Objective Function

Regression

​

Classification

Model: Neural Network

Linear Regression

Neural Network Regression

Weights

…

…

Non-Linearity

Model: Neural Network (Non-linearities)

Rectified Linear Unit (ReLU)

Sigmoid

Tanh

Model: Neural Network

Linear Regression

Neural Network Regression

Weights

…

…

Non-Linearity

Model: Neural Network

Linear Regression

Neural Network Regression

Weights

…

…

Non-Linearity

Model: Neural Network

Linear Regression

Neural Network Regression

Weights

…

…

Non-Linearity

Model: Neural Network

Linear Regression

Neural Network Regression

Weights

…

…

Non-Linearity

Objective Function: Neural Network

Neural Network Regression

Regression Loss

Non-Linearity

Model: Neural Network

Logistic Regression

Neural Network Binary Classification

Weights

…

…

Non-Linearity

Sigmoid

Sigmoid

Objective Function: Neural Network

Binary Classification Loss

Neural Network Binary Classification

Non-Linearity

Sigmoid

Learning Algorithm: Backpropagation

Gradient Descent with Backpropagation

Model: Deep Neural Network

Deep Neural Network Regression

…

Model: Deep Neural Network

Deep Neural Network Binary Classification

…

Learning Algorithm: Backpropagation

Gradient Descent with Backpropagation

Learning Algorithm: Backpropagation

Gradient Descent with Backpropagation

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​

Basic Introduction to �Deep Learning

Part 3: Convolutional Neural Networks

Dr. Patrick McClure

National Institute of Mental Health

Data: Time Series

EEG Amplitude

Time (seconds)

…

Model: 1-D Convolutional Neural Network

EEG Amplitude

Time (seconds)

…

Model: 1-D Convolutional Neural Network

EEG Amplitude

Time (seconds)

…

Model: 1-D Convolutional Neural Network

EEG Amplitude

Time (seconds)

…

Model: 1-D Convolutional Neural Network

EEG Amplitude

Time (seconds)

…

…

Model: 1-D Convolutional Neural Network

…

…

…

…

Regression

Classification

Model: 1-D Convolutional Neural Network

…

…

…

…

…

Model: 1-D Convolutional Neural Network

42

…

…

…

…

…

Model: 1-D Convolutional Neural Network

…

…

…

…

…

Model: 1-D Convolutional Neural Network

Increasing Receptive Field Sizes

Input

Convolutional

Layer 1

Convolutional

Layer 2

…

Data: Images and Volumes

Images (2-D)

Volumes (3-D)

Model: n-D Convolutional Neural Networks

2-D Convolution

3-D Convolution

Key Components of Machine Learning

1. Data

What are the inputs and the true outputs?

​

• Model

How are we transforming the inputs into outputs?

​

• Objective Function

How do we determine if the outputs of the model are good?

​

• Learning Algorithm

How do we find model parameters that lead to good model outputs?

​