�

​

​

​

​

Data Mining_Anoop Chaturvedi

1

Swayam Prabha

Course Title

Multivariate Data Mining- Methods and Applications

Lecture 25

Convolutional Neural Networks

By

Anoop Chaturvedi

Department of Statistics, University of Allahabad

Prayagraj (India)

​

Slides can be downloaded from https://sites.google.com/view/anoopchaturvedi/swayam-prabha

Convolutional Neural Networks (CNN)

• Multilayer perceptron (MLP) ⇒ Not invariant to transformations such as rotation, scaling, mirror rotation, translation.
• CNN are designed to be invariant to such transformations.
• Also known as Shift Invariant or Space Invariant Artificial Neural Networks (SIANN).
• Inspired by the organization of the animal visual cortex.
• In images there is strong correlation between nearby pixels.

​

Data Mining_Anoop Chaturvedi

2

​

​

​

​

• Handwritten digits ⇒ Not perfect and can be made with a variety of flavors
• Digit recognition ⇒ Nearness property applies even after the transformations.
• Pixels correlated before the transformation are also correlated after the transformation.

​

Data Mining_Anoop Chaturvedi

3

Data Mining_Anoop Chaturvedi

4

Rotation

Mirroring

Original

Translation

Scaling

MLP is trained for learning a map from input to output, where input is from a fixed location.

It is not feasible to train the MLP for all possible transformations and combinations.

Thus, MLP may not be able to identify the digits or images in the presence of these transformations.

CNN

• Exploit information from small patches, such as borders, colour patches, basic shapes etc.
• Utilizes these ideas through weight sharing, subsampling, receptive fields.
• It is a feed-forward neural network that learns feature engineering by itself via filters or kernel optimization
• Image is classified into one of the classes based on the identity of its main object, e.g., dog, airplane, bird, or different letters in handwriting recognition.

​

​

​

​

​

Data Mining_Anoop Chaturvedi

5

• Incorporate pooling layers, such as max pooling or average pooling. Pooling helps make the learned features more robust to small variations in input and reduces the computational burden.

Significance of Convolution Layers

• Exploit the spatial structure of the input data. Each neuron is connected only to a local region of the input volume, which captures local patterns such as edges, textures, or colors.

​

Data Mining_Anoop Chaturvedi

6

• Same set of weights (called the kernel or filter) is shared across all spatial positions, which significantly reduces the number of parameters in the network, making it computationally efficient and reducing the risk of overfitting.
• Weights sharing across the input space makes the Convolutional layers robust to translations. and capable of detecting features irrespective of their location.
• The lower layers detect basic features like edges and textures, while higher layers learn more abstract features such as object parts or entire objects.

​

Data Mining_Anoop Chaturvedi

7

• Each neuron is connected to only a subset of neurons in the previous layer. This sparsity helps in reducing the computational cost of the network and allows to scale efficiently large input volumes, such as high-resolution images or videos.

Convolutional layers enable CNNs to learn

1. Hierarchical representations of input data
2. Achieve translation invariance, and
3. Efficiently extract features relevant to the task at hand.

Data Mining_Anoop Chaturvedi

8

Data Mining_Anoop Chaturvedi

9

Data Mining_Anoop Chaturvedi

10

Data Mining_Anoop Chaturvedi

11

Data Mining_Anoop Chaturvedi

12

Data Mining_Anoop Chaturvedi

13

Data Mining_Anoop Chaturvedi

14

Data Mining_Anoop Chaturvedi

15

Data Mining_Anoop Chaturvedi

16

Data Mining_Anoop Chaturvedi

17

Data Mining_Anoop Chaturvedi

18

Data Mining_Anoop Chaturvedi

19

Data Mining_Anoop Chaturvedi

20

Data Mining_Anoop Chaturvedi

21

Data Mining_Anoop Chaturvedi

22

Data Mining_Anoop Chaturvedi

23

Data Mining_Anoop Chaturvedi

24