1 of 23

�

​

​

​

​

Data Mining_Anoop Chaturvedi

1

Swayam Prabha

Course Title

Multivariate Data Mining- Methods and Applications

Lecture 16

Sparse PCA and Nonlinear Dimensionality Reduction

By

Anoop Chaturvedi

Department of Statistics, University of Allahabad

Prayagraj (India)

​

Slides can be downloaded from https://sites.google.com/view/anoopchaturvedi/swayam-prabha

2 of 23

  •  

Data_Mining_Anoop Chaturvedi

2

3 of 23

  •  

Data_Mining_Anoop Chaturvedi

3

4 of 23

  •  

Data_Mining_Anoop Chaturvedi

4

5 of 23

  •  

Data_Mining_Anoop Chaturvedi

5

6 of 23

SPCA ⇒

  • Attempts to find sparse loadings, i.e., a weight vector with only a few ’active’ values.
  • Principal components are formed as a linear combination of only a few of the original variables.
  • Avoids overfitting in a high-dimensional data setting.

Data_Mining_Anoop Chaturvedi

6

7 of 23

  •  

Data_Mining_Anoop Chaturvedi

7

8 of 23

�

Data_Mining_Anoop Chaturvedi

8

9 of 23

Data_Mining_Anoop Chaturvedi

9

10 of 23

  •  

Data_Mining_Anoop Chaturvedi

10

11 of 23

  •  

Data_Mining_Anoop Chaturvedi

11

12 of 23

  •  

Data_Mining_Anoop Chaturvedi

12

13 of 23

Nonlinear Dimensionality Reduction:

Linear PCA ⇒ Applied to discover low dimensional linear structures hidden in data.

What to do when data is concentrated on a lower dimensional non-linear manifold?

Applying linear PCA to non-linear datasets might result in non-optimal dimensionality reduction.

Bioconductor ⇒ A free, open source, and open development software project for the analysis of genomic data using R.

​

Data Mining_Anoop Chaturvedi

13

14 of 23

​

​

​

Data Mining_Anoop Chaturvedi

14

3-D plot of RNA-Sequencing gene expression count data

(available in Bioconductor)

15 of 23

  •  

Data Mining_Anoop Chaturvedi

15

16 of 23

  •  

Data Mining_Anoop Chaturvedi

16

17 of 23

Kernel Methods

A set of techniques used in machine learning to address classification, regression, and other prediction issues.

  • Kernel function: Measures the similarity of two data points to one another in a high-dimensional feature space.
  • Kernel method converts the input data into a high-dimensional feature space, which makes it simpler to distinguish between classes or generate predictions.
  • Examples of kernel methods ⇒ Kernel PCA, kernel density estimation, kernel support vector machine, etc.

Data Mining_Anoop Chaturvedi

17

18 of 23

  •  

Data_Mining_Anoop Chaturvedi

18

19 of 23

  •  

Data_Mining_Anoop Chaturvedi

19

20 of 23

  •  

Data_Mining_Anoop Chaturvedi

20

21 of 23

  •  

Data_Mining_Anoop Chaturvedi

21

22 of 23

  •  

Data_Mining_Anoop Chaturvedi

22

23 of 23

  •  

Data_Mining_Anoop Chaturvedi

23