1 of 27

�

​

​

​

​

Data Mining_Anoop Chaturvedi

1

Swayam Prabha

Course Title

Multivariate Data Mining- Methods and Applications

Lecture 32

Clustering based upon Mixture Models

By

Anoop Chaturvedi

Department of Statistics, University of Allahabad

Prayagraj (India)

​

Slides can be downloaded from https://sites.google.com/view/anoopchaturvedi/swayam-prabha

2 of 27

Data Mining_Anoop Chaturvedi

2

 

3 of 27

Data Mining_Anoop Chaturvedi

3

Histogram of a random sample of size 10000 from mixture of normal distributions

4 of 27

Clustering based upon Mixture Models

  • Creating clustering methods based upon a statistical model with stochastic elements
  • Traditional statistical inference framework could be applied.
  • Expectation Maximization (EM) algorithm for missing data is used for making inferences. �

Data Mining_Anoop Chaturvedi

4

5 of 27

 

Data Mining_Anoop Chaturvedi

5

6 of 27

  •  

Data Mining_Anoop Chaturvedi

6

7 of 27

  •  

Data Mining_Anoop Chaturvedi

7

8 of 27

  •  

Data Mining_Anoop Chaturvedi

8

9 of 27

 

Data Mining_Anoop Chaturvedi

9

10 of 27

  •  

Data Mining_Anoop Chaturvedi

10

11 of 27

  •  

Data Mining_Anoop Chaturvedi

11

12 of 27

  •  

Data Mining_Anoop Chaturvedi

12

13 of 27

  •  

Data Mining_Anoop Chaturvedi

13

14 of 27

  •  

Data Mining_Anoop Chaturvedi

14

15 of 27

  •  

Data Mining_Anoop Chaturvedi

15

16 of 27

  •  

Data Mining_Anoop Chaturvedi

16

17 of 27

  •  

Data Mining_Anoop Chaturvedi

17

18 of 27

  •  

Data Mining_Anoop Chaturvedi

18

19 of 27

  •  

Data Mining_Anoop Chaturvedi

19

20 of 27

  •  

Data Mining_Anoop Chaturvedi

20

21 of 27

Possible model selection procedures for selecting number of clusters and/or cluster models

1. Log-likelihood ratio test statistics

2. Information theoretic approaches (AIC, BIC)

3. Bayes factors

4. Markov chain Monte Carlo methods using reversible jump MCMC or birth and death process methodology.

5. Nonparametric bootstrap assessment of the number of modes in the data using a kernel density estimator

Data Mining_Anoop Chaturvedi

21

22 of 27

  •  

Data Mining_Anoop Chaturvedi

22

23 of 27

Example: Ruspini data set, consisting of 75 points in four groups that is popular for illustrating clustering techniques. It is a very simple data set with well separated clusters.

​

Data Mining_Anoop Chaturvedi

23

24 of 27

Gaussian mixture model is used to identify the clusters.

BIC and ICL (Integrated Completed Likelihood) criterion are used to find the number of clusters.

R-packages used ⇒ dbscan, mclust,

​

​

​

​

​

​

​

​

​

Data Mining_Anoop Chaturvedi

24

25 of 27

​

​

​

​

​

​

​

​

​

Data Mining_Anoop Chaturvedi

25

26 of 27

We rerun the program with four clusters and obtain

​

​

​

​

​

Data Mining_Anoop Chaturvedi

26

27 of 27

Data Mining_Anoop Chaturvedi

27