Determining the Number of Clusters

• Course: Databases and Data Mining
• Cluster Validation Techniques
• Instructor: Jamolbek Mattiev

Learning Objectives

• • Understand importance of K selection
• • Learn Elbow and Silhouette methods
• • Compare validation techniques
• • Apply methods in WEKA

Why Choosing K Matters

• • Too few clusters → underfitting
• • Too many clusters → overfitting
• • Affects interpretability
• • Impacts decision-making

Cluster Validation Categories

• • Internal validation
• • External validation
• • Relative validation

Internal Validation Metrics

• • WCSS (compactness)
• • Silhouette score
• • Davies–Bouldin index

Elbow Method

• • Compute WCSS for different K
• • Plot K vs WCSS
• • Look for elbow point
• • Choose K at inflection

WCSS Formula

• WCSS = Σ ||x - μ||²
• Measures within-cluster variance
• Lower is better

Limitations of Elbow

• • Elbow not always clear
• • Subjective interpretation
• • Depends on dataset structure

Silhouette Method

• • Measures cluster separation
• • Range: -1 to 1
• • Higher value indicates better clustering

Silhouette Interpretation

• • Close to 1 → good clustering
• • Around 0 → overlapping clusters
• • Negative → incorrect assignment

Davies–Bouldin Index

• • Ratio of intra/inter-cluster distance
• • Lower value → better clustering

Gap Statistic

• • Compares clustering with random reference
• • Choose K with maximum gap

Practical Steps in WEKA

• • Run clustering for K=2 to 10
• • Record WCSS
• • Compare cluster sizes
• • Use visualization tool

Experimental Workflow

• 1. Select clustering algorithm
• 2. Test multiple K values
• 3. Record evaluation metrics
• 4. Select optimal K

Comparing Methods

• Elbow → simple
• Silhouette → informative
• DB Index → separation-based
• Use multiple metrics together

Common Mistakes

• • Choosing arbitrary K
• • Ignoring data scaling
• • Overinterpreting weak clusters

Real-World Considerations

• • Domain knowledge matters
• • Business interpretability
• • Computational cost

Advanced Considerations

• • Stability analysis
• • Resampling validation
• • Consensus clustering

Discussion Questions

• • Can methods suggest different K?
• • Which method is most reliable?
• • How to validate results?

Summary

• • K selection is critical
• • Use multiple validation approaches
• • Combine metrics and domain insight
• • Always validate clustering results

Elbow Method Visualization

Silhouette Score Visualization