1 of 25

Customer Churn Prediction & Factors Identification Using Data Mining and Explainable AI

Group Members:

Md. Sadman Sakib (180021126)

Tourkir Rahman (180021114)

MD. Nazmus Sadat (180021121)

Kazi Raine Raihan (180021102)

Eid Ali Mohamed (170021157)

​

Supervisor:

Mr. Asif Newaz

Lecturer

Department of Electrical and Electronic Engineering

Islamic University of Technology

​

​

2 of 25

Motivation and background

Why do we need churn prediction?

  • Retaining customers is more cost-effective than acquiring new ones.
  • Customer loyalty fosters sustainable growth and brand reputation.
  • To maximize profit and stay competitive in the challenging telecom industry.
  • To enable targeted, personalized offers, improving customer satisfaction and retention.

3 of 25

Goals of the project

  • Detect customers at the highest risk of leaving.
  • Enhance business sustainability by improving understanding of churn factors.
  • Leverage Explainable AI to provide actionable insights for customer retention.
  • Utilize insights from customer clustering for targeted retention strategies.

4 of 25

Literature Review

5 of 25

Tools used

Programming language

  • Python

Libraries

  • numpy, pandas
  • mlxtend, imblearn
  • sklearn

​

  • Explainable AI:
    • SHAP

6 of 25

Dataset Description

Dataset 1

The dataset used for our experiment was obtained from IBM Watson Cognos Analytics. It has 7043 samples and 21 features.

  • Class imbalance present in churn (2.8:1)

To make it usable for model training, data was preprocessed.

​

7 of 25

Model Training, Feature Selection

& Performance Metrics

For model evaluation, repeated stratified cross validation was used (10 folds, 3 repeats).

Feature Selection

  • Correlation, Chi square
  • Sequential Feature Selection
  • Recursive Feature Elimination

Supervised Learning

  • Random Forest
  • XGBoost
  • Balanced Random Forest

​

​

Unsupervised Learning

  • KMeans, Gaussian Mixture Model

Performance metrics

  • Accuracy
  • Precision
  • Recall/Specificity
  • ROC-AUC
  • Geometric mean
  • MCC

8 of 25

Feature Selection (Dataset 1)

Fig: Feature selection SFS and RFECV.

9 of 25

Results (Churn Prediction-Dataset 1)

Balanced Random Forest, with features selected by Sequential feature selection method had the best overall results.

​

accuracy

precision

recall

roc_auc

gmean

specificity

mcc

time

all features

0.744028

0.512683

0.772069

0.834443

0.752588

0.733876

0.456367

7.154706

correlation

0.741753

0.509844

0.763858

0.828156

0.748417

0.733747

0.449273

3.621853

chi2

0.70914

0.470551

0.744782

0.793019

0.719911

0.696234

0.394823

3.287314

sfs

0.750048

0.52013

0.781697

0.832253

0.759609

0.738587

0.469477

3.658134

rfe

0.746209

0.515339

0.774562

0.833594

0.754798

0.735941

0.460638

3.884431

10 of 25

Demystifying AI: From Black Box to White Box

Why is Explainable AI Essential?

  • Traditional ML Models: Known as "Black Boxes" - they provide predictions without explanations.
  • The Need for Explainable AI (XAI): To understand the 'why' behind model predictions.
  • SHAP: One of the most popular XAI tools to provide transparency and model-agnostic explanations.

​

​

11 of 25

Explainable AI

SHAP (SHapley Additive exPlanations) is an approach based on game-theory that help to explain the output of any machine learning model.

​

  • Base Value: Average prediction over the training dataset.
  • SHAP Values: Quantify contribution of each feature to prediction.
  • Positive SHAP values increase prediction, negative values decrease it.
  • SHAP offers various visualization tools to help analyze individual predictions and contribution of the features.

​

12 of 25

SHAP - Force Plot (Dataset 1)

13 of 25

SHAP - Summary Plot (Dataset 1)

14 of 25

SHAP - Beeswarm Plot (Dataset 1)

15 of 25

Understanding Churn: From Analysis to Segmentation

​

​

Why analyze churned customers?

  • Analyze patterns among churned customers to anticipate potential churn scenarios.
  • Leverage unsupervised learning to discover hidden patterns in unlabeled data.
  • Segment churned customers into similar groups to better understand distinctive behaviors.
  • Use correlation analysis to identify key features separating these groups.
  • Optimize and target retention strategies based on these insights.

16 of 25

Analyzing Churned Customers (Dataset 1)

​

17 of 25

Dataset Description

Dataset 2

The dataset used for our experiment was obtained from bigml. It has 3333 samples and 20 features.

  • Class imbalance present in churn (6:1)

To make it usable for model training, data was preprocessed.

18 of 25

Feature Selection (Dataset 2)

Fig: Feature selection SFS and RFECV.

19 of 25

Results (Churn Prediction-Dataset 2)

XGBoost, with features selected by Sequential feature selection method had the best overall results.

​

accuracy

precision

recall

roc_auc

gmean

specificity

mcc

time

all features

0.958401

0.928206

0.774518

0.915903

0.874706

0.989591

0.824483

2.190179

correlation

0.935096

0.846156

0.674972

0.890019

0.811426

0.979181

0.719433

1.411915

chi2

0.953199

0.904889

0.759864

0.910206

0.864936

0.985965

0.802688

1.411269

sfs

0.961001

0.93955

0.783433

0.918413

0.880531

0.991111

0.836079

1.145718

rfe

0.9593

0.934208

0.775184

0.916085

0.875684

0.990526

0.828331

1.25076

20 of 25

SHAP - Force Plot (Dataset 2)

21 of 25

SHAP - Summary Plot (Dataset 2)

22 of 25

SHAP - Beeswarm Plot (Dataset 2)

23 of 25

Analyzing Churned Customers (Dataset 2)

​

24 of 25

Conclusion

  • Utilized SHAP for Explainable AI, shedding light on factors driving customer churn and to make our model more transparent and understandable.
  • Unsupervised learning allowed us to identify distinct groups among churned customers and recognized the most critical group of churned customers.
  • Insights gained will inform improved business strategies for enhanced customer retention and sustainability.

25 of 25

Thank You