A Deep Learning Based

Ensemble Approach

for Gastrointestinal

Disease Detection with XAI

SABAH SUBSECTION

Dewan Ziaul Karim, Tasfia Anika Bushra

and Shoaib Ahmed Dipu

6^th IEEE International Conference on Artificial Intelligence in Engineering and Technology

26-28 August 2024

Organizer and Technical Sponsor:

Co-Organizers:

IICAIET 2024

Content

01

02

03

04

05

06

07

Introduction

Prior Work

Proposed Methodology

Proposed Solutions

Result Analysis and Discussion

Future Works

Conclusion

IICAIET 2024

Introduction

Gastrointestinal disorders are conditions that affect the digestive system as a whole.

Polyps, tumors, infections, ulcerative colitis, and diverticulitis are some of the most prevalent reasons of death in people [1] [2].

Analyzing pictures and films of the Gastrointestinal tract is becoming tedious for gastroenterologist as number of patients and data have been increasing rapidly.

DL can play a major role to ease the whole process as it extracts the features automatically.

IICAIET 2024

Proposed

Research Project

IICAIET 2024

Prior Work

IICAIET 2024

• A system built on convolutional neural network and time frequency analysis. [3]
• Concatenated neural network model was built by combining the retrieved features of the VGGNet and InceptionNet networks. [4]
• A system averaging three deep learning architectures known as GoogleNet, AlexNet, and ResNet-50 was built. [5]
• Transfer learning-based approach was adopted for gastrointestinal tract-based disease detection. [6]
• Approach based on the integration of deep convolutional and geometric features was adopted. [7]

IICAIET 2024

Proposed Methodology

IICAIET 2024

• Divided into 4 main parts
• Data Preprocessing
• Custom CNN Model
• Pretrained Models
• Ensemble Model
• How do we do the ensemble?
• Taking the Custom CNN model and top 2 pretrained models

Custom CNN Model

+

Top 2 Pretrained Models

Fig 2. Way to build Ensemble Model

Ensemble Model

Fig 1. Workplan

IICAIET 2024

Dataset Details

IICAIET 2024

• Collected from Kvasir dataset [8].
• Considered 3 classes: esophagitis, ulcerative colitis and polyps.
• Initially, 1000 images in each class.
• Augmented to 2000 images per class.
• In total 2000*3=6000 images
• Total: Training: 4500, Validation: 1200, Testing: 300.
• Individual Class: Training: 1500, Validation: 400, Testing: 100.

Fig 3. Sample images from the dataset

IICAIET 2024

Dataset Preprocessing

IICAIET 2024

• Outliers Removal
• The outliers were eliminated to provide a dataset that might be error-free.
• Image Resizing
• Resized to 224x224, RGB images.
• Normalization
• Minmax Normalizer
• Z = X - X_min / (X_max- X_min)
• Augmentation
• Shearing, rotating, shifting and flipping

IICAIET 2024

Custom CNN Model Architecture

IICAIET 2024

Fig 4. Custom CNN Model Architecture

• 8 Convolution layers. Filter size: 32, 64, 64, 128, 256, 512, 512, 1024
• Followed by

Batch Normalization [9] and

Max Pooling [10] layers.

• Followed by

4 Dense layers.

Units: 1024, 512, 64, and 3 respectively.

• Regularizer:

L2 (.00001).

• Activation Function: ReLU.

IICAIET 2024

Pretrained Models

IICAIET 2024

8 Pretrained Models

​

• DenseNet121
• ResNet50V2
• VGG16
• VGG19
• InceptionV3
• InceptionResNetV2
• MobileNetV2 [26]
• Xception

• Weights: ImageNet
• Last layer is removed
• Other hyperparameters remained same

IICAIET 2024

Ensemble Model

IICAIET 2024

• With the technique of ensemble modeling, a machine learning model integrates various base models to provide generalized predictions by combining the predictive capability of each of its parts.
• Advantages :
• Performance enhancement
• Increasing reliability
• Provides more generalization and accuracy
• Reduces variation or spread of prediction
• Which models will create the ensemble?
• Custom CNN model + Top 2 pretrained models

IICAIET 2024

Training Parameters

IICAIET 2024

• Data Utilization: 6000 (2000 per class)
• Train: 4500
• Validation: 1600
• Test: 400
• Augmentation Techniques:
• Shear
• Rotation
• Flip
• Width_shift
• Batch Size: 32
• Epochs: 50

• Optimizer: Adam
• Regularizer: L2 (0.000001)
• Minimum Learning Rate: .00000001
• Loss: Categorical Cross Entropy
• Callback: ReduceLROnPlateau
• Metrics Used: Accuracy, Loss, Precision & Recall

IICAIET 2024

Result Analysis

Custom CNN

IICAIET 2024

• Accuracy
• 98.68% training accuracy
• 97.58% validation accuracy

Fig 5. Custom CNN Model Accuracy

• Loss
• 3.7% training loss
• 8.1% validation loss

Fig 6. Custom CNN Model Loss

IICAIET 2024

Result Analysis

Comparison with Pretrained Models

IICAIET 2024

IICAIET 2024

Result Analysis

Ensemble Model Results

IICAIET 2024

• Custom CNN + ResNet50V2 + Xception = Ensemble Model
• Accuracy
• Training Accuracy = 98.95%
• Validation Accuracy = 98.00%
• Loss
• Training Loss = 3.6%
• Validation Loss = 7.7%

Fig 7. Ensemble Model Accuracy

IICAIET 2024

Result Analysis

Ensemble Model vs Custom Model

IICAIET 2024

Fig 8. Accuracy Comparison

Fig 9. Loss Comparison

IICAIET 2024

Result Analysis

Evaluation on Test Dataset

IICAIET 2024

Confusion Matrix

• The unseen test dataset of 300 images, divided into 3 classes of 100 images each
• All 100 unseen samples from the esophagitis class were correctly identified.
• For the other two classes – polyps and ulcerative colitis, only 2 samples were incorrectly classified for each class.

Fig 10. Confusion Matrix

IICAIET 2024

Result Analysis

Evaluation on Test Dataset

IICAIET 2024

Classification Report

• The unseen test dataset of 300 images, divided into 3 classes of 100 images each
• For esophagitis, precision, recall and F1 score - all produce 100% score
• For the other two classes – polyps and ulcerative colitis, 98% was achieved for all attributes
• Final accuracy on Test dataset = 99%

Fig 11. Classification Report

IICAIET 2024

Result Analysis

Evaluation on Test Dataset

IICAIET 2024

ROC Curve and AUC Score

• The unseen test dataset of 300 images, divided into 3 classes of 100 images each
• Esophagitis vs the Rest = 1.00
• Polyps vs the Rest = .98
• Ulcerative-colitis vs the Rest = .98
• AUC Score based on 3 classes = .9899

Fig 12. ROC Curve for 3 Classes

IICAIET 2024

Result Visualization Using XAI - LIME

IICAIET 2024

Fig 13. (a) Random Esophagitis Image (b) Top 5 superpixels using LIME

IICAIET 2024

IICAIET 2024

Fig 14. (a) Potential Positive Areas (b) Areas Having Weight Value of 0.1

Result Visualization Using XAI - LIME

IICAIET 2024

IICAIET 2024

• Heatmap visualization based upon weights
• The colorbar shows the values of the weights
• Blue denotes Positive
• Red denotes Negative
• The darker the color, the more positive or negative weight that area holds

Fig 15. Heatmap Visualization with LIME

Result Visualization Using XAI - LIME

IICAIET 2024

Future Works

IICAIET 2024

• While this research only focuses on classification without segmentation tasks, U-net topologies might improve segmentation efforts.
• This research can also be expanded to categorize different illnesses.
• Moreover, developing a mobile application will increase accessibility and user experience.

IICAIET 2024

Conclusion

IICAIET 2024

• Proposing a deep learning model for classifying GI illnesses was the aim of this study.
• This work suggested and analyzed a complete ensemble based approach on top of a custom CNN model and eventually achieved 98.95% training and 98.00% validation accuracy.
• It is expected that this research will minimize the work of gastroenterologists in disease identification tasks and improve the overall medical image classification system significantly.

IICAIET 2024

References

IICAIET 2024

1. Tehranian, S., Klinge, M., Saul, M., Morris, M., Diergaarde, B. and Schoen, R.E., 2020. Prevalence of colorectal cancer and advanced adenoma in patients with acute diverticulitis: implications for follow-up colonoscopy. Gastrointestinal endoscopy, 91(3), pp.634-640.
2. V. Autores, “The top 10 causes of death.” url: https://www.who.int/newsroom/fact-sheets/detail/the-top-10-causes-of-death, 2018.
3. Mohapatra, S., Nayak, J., Mishra, M., Pati, G.K., Naik, B. and Swarnkar, T., 2021. Wavelet transform and deep convolutional neural network-based smart healthcare system for gastrointestinal disease detection. Interdisciplinary Sciences: Computational Life Sciences, 13, pp.212-228.
4. Haile, M.B., Salau, A.O., Enyew, B. and Belay, A.J., 2022. Detection and classification of gastrointestinal disease using convolutional neural network and SVM. Cogent Engineering, 9(1), p.2084878.
5. Hmoud Al-Adhaileh, M., Mohammed Senan, E., Alsaade, W., Aldhyani, T.H.H., Alsharif, N., Abdullah Alqarni, A., Uddin, M.I., Alzahrani, M.Y., Alzain, E.D. and Jadhav, M.E., 2021. Deep learning algorithms for detection and classification of gastrointestinal diseases. Complexity, 2021, pp.1-12.
6. Escobar, J.P., Gomez, N., Sanchez, K. and Arguello, H., 2020, August. Transfer learning with convolutional neural network for gastrointestinal diseases detection using endoscopic images. In 2020 IEEE Colombian Conference on Applications of Computational Intelligence (IEEE ColCACI 2020) (pp. 1-6). IEEE.
7. Sharif, M., Attique Khan, M., Rashid, M., Yasmin, M., Afza, F. and Tanik, U.J., 2021. Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images. Journal of Experimental & Theoretical Artificial Intelligence, 33(4), pp.577-599.
8. Konstantin Pogorelov, Kristin Ranheim Randel, Carsten Griwodz, Sigrun Losada Eskeland, Thomas de Lange, Dag Johansen, Concetto Spampinato, Duc-Tien Dang-Nguyen, Mathias Lux, Peter Thelin Schmidt, et al. Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. In Proceedings of the 8th ACM on Multimedia Systems Conference, pages 164–169, 2017.
9. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. pmlr, 2015.
10. Dominik Scherer, Andreas Muller, and Sven Behnke. Evaluation of ¨ pooling operations in convolutional architectures for object recognition. In Artificial Neural Networks–ICANN 2010: 20th International Conference, Thessaloniki, Greece, September 15-18, 2010, Proceedings, Part III 20, pages 92–101. Springer, 2010.

IICAIET 2024

THANK YOU

https://www.bracu.ac.bd/about/people/shoaib-ahmed-dipu

shoaib.ahmed@bracu.ac.bd

Kha 224 Bir Uttam Rafiqul Islam Avenue, Merul Badda, Dhaka, Bangladesh

Shoaib Ahmed Dipu

IICAIET 2024

SABAH SUBSECTION

6^TH IEEE INTERNATIONAL

CONFERENCE ON ARTIFICIAL INTELLIGENCE IN ENGINEERING AND TECHNOLOGY

26-28 AUGUST 2024

IICAIET 2024