1 of 22

Chest x-ray images for the detection of COVID-19: final report

Group 30:

110526007 翁崇恒

111522100 張瑋菱

111522104 廖柏諭

​

2 of 22

Outline

  • Introduction
  • Experiment Design and Result
  • Conclusion

3 of 22

Introduction

3

4 of 22

Goal of project

  • The accuracy of Classification is higher than the baseline.
  • Build an unsupervised learning for the segmentation of abnormal region.

​

​

4

5 of 22

Dataset

  • Dataset is from kaggle
  • It contains 29,986 images from 16,648 patients
  • 13,992 images are Negative, 15,994 images are Positive
  • source: https://www.kaggle.com/datasets/andyczhao/covidx-cxr2

​

​

5

negative

positive

6 of 22

Experiment Design and Result

6

7 of 22

Experiment Design

  • First, re-implement COVID‑Net, ensure the baseline.
  • Second, training a ViT-based classification model, compare its performance with COVID-Net.y
  • Third, by using patch-merging, explore probability of unsupervised segmentation.

​

​

7

8 of 22

Re-implement COVID‑Net

  • Official Github repository doesn’t provide ready to use model and training code.
  • We reference a repository that re-implement with pytorch and its performance is inferior: 91.22% , but without pretraining on ImageNet dataset.
  • Confirm the reference repository is correct: by adapt its model, we also get 91.25% accuracy.

​

​

8

https://github.com/lindawangg/COVID-Net

https://github.com/iliasprc/COVIDNet

9 of 22

Training a ViT-based classification model

  • By pre-trained model and strong regularization, we could train ViT model with this small dataset.
  • We achieve 97.5 for ViT model.

​

​

9

10 of 22

Unsupervised segmentation via using patch-merging (1)

  • Visualization especially design for Vision Transformer.
  • TOKEN MERGING: YOUR VIT BUT FASTER(ToMe)

​

​

​

10

We see the parrot’ contour is showed by patch merging

Will model shows affected parts?

11 of 22

Unsupervised segmentation via using patch-merging (2)

  • Apply ToMe on classification model we just trained.

​

​

11

Input image

Merging rate

4

Merging rate

8

Merging rate

20

We see some lung region is been show up.

But is it correct?

12 of 22

Unsupervised segmentation via using patch-merging (3)

  • Leverage another dataset: QaTa-COV19 Dataset
  • For those performs good on CXR2, should also performs good on QaTa-COV19 Dataset.

​

12

https://www.kaggle.com/datasets/aysendegerli/qatacov19-dataset

Experts(doctor) draws the segmentation masks of infected region.

13 of 22

Unsupervised segmentation via using patch-merging (4)

  • Result of model trained on CXR2 on QaTa-COV19

13

Input image/

Ground-truth

Merging rate

4

Merging rate

8

Merging rate

20

Not showing meaningful segmentation here.

14 of 22

Unsupervised segmentation via using patch-merging (5)

  • Result of model trained on CXR2 on QaTa-COV19

14

Input image/

Ground-truth

Merging rate

4

to

Merging rate

20

It barely draw the lung region but infected region.

More precisely, it may just merge the dark region.

15 of 22

Unsupervised segmentation via using patch-merging (6)

  • Why the patch merging trick fail on QaTa-COV19 dataset?
  • Since Vision Transformer see things globally, so it lacks of activated local features?
  • The classification task may not need “region” information to get good performance.

15

16 of 22

Unsupervised segmentation via using patch-merging (7)

  • To confirm the assumption, we trained the “supervised” segmentation with ViT. Then we applied ToMe on it, to see whether the result becomes better.

16

17 of 22

Unsupervised segmentation via using patch-merging (8)

  • ViT segmentation.

​

17

input

Ground truth

prediction

For “supervised” segmentation, model performs pretty good on QaTa-cov19 test dataset.

We see it locates not lung region but infected parts.

18 of 22

Unsupervised segmentation via using patch-merging (9)

  • Result of model trained on supervised segmentation on QaTa-COV19.

​

18

Input image/

Ground-truth

Merging rate

4

to

Merging rate

20

Still it draw the lung region instead of infected region.

19 of 22

Unsupervised segmentation via using patch-merging (10)

  • After all the effort we do for our unsupervised segmentation method, we can proudly, sentence this method to death.

19

20 of 22

Conclusion

20

21 of 22

Conclusion

  • We re-implemented COVID‑Net, and using ViT backbone, achieve very high performance.
  • We propose a new unsupervised segmentation method. Wish to locate the infected region of Covid19. But with carefully experiment, this method is been proved failed.

21

22 of 22

Thank you for listening

22