1 of 16

Automated Perineural Invasion detection

from Whole Slide image using organ-specific approach

Gawon Lee^1*, Da-young Baik¹, Seung-un Jang¹, HwanSeung Yoo¹, Junhyeok Lee¹

¹Division of Biomedical Engineering, Hankuk University of Foreign Studies, Republic of Korea

^*^{gwlee163@gmail.com}

2 of 16

Contents

01. Background

02. Method

03. Results

04. Summary

3 of 16

Task:

Detection of perineural invasion(PNI) in multiple organ cancer(Colon, Pancreas and Prostate)

Data Description:

Train set annotations : Nerve / PNI / Tumor / Non-nerve and Non-tumor

	Colon	Pancreas	Prostate
Train set	50	50	50
Validation set	10	10	10
Test set	20	20	20

01. Background

4 of 16

02. Method – Key points

Huge image size

Patch-based learning

Different shape of tumor

depending on organ

Organ-specific Classification

Very tiny PNI

compared to WSI size

Two-step approach

Classification step and Segmentation step

For more precise results

Utilize multi-scale WSI (5x, 20x)

5 of 16

02. Method – Pipeline

Patches

with High

Probabilities

6 of 16

02. Method – Preprocessing for training

Patch extraction for patch-based training

WSI

Make annotations locate center of patch

Multi-magnification patches

Dilation of mask for PNI segmentation

Original label

Dilated label

224 x 224

20x

5x

7 of 16

02. Method – Model Training (Classification)

Multi-scale & organ-specific classification model training
Input: patch(224 x 224) → Output: class(Nerve / PNI / Tumor / Non-nerve and Non-tumor)
Model: EfficientNet-b0
Loss function: Cross entropy loss
Data augmentation: Affine, Gaussian noise, blurring, adjusting brightness and contrast.

20x

5x

Model(5x)

Nerve?

PNI?

Tumor?

Benign?

Model(20x)

Colon

Prostate

Pancreas

Train a total of 6 classification models

Optimizer: Stochastic gradient descent (SGD)

8 of 16

02. Method – Model Training (Segmentation)

Transferred and organ-specific segmentation model training
Input: patch(224 x 224) → Output: PNI probability map of patch
Model: Unet with pretrained encoder(EfficientNet-b0)
Loss function: Dice loss
Data augmentation: Affine, Gaussian noise, blurring, adjusting brightness and contrast.

5x

Segmentation

model

Colon

Prostate

Pancreas

Integrated Organ Classification

Encoder

5X

Nerve?

PNI?

Tumor?

Benign?

Optimizer: Adam

9 of 16

02. Method – Inference (classification)

Get an Averaged Probability map (5X and 20X)

Overlapped patches were utilized (73% for 5X, 93% for 20X)

In inference step, patches from whole slide image are tested in sliding window way with overlap. The overlap of 73% was applied to micro magnification of two hundreds and the overlap of 93% was applied to micro magnification of fifty, respectively. Then, the final probability map is generated by averaging two probability maps from both magnifications.

(수정본)

In inference step, patches from whole slide image are tested in sliding window way with overlap. The overlap of 73% was applied to micro magnification of two hundreds and the overlap of 93% was applied to micro magnification of fifty, respectively. Each patch was filled with 1 if it was classified into PNI. But in order not to be over 1, total probability in each pixel was divided by the number of overlaps. Two PNI probability maps from both magnifications are produced in the same way as previously stated and resized into micro magnification of fifty. Then, the final probability map is generated by averaging two probability maps from both magnifications.

10 of 16

02. Method – Inference (classification)

Patches with PNI probability > 70%

Input of the Segmentation Network

(Red boxes)

11 of 16

02. Method – Inference (segmentation)

Colon

20X

5X

Encoder

PNI Probability Map

Mean

Patches with high probabilities (>0.7)

U-Net

PNI Segmentation

(Sliding Window Inference)

Patches

5X

50 Slides

Prostate

Pancreas

PNI Segmentation

Organ-specific Classification

Overlapped patches were utilized (98% for 5X)

12 of 16

02. Method – Inference (post-processing)

Skeletonized

Segmentation prediction

Skeletonization

Remove small objects containing fewer than 4 pixels

Result with noise

Clear result

13 of 16

03. Results

Patches of 5x

prediction

14 of 16

03. Results

Ground Truth

Prediction

PNI Probability Map

PNI

Non-PNI

WSI

ROI

Non-PNI

15 of 16

04. Summary

Objective

Ours

Detection of perineural invasion in multiple organ cancer

Organ-specific approach
Two-step approach (Classification and Segmentation step)
Utilize multi-scale WSIs (5x and 20x)

16 of 16

Thank you !

^{gwlee163@gmail.com}