Feature Extractor
May 1st, 2025
1
5/21/2025
Data
2
5/21/2025
METHODS: Semi-automated Object Detection
3
5/21/2025
Semi-automated Object Detection
4
5/21/2025
threshold
Microscope Image
Thresholded Image
Contkid1_left_results3
Semi-automated Object Detection
2. Connected components function (sitk package, example segments below)
3. Groups as below based on number of pixels in each group (hyperparameter)
5
5/21/2025
Contkid1_left_results3
Semi-automated Object Detection
4. Manual intervention of results to remove unwanted detections
6
5/21/2025
Remove
Add
Contkid1_left_results3
Semi-automated Object Detection
7
5/21/2025
YOLO Object Detection
8
5/21/2025
Full image feature extraction
9
5/21/2025
Updates
10
5/21/2025
GROUPS
11
5/21/2025
HOG - GLOMERULUS
12
5/21/2025
HOG – RENAL TUBE
13
5/21/2025
SOBEL - GLOMERULUS
14
5/21/2025
SOBEL - GLOMERULUS
15
5/21/2025
SOBEL - RENAL
16
5/21/2025
SOBEL - RENAL
17
5/21/2025
CANNY - GLOMERULUS
18
5/21/2025
SOBEL - RENAL
19
5/21/2025
Questions
20
5/21/2025
To Do
21
5/21/2025
Statistics
22
5/21/2025
23
5/21/2025
Image Semi-Automated Results
Examples
24
5/21/2025
PAN GROUP
PANd7kid1-left�_result1
25
5/21/2025
PANd7kid1-left�_result2
26
5/21/2025
PANd7kid1-left�_result3
27
5/21/2025
PANd7kid1-left�_result4
28
5/21/2025
PANd7kid1-left�_result4
29
5/21/2025
PANd7kid1-left�_result4
30
5/21/2025
31
5/21/2025
CONTROL GROUP
Contkid1-left�_result1
32
5/21/2025
Contkid1-left�_result2
33
5/21/2025
Contkid1-left�_result3
34
5/21/2025
Contkid1-left�_result4
35
5/21/2025
Contkid1-left�_result4
36
5/21/2025
Clinical Details – from Professor Matsui
37
5/21/2025
Adding Clinical Annotations
38
5/21/2025
Experiment Updates
May 23, 2025
39
5/21/2025
Recommendations from Prof.Bise-san
1. Patch-based analysis� 1-1. Filter bank (e.g., Gabor filters)�•Select representative (distinctive) patches (around 10 samples)�→ Use PCA or similar methods to check whether the features can be separated� *Compare regions where the double lines are clearly visible in the control vs. diseased regions� *Example patch size: For glomeruli, use a rectangular patch roughly the size of the minor axis of a tubular epithelium ellipse
1-2. Use methods such as:�◦VAE (supervised)
◦SimCLR�→ Apply PCA → Use linear SVM or similar classifier
2. Pixel-wise filter bank analysis�•For example, to investigate differences in mitochondria between the inner and outer regions of renal tubules:�•Extract pixel groups from the inner region and the outer region of the ellipse (the renal tubule)
→ Perform clustering for each group using a Bag-of-Features-like approach
40
5/21/2025
Recommendations from Prof.Bise-san
1. Patch-based analysis� 1-1. Filter bank (e.g., Gabor filters)�•Select representative (distinctive) patches (around 10 samples)�→ Use PCA or similar methods to check whether the features can be separated� *Compare regions where the double lines are clearly visible in the control vs. diseased regions� *Example patch size: For glomeruli, use a rectangular patch roughly the size of the minor axis of a tubular epithelium ellipse
1-2. Use methods such as:�◦VAE (supervised) ** looking for pretrained model don’t really exist easily - what about monai zoo pathology https://github.com/Project-MONAI/model-zoo/tree/dev/models/pathology_nuclei_classification �◦SimCLR ** tried resent 50�→ Apply PCA → Use linear SVM or similar classifier
2. Pixel-wise filter bank analysis�•For example, to investigate differences in mitochondria between the inner and outer regions of renal tubules:�•Extract pixel groups from the inner region and the outer region of the ellipse (the renal tubule)
→ Perform clustering for each group using a Bag-of-Features-like approach
41
5/21/2025
Completed / TO DO
Datasets
42
5/21/2025
Patch wise small analysis
43
5/21/2025
Glomerulus
Inner
NOTE* visuals look different sizes but they are set to the same size this was just to illustrate locations
Experiments
44
5/21/2025
RESULTS
1. Clinician Annotation Regions
A) SimCLR (Resnet50) - PCA visual grouping * Glomerulus only *
2. ROI (region of interest)
A) HOG – region means are different in Histograms
45
5/21/2025
SimCLR Resnet Features - GLOMERULUS
46
5/21/2025
Channel 5: Combine channels 1,2,3 *scattering components
Visual Separation in Channel 4 and 5.
Very interesting alone 1-2-3 could not tell, but together they can !
Clinician Annotation Regions
HOG Features - GLOMERULUS
47
5/21/2025
ROI Annotation Regions
2. A)
HOG Features - INNER
48
5/21/2025
Note differences
ROI Annotation Regions
2. A)
HOG Features - OUTER
49
5/21/2025
Note differences
ROI Annotation Regions
2. A)
Conclusions and Next steps
50
5/21/2025
Clinician Annotations EXPERIMENTS
ROI – small cut sections of glomerulus/proximal tubes/distal tubes
51
5/21/2025
HOG Features
52
5/21/2025
SimCLR Features
53
5/21/2025
SimCLR Vgg19 Features - GLOMERULUS
54
5/21/2025
Channel 5: Combine channels 1,2,3 *scattering components
SimCLR Vgg19 Features - DISTAL
55
5/21/2025
SimCLR Vgg19 Features - PROXMIAL
56
5/21/2025
Channel 5: Not added for this experiment
SimCLR Resnet Features - GLOMERULUS
57
5/21/2025
Channel 5: Combine channels 1,2,3 *scattering components
SimCLR Resnet Features - PROXMIAL
58
5/21/2025
SimCLR Resnet Features - DISTAL
Not done yet
59
5/21/2025
Gabor Features - GLOMERULUS
To add
60
5/21/2025
Gabor Features - PROXMIAL
To add
61
5/21/2025
Gabor Features - DISTAL
To add
62
5/21/2025
ROI EXPERIMENTS
ROI – small cut sections of glomerulus/inner/outer sections
63
5/21/2025
Patch wise small analysis
64
5/21/2025
Glomerulus
Inner
NOTE* visuals look different sizes but they are set to the same size this was just to illustrate locations
SimCLR Resnet Features - GLOMERULUS
Reshaped to 256 by 256
** normalizing images before calculating features
65
5/21/2025
SimCLR Resnet Features - OUTER
66
5/21/2025
SimCLR Resnet Features - INNER
67
5/21/2025
HOG Features - GLOMERULUS
NOT normalizing
68
5/21/2025
Contkid1_leftkidney_2_featclassGlo_bb_annotation_2
HOG Features - GLOMERULUS
69
5/21/2025
HOG Features - GLOMERULUS
70
5/21/2025
Channel 1+2+3
**Mean values are .1 different per group
*Same graph different plots
HOG Features - GLOMERULUS
71
5/21/2025
HOG Features - OUTER
HOG Specific information see documentation.
orientations: 9
pixels_per_cell: 16 #assumes cubic shape
cells_per_block: 8 #assumes cubic shape
hog_intensity_rescale: 1 #0.1 #for visuals, rescale
72
5/21/2025
30 by 30
256 by 256
HOG Features - OUTER
73
5/21/2025
Channel 1+2+3
*Same graph different plots
HOG Features - OUTER
74
5/21/2025
Note differences
HOG Features - INNER
75
5/21/2025
HOG Features - INNER
76
5/21/2025
Channel 1+2+3
*Same graph different plots
HOG Features - INNER
77
5/21/2025
Note differences
Gabor Features
78
5/21/2025
Gabor Features - GLOMERULUS
79
5/21/2025
Contkid1_leftkidney_3_bb_2
PANd7kid1_middlekidney_3_bb_4
Maybe we could also improve ROIS?
256 by 256 patches, 0.1 frequency
Gabor Features - GLOMERULUS
80
5/21/2025
Contkid1_leftkidney_3_bb_2
PANd7kid1_middlekidney_3_bb_4
256 by 256 patches, 0.1 frequency
Gabor Features - GLOMERULUS
81
5/21/2025
Contkid1_leftkidney_3_bb_2
PANd7kid1_middlekidney_3_bb_4
256 by 256 patches, 0.1 frequency
Gabor Features - GLOMERULUS
82
5/21/2025
How should we better compare these?
256 by 256 patches, 0.1 frequency
Gabor Features - GLOMERULUS
83
5/21/2025
Zoom in last slide, also blue is shaded red is outline
256 by 256 patches, 0.1 frequency
Gabor Features - GLOMERULUS
84
5/21/2025
Maybe some outliers?
256 by 256 patches, 0.1 frequency
Gabor Features - OUTER
85
5/21/2025
256 by 256 patches, 0.1 frequency
20250220_Contkid1_leftkidney 5_bb_2
20250223_PANd7kid1_middlekidney_1_bb_4
Gabor Features - OUTER
86
5/21/2025
256 by 256 patches, 0.1 frequency
Gabor Features - OUTER
87
5/21/2025
256 by 256 patches, 0.1 frequency
Gabor Features - INNER
88
5/21/2025
256 by 256 patches, 0.1 frequency
PANd7kid1_middlekidney_2_1patch
Contkid1_leftkidney_4_0patch
Gabor Features - INNER
89
5/21/2025
256 by 256 patches, 0.1 frequency
PANd7kid1_middlekidney_2_1patch
Contkid1_leftkidney_4_0patch
Gabor Features - INNER
90
5/21/2025
256 by 256 patches, 0.1 frequency
Gabor Features - INNER
91
5/21/2025
256 by 256 patches, 0.1 frequency
Gabor Features - INNER
92
5/21/2025
256 by 256 patches, 0.1 frequency
Gabor Features
93
5/21/2025
Example Contkid1_leftkidney_3_bb_2
0.2 frequency
0.1 frequency
Pyfeat - GLOMERULUS
94
5/21/2025
Pyfeat - OUTER
95
5/21/2025
Pyfeat - INNER
96
5/21/2025
SUMMARY May 23 Report
1. Patch-based analysis� 1-1. Filter bank (e.g., Gabor filters)�•Select representative (distinctive) patches (around 10 samples)�→ Use PCA or similar methods to check whether the features can be separated� *Compare regions where the double lines are clearly visible in the control vs. diseased regions� *Example patch size: For glomeruli, use a rectangular patch roughly the size of the minor axis of a tubular epithelium ellipse
1-2. Use methods such as:�◦VAE (supervised) ** looking for pretrained model – more likely what about monai zoo pathology https://github.com/Project-MONAI/model-zoo/tree/dev/models/pathology_nuclei_classification �◦SimCLR ** tried resent 50�→ Apply PCA → Use linear SVM or similar classifier
2. Pixel-wise filter bank analysis�•For example, to investigate differences in mitochondria between the inner and outer regions of renal tubules:�•Extract pixel groups from the inner region and the outer region of the ellipse (the renal tubule)→ Perform clustering for each group using a Bag-of-Features-like approach
97
5/21/2025