Mahsa Famil Barraghi
Department of Computer Science
University of Tabriz, Iran
ICAISV-2023
Habib Izadkhah
Department of Computer Science
University of Tabriz, Iran
Deep Parallel Self-Organizing Maps
for Visual Classification
SELF
ORGANIZING
MAP
Related
work
Deep self-
organizing
maps
Experimental
results
& discussion
1/18
Concolusion
Content
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
2/18
Definition
Self-Organizing Map
Better capability of revealing the overlapping structure in clusters
Kohonen 1982
Visualize high-dimensional data sets
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
3/18
Algorithm
Self-Organizing Map
SOM Algorithm
Initialize all weights of the SOM
Select a vector x randomly in the
training set.
Find the best matching unit BMU
Update the weight vector of BMU
and its neighboring
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
4/18
Liu, Wang and Yong 2015, Wickramasinghe,
Amarasinghe and Manic 2019
Related Works
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
5/18
Architecture
DeepSOM
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
6/18
Detail
DeepSOM
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
7/18
Training
DeepSOM
Combines information of maps.
The array is reshaped to a 2D grid
Step 1: Weight initialization
Step 2: SOM Layer
Step 3: Sampling Layer:
This process is repeated until the last layer
Step 4: Repeat Steps 2-3
Steps are repeated until the maximum number of iterations (epochs) has been reached
Step 5: Repeat 2-4:
Weights of the network are randomly initialized.
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
8/18
Parameters
Results
Layers | Kernel size | Stride | SOM map Size |
Hidden Layer 1 | 10×10 | 2 | Changeable |
Hidden Layer 2 | 6×6 | 1 | Changeable |
Output Layer | 5×5 | 1 | 8×8 |
model | Map sizes of the first layer | Map sizes of the second layer | ||
1 | 20×20 | 14× 14 | 15× 15 | 14× 14 |
2 | 22× 22 | 16× 16 | 15 ×15 | 14 ×14 |
3 | 22 ×22 | 14 ×14 | 16 ×16 | 14 ×14 |
4 | 24 ×24 | 16 ×16 | 16 ×16 | 14 ×14 |
MNSIT
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
9/18
Accuracy
Results
Model | Map sizes of the first layer | Map sizes of the second layer | Train accuracy | Test accuracy | ||
1 | 20×20 | 14× 14 | 15× 15 | 14× 14 | 84.47 | 81.56 |
2 | 22× 22 | 16× 16 | 15 ×15 | 14 ×14 | 85.75 | 82.26 |
3 | 22 ×22 | 14 ×14 | 16 ×16 | 14 ×14 | 87.81 | 84.19 |
4 | 24 ×24 | 16 ×16 | 16 ×16 | 14 ×14 | 86.63 | 83.53 |
Model | Map sizes
| Train accuracy | Test accuracy |
1 | 8×8 | 62.23 | 61.70 |
2 | 12×12 | 63.20 | 62.56 |
3 | 16×16 | 65.43 | 65.03 |
4 | 20×20 | 66.97 | 66.35 |
5 | 24×24 | 69.45 | 68.89 |
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
10/18
Tools
Visual Data Mining
Hit maps
U-Matrix
Class label distribution
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
11/18
U-Matrix
Visual Data Mining
Yellow areas
mean that they
are far apart
It shows the distance between weight vectors of neighboring neurons
Dark blue area, means that the nodes in this area are very
similar
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
12/18
U-Matrix
Visual Data Mining
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
13/18
Hit maps
Visual Data Mining
Based on the number of times a particular neuron becomes a winning neuron for a particular class
How often a neuron is selected as the winning neuron
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
14/18
Hit maps
Visual Data Mining
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
15/18
Class label distribution
Visual Data Mining
The similar the two numbers are to each other, the closer they get to each other on the map
Numbers that are similar to each other are placed close together
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
16/18
Class label distribution
Visual Data Mining
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
17/18
Concolusion
It is easy to use and understand compared to other deep models.
It improves the accuracy of simple SOM by 23.65%.
It improves accuracy by 27.5% compared to the three-layer convolutional method.
Two different map sizes creates a balance between over-fitting and under-fitting.
It improves the accuracy of the model of Wickramasinghe by 1.09%
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
18/18
References
[1] T. Kohonen.Self-Organizing Maps.Book published by Springer-Verlag Berlin Heidelberg New York. (2001)
[2] D. Alahakoon, S. Halgamuge, B. Srinivasan. Dynamic self-organizing maps with controlled growth for knowledge discovery. IEEE Trans Neural Netw
11(3):601-14. doi: 10.1109/72.846732. (2000)
[3] A. Rauber, D. Merkl,. M. Dittenbach. The Growing Hierarchical Self-Organizing Map: Exploratory Analysis of High-Dimensional Data. IEEE TRANSACTIONS
ON NEURAL NETWORKS. (2002)
[4] A. Fonseka, D. Alahakoon. Exploratory data analysis with Multi-Layer Growing Self-Organizing Maps. Fifth International Conference on Information and
Automation for Sustainability. (2010)
[5] H. Dozono, G. Niina, S. atoru Araki. Convolutional Self Organizing Map. International Conference on Computational Science and Computational
Intelligence (CSCI). (2016)
[6] M. Sakkari, M. Zaied1. A Convolutional Deep Self-Organizing Map Feature extraction for machine learning. Springer Science Business Media, LLC, part of
Springer Nature (2020)
[7] S. Aly, S. Almotairi. Deep Convolutional Self-Organizing Map Network for Robust Handwritten Digit Recognition. IEEE 10.1109/ACCESS.2020
.3000829, Page(s): 107035 - 107045. (2020)
[8] L. Elend, O. Kramer. Self-Organizing Maps with Convolutional Layers. Springer Nature Switzerland AG WSOM 2019,AISC 976,pp. 23–32.(2020)
[9] N. Liu, J. Wang, Y. Yong. Deep Self-Organizing Map for Visual Classification. 978-1-4799-1959-8/15/ IEEE. (2015)
[10] C. Wickramasinghe, K, Amarasinghe, M. Manic. Parallalizable Deep Self-Organizing Maps for Image Classification. DOI
10.1109/TII.2019.2906083, IEEE (2019)
Deep Self-Organizing Maps for Visual Classification Mahsa Famil Barraghi & Habib Izadkhah
THANK YOU!
ICAISV-2023