Dynamical System Modeling and Stability Investigation�DSMSI-2023

Dedicated to the 77th anniversary of the outstanding Ukrainian scientist

professor Denys Khusainov

December 19-21, 2023, Kyiv, Ukraine

USE OF CONVOLUTIONAL NEURAL NETWORKS FOR IDENTIFYING ADDITIONAL FEATURES ON A DIGITAL IMAGE OF HUMAN FACE

​

​

Kateryna Merkulova and Bohdan Pavliukh

Taras Shevchenko National University of Kyiv

�

Introduction

Currently, recognition and identification technologies are very widely used in all areas. Technology designed to detect additional features on a person's face, such as the presence of glasses, a headgear, a medical mask and a beard, can be used to solve a number of important tasks.

The most obvious task that the researched technology can handle is the collection of statistical information (which may have commercial value). Such technology can be used in access control systems to improve security, for example by requiring users to remove a mask or glasses for identification, which can help prevent unauthorized access or bypassing.

The tool for recognizing additional features in a digital image of a human face can also be used for narrowly targeted purposes, for example, to track the observance of masking by visitors to a supermarket (or any other crowded place).

​

2

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Task definition and solution methods

3

Dynamical System Modeling and Stability Investigation, DSMSI-2023

(1)

(2)

(3)

(4)

Experimental research

The purpose of this study is to determine the effectiveness of using convolutional neural networks for recognizing additional features in images of human faces obtained from streaming video.

It was decided to recognize 4 additional features of the face (presence of glasses, medical mask, headdress and beard) and additionally the gender of the person, accordingly, 5 convolutional neural networks were developed.

For some of the neural networks, datasets were found in public sources (for example, on Kaggle), for others datasets had to be generated independently (due to this, such datasets have fewer images).

4

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Experimental research

Table 1. Accuracy and loss of results when training a CNN to detect the presence of a medical mask

5

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Experimental research

Figure 1. Accuracy graphs of the developed CNNs for: a) gender identification; b) glasses recognition; c) beard recognition; d) headdress recognition; e) medical mask recognition

6

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Experimental research

Figure 2. Determination of additional features on human faces with the help of developed CNNs: a) a man with a beard in a headdress and a woman in glasses; b) a man with an improperly worn medical mask and sunglasses and a man without additional features on his face; c) a man with a beard and a headdress and a woman with a headdress; d) a man in a medical mask and a man with a beard and glasses.

7

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Experimental research

Table 2. Final results of accuracy and loss on the validation sets for the developed CNNs

8

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Conclusion

This work is devoted to the study of the effectiveness of using convolutional neural networks for the task of recognizing additional features on a digital image of a human face. For this purpose, CNNs was developed to recognize the following additional features: the presence of glasses, a beard, a headdress, a medical mask, and a person's gender.

All the developed models showed a good result both on training and validation data sets (which is indicated by the presented accuracy tables and graphs), and during their experimental application for streaming video. The final values of accuracy and loss on the validation data sets for developed CNNs are as follows: for the detection of glasses, the accuracy is 99.72%, the loss is 1.39%; for detecting a medical mask, the accuracy is 96.14%, the loss is 15.37%; for headdress detection, the accuracy is 100%, the loss is 0.05% (such a high result is probably related to the not very good content of the input data set); for detecting a beard, the accuracy is 85%, the loss is 44.28%; for gender determination, the accuracy is 98.62%, the loss is 3.77%. Among the shortcomings of the developed models, a decrease in the accuracy of work when the image was taken in poor lighting conditions (which was expected) can be highlighted.

9

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Conclusion

One of the interesting results obtained is the fact that the fewest images in the training sets had a beard and a headdress, and, at the same time, their models gave the opposite result - based on the obtained accuracy results, the model for recognizing the beard has the largest loss, and the model for recognizing headdress has the smallest. This result is most likely related to the small number of images in the training sets.

For further research on this topic, models can be developed to identify other additional features, for example, a mustache, as well as such features of a person as age, race, emotions. It is also possible to increase the number of resulting classes for CNNs, for example, to define different types of headdress or to define more types of glasses.

To improve the quality of the obtained results, it is possible to expand data sets, as well as apply additional methods of image preprocessing, which would help solve some difficulties, for example, reduce the influence of the level of illumination on the result of the work of CNNs.

10

Dynamical System Modeling and Stability Investigation, DSMSI-2023

Thank you for your attention