Dynamical System Modeling and Stability Investigation�DSMSI-2025

May 08-10, 2025, Kyiv, Ukraine

Audio models for Object Identification

​

Andriy Shatyrko, Andrii Ivanenko�

Introduction

The aim of this work is to investigate the capabilities of deep learning models for the acoustic detection and classification of five classes of objects and events: airplane, car/truck, UAV (drone), gunshots, and tank. We analyze the effectiveness of modern audio processing architectures (DistilHuBERT, wav2vec, BEATS) using recurrent, fully connected networks and transformers for extracting relevant features and classification, utilizing a dataset created from open sources. The results of this study aim to demonstrate the potential of deep learning for creating automated acoustic monitoring systems to enhance security measures and defense operations.

2

Dynamical System Modeling and Stability Investigation, DSMSI-2025

Study Focus & Objective

• Subject of the research: The application of modern Self-Supervised Learning (SelfSL) models for the automated acoustic classification of military objects and events.
• Object of the research: Deep learning models (DistilHuBERT, wav2vec, BEATS) and their architectures, particularly with Recurrent Neural Networks (RNN), Transformer layers, and classification heads, in the context of classifying audio data of airplanes, cars/trucks, UAVs (drones), gunshots, and tanks.
• Research Objective: To evaluate the effectiveness of modern SelfSL models and their architectural variations for the accurate classification of a specific set of military object and event sounds, identify the models and configurations that demonstrate the highest classification performance,

​

3

Dataset

In this study, audio data from several sources was used to form the training and test samples:

• Freesound: A large open online library of audio recordings distributed under Creative Commons licenses. Utilized for obtaining a wide range of sounds.
• BBC Sound Effects: An archive of high-quality sound effects recorded and cataloged by the British Broadcasting Corporation. Provided high-quality audio samples with detailed descriptions for the target classes.
• Drone Audio Dataset: A specialized dataset focused on the sounds of unmanned aerial vehicles (UAVs), containing recordings of various drone models indoors. Only clean drone recordings were used.
• YouTube: Several videos under Creative Commons licenses.

Using data from various sources allowed us to create our own dataset, including both drone sounds and other target sound classes. The total duration of audio in the dataset was:

• Aircraft: 9 minutes
• Car: 19 minutes
• Drone: 25 minutes
• Gunshots: 20 minutes
• Tank: 15 minutes

​

4

Architectures

• Figure illustrates the architectures of the sound classification models. Input features are extracted from pre-trained Self-Supervised Learning models (wav2vec 2.0, DistilHuBERT, BEATS). These features are then fed into one of three classification modules:
• Recurrent Neural Networks (RNN): Process sequences of audio features, considering temporal dependencies and context.
• Transformers: Also process sequences, utilizing a self-attention mechanism to model long-range dependencies.
• Fully Connected Classification "Head": Added on top of the original representations of wav2vec 2.0 and DistilHuBERT models for direct classification.

​

5

Accuracy

The BEATS model utilizing the Transformer architecture achieved the highest classification accuracy at 93%. The other models in the comparison exhibited accuracy results around 80%, which is significantly lower. It's also worth noting that the BEATS model with RNN showed moderate performance, further highlighting the effectiveness of the BEATS model with the Transformer modification.

6

Loss�

BEATS Transformer Model: Shows signs of early stopping in improvement on the validation set. Overfitting is noticeable as early as the first training epoch.

DistilHuBERT and Wav2Vec2 Models: The overfitting point is reached around the fourth iteration.

BEATS RNN Model: Unlike the other models, the BEATS RNN architecture, has not yet reached the overfitting point. However, its learning curve on the validation set may suggest it is approaching this limit.

​

7

Confusion matrix Wav2Vec2 and DistilHuBERT

8

The confusion matrices illustrate that the Wav2Vec2 and DistilHuBERT models effectively recognize the sounds of drones, cars, and gunshots. However, the classification of airplanes and tanks proved to be significantly more challenging, as indicated by the lower accuracy for these classes. Overall, while the models demonstrate the ability to detect the target sound events, they require further improvement in recognizing airplanes and tanks.

Confusion matrix RNN/Transformer BEATS�

The matrix shows that BEATS with Transformer achieved the best overall performance, but struggled with classifying airplanes and tanks. BEATS with RNN performed worse at recognizing drones, often confusing them with cars, but identified tanks better than other models.

9

Conclusion

This paper explored using deep learning and self-supervised learning (SelfSL) models (HuBERT, wav2vec, BEATS) for classifying military object sounds (airplane, car/truck, UAV, gunshots, tank). The BEATS model with a Transformer layer achieved the highest accuracy (93%). While UAV, car, and gunshot sounds were well-identified, airplane and tank classification was challenging. BEATS with RNN showed better tank detection but struggled with UAVs. The study confirms the potential of SelfSL for acoustic monitoring but highlights the need to improve airplane and tank recognition through new training methods, larger datasets, and multisensor integration.

10

Discussion and Future Extensions

Future research will include:

• Adding background sounds: To create a more realistic acoustic environment and improve the model's robustness to noise, various extraneous sounds such as human speech, birdsong, animal sounds, and natural phenomena (e.g., wind noise, rain) are planned for inclusion in the training data. This will enable the model to better distinguish target sounds against a backdrop of real-world environments.
• Direction of Arrival (DOA) estimation: Research into methods for determining the direction of arrival of a sound wave. Integrating DOA will allow the system to identify the precise origin of a sound and potentially focus analysis on sounds arriving from a specific direction, thereby filtering out irrelevant noise and increasing the accuracy of target object recognition.

​

11

Thank you for your attention

12