Submitted by

Rachana C Nair

�

Vigilant Worker Protection and Efficiency Measure System

ARM AI Developer Challenge

AGENDA

ABSTRACT

PROBLEM STATEMENT

OBJECTIVE

METHODOLOGY

IMPLEMENTATION

DATASET DETAILS

BLOCK DIAGRAM

CIRCUIT DIAGRAM

THEORITICAL EXPLANATION

MATHEMATICAL EXPLANATION

OUTCOMES

RESULTS

REFERENCES

​

ABSTRACT

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

• SafeMineAI System: An innovative system integrating IoT, computer vision, and real-time analytics to enhance miner safety and operational efficiency.
• Wearable Sensors & Data Processing: Utilizes wristband sensors to monitor worker health and environmental conditions, with data processed by an STM32MP135-FDK unit and analyzed on a local server.
• Real-time Monitoring & Alerts: Provides continuous health tracking, instant alerts, lift/rest area management, video documentation, and object detection for safety compliance.
• Machine Learning Integration: Employs a CNN model on an STM32 microcontroller for object detection and a time-series algorithm for predictive worker health analysis.
• Impact & Future Scope: Aims to improve mine safety through automation, with future enhancements focusing on data reliability, scalability, and seamless communication in mining environments.

PROBLEM STATEMENT

The mining industry presents a challenging landscape where workers face hazardous conditions that endanger their safety and well-being. Conventional safety measures often fall short in providing real-time health monitoring and proactive hazard detection. Timely intervention to mitigate health risks is lacking, leading to workplace accidents, injuries, and even fatalities. This critical gap underscores the pressing need for an innovative solution that ensures continuous worker safety and well-being in such environments.

�

OBJECTIVE

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

• To develop an advanced safety system for mining environments using IoT, computer vision, and real-time data analytics.
• To implement wearable sensor technology for continuous health and environmental monitoring of miners.
• To provide real-time alerts and notifications for hazardous conditions, ensuring timely intervention.
• To integrate machine learning models for object detection and predictive health analysis of workers.
• To enhance compliance with safety protocols through automated monitoring and incident documentation.
• To establish a reliable communication system using wired WiFi to ensure stable data transmission in mines.
• To improve overall mining safety and efficiency by reducing health risks and optimizing operational workflows.

IMPLEMENTATION

1. CNN-Based Model Deployment on STM32 Microcontroller Using TensorFlow Lite

This involves converting a trained CNN model to TensorFlow Lite format and then optimizing it for deployment on the STM32 microcontroller. Here are the steps:

​

Steps:

​

1. Train your CNN model using TensorFlow Lite
2. Convert the trained model to TensorFlow Lite:

After training, convert the model to TensorFlow Lite format (.tflite) using the TensorFlow Lite Converter:

python

import tensorflow as tf # Load your trained model model = tf.keras.models.load_model('path_to_model') # Convert the model to TensorFlow Lite format converter = tf.lite.TFLiteConverter.from_keras_model(model) tflite_model = converter.convert() # Save the converted model with open('model.tflite', 'wb') as f: f.write(tflite_model)

3. Optimize the model for deployment on STM32:
• Use tools like TensorFlow Lite for Microcontrollers (TFLite Micro) to optimize the model for low-power embedded systems.
• Convert your model further (if necessary) to reduce size and improve performance. For example, use quantization to reduce precision (e.g., from float32 to int8)
• .
4. Prepare STM32 Environment:
• Install STM32CubeIDE to manage STM32 development.
• Install TensorFlow Lite for Microcontrollers SDK, which allows you to run your .tflite model on microcontrollers.

​

• Integrate the TFLite model with STM32 Code:
• Use STM32CubeMX to configure peripherals and create a project with STM32CubeIDE.
• Include the TensorFlow Lite micro library and your .tflite model file in the STM32 project.
• Write the inference code to load the model, process inputs, and get the output for object detection. Here’s an example of what that might look like in embedded C/C++:

cpp

#include "tensorflow/lite/micro/all_ops_resolver.h" #include "tensorflow/lite/micro/micro_interpreter.h" // Define the TFLite model buffer and interpreter const tflite::Model* model = ::tflite::GetModel(g_model_data); // g_model_data is the model buffer tflite::ops::micro::AllOpsResolver resolver; tflite::MicroInterpreter interpreter(model, resolver, tensor_arena, tensor_arena_size, &error_reporter); // Run inference interpreter.Invoke();

​

IMPLEMENTATION

6. Test and Deploy:
• Compile the code and flash the firmware onto the STM32 microcontroller.
• Test the model using a camera or sensor data for object detection.

​

2. Time-Series Algorithm for Predictive Health

This involves using time-series data (e.g., heart rate, temperature, or other physiological parameters) to predict workers' fitness for work.

​

Steps:

​

1. Data Preprocessing:
1. Clean the time-series data, ensuring it's in the correct format and normalized.
2. If necessary, use sliding windows to create features for training.
2. Train a Predictive Model (e.g., LSTM):
• Use an LSTM (Long Short-Term Memory) network to model sequential dependencies in the data. Here's an example using Keras:

python

import tensorflow as tf from tensorflow.keras.models import Sequential from tensorflow.keras.layers import LSTM, Dense # Assuming `X_train` is the time-series data and `y_train` is the target (fitness for work) model = Sequential() model.add(LSTM(64, activation='relu', input_shape=(X_train.shape[1], X_train.shape[2]))) model.add(Dense(1, activation='sigmoid')) # For binary classification (fit/unfit) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) model.fit(X_train, y_train, epochs=10, batch_size=32)

1. Convert the Model to TensorFlow Lite:
2. Similar to the CNN model, convert this LSTM model to TensorFlow Lite format using:

python

tflite_model = tf.lite.TFLiteConverter.from_keras_model(model).convert()

1. Optimize and Deploy to STM32:
3. Use the same steps as with the CNN model for optimization and deployment to STM32.

DATASET DETAILS

Source Link:

• https://www.kaggle.com/datasets/shayanfazeli/heartbeat
• https://ieee-dataport.org/datasets
• https://www.data.gov.in/catalogs/?ministry=Election%20Commission%20of%20India

​

Acquisition Details:

• From wearable sensors
• IoT devices
• Real-time monitoring systems

​

Data Properties:

​

​

• Number of records/samples collected : 150 per parameter
• Sensor types and parameters recorded : Temperature, Humidity, Air Quality, Gas Status,

Body Temperature, Pulse and Spo2

• Data frequency : Real-time
• Preprocessing steps : Took the average of the entire 4 different patient’s data into 1-month average data using App Script

​

Sample Data: https://docs.google.com/spreadsheets/d/1Ckov0iiIOcWbuoPuK46qMWK23MqrVFQIxosv-OpELWI/edit?gid=1805652497#gid=1805652497

Fig. 1. Monthly Average

Fig. 2. Heart Patient’s Data

BLOCK DIAGRAM

Fig. 3. Block Diagram

CIRCUIT DIAGRAM

Fig. 4. Model Diagram

Fig. 5. Connection Diagram

THEORITICAL EXPLANATION

CNN-Based Model Deployment on STM32 Microcontroller Using TensorFlow Lite

1. Model Training:
• A Convolutional Neural Network (CNN) is trained using a dataset to recognize patterns or objects, which in your case, might involve object detection. This involves building layers that can detect features like edges, textures, and shapes at different levels of abstraction.

​

• Model Conversion to TensorFlow Lite:
• After training the CNN, the model is converted into a TensorFlow Lite (TFLite) format. TFLite is an optimized version of TensorFlow designed for running on mobile and embedded devices, reducing the model size and computation time, which is crucial for microcontrollers with limited resources.

​

• Model Optimization:
• Optimization techniques like quantization (reducing precision from 32-bit floating point to lower precision like 8-bit integers) are applied to further reduce model size and improve inference speed, enabling it to run efficiently on embedded systems with limited processing power.

​

• STM32 Integration:
• TensorFlow Lite for Microcontrollers allows the model to be integrated into an STM32 microcontroller environment. The microcontroller, with limited RAM and processing power, uses this framework to perform inference on the input data, such as images captured by a camera.

​

• Inference and Deployment:
• Once the model is deployed, it runs inference on incoming data (e.g., camera input for object detection). The microcontroller executes the model to make predictions or classifications, using the TFLite interpreter to process and return results.

​

THEORITICAL EXPLANATION

Time-Series Algorithm for Predictive Health (Fitness for Work)

1. Time-Series Data Collection:
• Time-series data, such as workers' health parameters (e.g., heart rate, temperature, fatigue levels), is collected over time. These sequential data points hold critical temporal dependencies that need to be modeled to predict future health status and determine fitness for work.

​

• Feature Engineering:
• Time-series data is processed into features that can be used for prediction, such as by creating sliding windows of historical data to capture temporal patterns. Normalization and preprocessing are performed to ensure that the data is in a format suitable for machine learning models.

​

• Model Selection (e.g., LSTM):
• Long Short-Term Memory (LSTM) networks are a type of Recurrent Neural Network (RNN) that are well-suited for time-series data. LSTMs capture long-range dependencies and trends in sequential data, making them ideal for predicting future states (e.g., a worker’s fitness based on past health readings).

​

• Model Training and Evaluation:
• The LSTM model is trained on the time-series data to predict the target variable (fitness for work). After training, the model is evaluated using metrics like accuracy or AUC (Area Under the Curve) to ensure it generalizes well and makes accurate predictions for unseen data.
• Model Deployment:

​

• After training, the model is converted to TensorFlow Lite format and optimized for deployment on resource-constrained devices, similar to CNN model deployment. The model can then be used in real-time applications to monitor workers’ health and predict whether they are fit for work based on their health data.

​

RESULTS

Fig. 6. UI Output

Fig. 7. UI Output

RESULTS

Fig. 6. UI Output

Fig. 7. UI Output

RESULTS

Fig. 8. Overall Setup

RESULTS

Fig. 9. Board Output

Fig. 10. Board Output - Features

Fig. 11. Wrist Band

Fig. 12,13. Wrist Band

REFERENCES

[1] C.J.Bohr, A.Kumar and G.P.Hancke, Smart helmet for detection of air quality and harzardous event detection in mining industry in IEEE International conference on industrial technology(ICIT) 14-17 march 2016 , PP: 2026 2031

​

[2] M. Noorin and K. Suma, "IoT based wearable device using WSN technology for miners," 2018 3rd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), Bangalore, India, 2018pp. 992-996, doi: 10.1109/RTEICT42901.2018.9012592

​

[3]Jagadeesh.R and Dr.R.Nagaraj, IoT based smart helmet for unsafe event detection for mining industry in International research journal of engineering and technology volume04: issued 01 jan 2017, PP: 1481491

​

[4] S. S. Patil and V. S. Bendre, "Coal Mine Safety Monitoring and Alerting System," 2022 International Conference on Smart Technologies and Systems for Next Generation Computing (ICSTSN), Pune, India, 2022, pp. 1-5, doi: 10.1109/ICSTSN53068.2022.9908737.

​

[5] S. S. Patil and V. S. Bendre, "Smart Coal Mine Safety & Monitoring System," 2022 International Conference on Smart Technologies and Systems for Next Generation Computing (ICSTSN), Pune, India, 2022, pp. 1-5, doi: 10.1109/ICSTSN53068.2022.9776791.

​

[6] A. K. Sahoo, S. K. Sahu, and S. K. Sahu, "IoT-Based Wearable Devices for Personal Safety and Accident Prevention Systems," 2022 2nd International Conference on Power Electronics & IoT Applications in Renewable Energy and its Control (PARC), Mathura, India, 2022, pp. 1-5, doi: 10.1109/PARC52418.2022.9726634.

​

[7] JARCCE ISSN (O) 2278-1021, ISSN (P) 2319-5940 International Journal of Advanced Research in Computer and Communication Engineering ISO 3297:2007 CertifiedImpact Factor 8.102Vol. 12, Issue 4, April 2023 DOI: 10.17148/IJARCCE.2023.12404 Internet of Things Based Coal Mine Safety Monitoring System T. Thilagavathi1, Dr. L. Arockiam2 Research Scholar, Department of Computer Science, St. Joseph’s College (Autonomous), Tiruchirappalli, Tamil Nadu, India1 Associate Professor, Department of Computer Science, St. Joseph’s College (Autonomous), Tiruchirappalli, Tamil Nadu, India2