Advanced Student Monitoring System for physical classes using pattern recognition and behavioral analysis
TMP-23-227
Meet Our Team
IT20189594
Rathnayka R. K. A. R.
IT20200206
Mallawarachchi S.M.A.
IT20191788
Wijesiriwardana H.G.N.D
IT20122850
Perera S.S.A.
Introduction
The system will incorporate facial recognition technology and computer vision algorithms to identify students, detect emotions and behaviors, and analyze the data to provide feedback to lecturers on how to improve their teaching strategies.
Research Question
Taking attendance in a large classroom can be a tedious and time-consuming task for teachers.
Traditional methods of taking attendance, can be inaccurate and easily manipulated.
Difficult to understand student behavior in large classrooms.
Teachers can't evaluate their performance due to the lack of student feedback.
Objectives
Develop a reliable face recognition algorithm that accurately identifies students in a classroom setting.
Efficiently store and manage student attendance data.
Develop a user-friendly interface for teachers to access the attendance data and analyze student behavior.
Optimize system to use in a large classroom setting, where multiple students may be present in the camera's field of view.
Security measures to ensure the privacy and security of the stored student data.
Overall System Diagram
Commercialization
Improve and train the system for potential markets where the face recognition system can be useful, such as educational institutions, corporate offices, and government agencies
Mobile app integration
Cloud based service, with a subscription based or one time purchase payment options
Estimated Budget
Description | Total Amount (Rs.) |
Charges for software tools | 4000 |
Cost for hardware products | 8000 |
Internet and Wi-Fi charges | 6000 |
Other charges | 2000 |
Total estimated cost only for behavior detection system | 20000 |
Gantt Chart
IT20189594 Rathnayka R. K. A. R.
Specialization : Information Technology
Student attendece marking
with face recognition
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Background
Traditional attendance tracking methods are time-consuming, unreliable, and prone to errors, leading to inaccurate attendance records.
The proposed system uses facial recognition technology and deep learning algorithms to accurately track and record attendance in real-time.
The system is expected to improve attendance tracking accuracy, reduce administrative workload, and provide valuable insights into student engagement and attendance patterns.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Research Summery
Research Paper [1] : This attendance tracking system digitizes the traditional method using facial recognition and advanced classifiers such as Haar, KNN, CNN, SVM, and generative adversarial networks. The system generates Excel reports, requires minimal installation, and has been tested under varying conditions.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Paper [2] : This attendance monitoring system achieves real-time face recognition using HOG, CNN, and SVM technologies, generating automated attendance reports. The system has achieved high accuracy rates, with 99.5% on the LFW database and 97.83% in classroom attendance monitoring. The system uses webcam and generates automated attendance reports.
Research Summery
Research Paper [3] : This research discusses the effectiveness face recognition technology as alternatives to traditional pen-paper attendance marking. Unlike fingerprint scanning, face recognition systems helps reducing COVID-19 transmission risks. Goal to provide smart and efficient attendance tracking solutions.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Paper [4] :This project proposes an automated attendance system using facial recognition technology to compare processed images against stored records, marking attendance in a database. The system has 4 phases: Image Capturing, Segmentation, Face Detection, Face Recognition, and Updating Attendance.
Research Paper [5] : This attendance system uses a fixed classroom camera to capture and recognize faces, automatically marking attendance and notifying parents of absences. It employs various face comparison methods, including Eigen faces
Research Gap
Technologies and methods | Research Paper [1] | Research Paper [2] | Research Paper [3] | Research Paper [4] | Research Paper [5] | Proposed System |
System used for physical classes | | | | | | |
System used to detect multiple users | | | | | | |
System can track students time inside the classroom by calculating time | | | | | | |
System can differentiate similar faces. | | | | | | |
System can handle challenging lightning conditions | | | | | | |
Use of web application platform | | | | | | |
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Existing Products
Fareclock
Fareclock is employee time attendance software, using face recognition
A cloud-based time and attendance software that uses face recognition technology to track employee attendance. It is easily customizable to any org needs and can integrate with third party software.
FA6 Class is a face recognition software which identifies students at the classroom and keeps track of their course attendance.
Securtime
FA6 CLASS – CLASSROOM ATTENDANCE.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Research Problem
Traditional attendance taking methods are tedious, time-consuming and prone to human error.
Additionally, can be easily fooled by students who sign in for their absent peers, use fake identification cards.
Therefore, there is a need for a more accurate and efficient attendance tracking system that can overcome these limitations.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Specific and sub objectives
Develop a reliable face recognition algorithm to identify students in a classroom.
Integrate the algorithm with a database system to efficiently manage attendance data.
Optimize the algorithm for use in large classrooms with multiple students in view.
Evaluate the system's accuracy and effectiveness through real-world testing and experiments.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
System Diagram
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Integrated development environment (IDE)
PyCharm or Anaconda
Python - handle algorithms
OpenCV framework
Tensorflow
Algorithms
Image processing
Back end
Database
Firebase / MongoDB
Technologies to be used
(Selection of the algorithms will be finalized when implementing based on the best approach)
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Functional Requirement
Non-Functional Requirement
• Detect and recognize students’ faces using a camera and a database of enrolled students
• Record and store students’ attendance data in a secure and accessible format
• Handle missing students, unrecognized faces, or faulty cameras
• High accuracy and reliability in face detection and recognition
• High scalability and availability in handling large numbers of students and classes
• Fast response time and low latency in processing images and data
Personal specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
Work Breakdown Chart
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
References
[1]“Student Attendance System using Face Recognition,” Student Attendance System using Face Recognition | IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/document/9215441
[2]B. C. Roy, I. Hossen, Md. G. Rashed, and D. Das, “Automated Student Attendance Monitoring System Using Face Recognition,” Automated Student Attendance Monitoring System Using Face Recognition | SpringerLink, Feb. 08, 2021. https://link.springer.com/chapter/10.1007/978-3-030-68154-8_54
[3]IJRASET, “Attendance Monitoring System Using Face Recognition,” Attendance Monitoring System Using Face Recognition. https://www.ijraset.com/research-paper/attendance-monitoring-system-using-face-recognition
[4]S. Fakhar et al., “Smart Classroom Monitoring Using Novel Real-Time Facial Expression Recognition System,” MDPI, Nov. 27, 2022. https://www.mdpi.com/2076-3417/12/23/12134
[5]A. H. B, A. C, C. K. N, and R. R, “Attendance Monitoring System Using Face Recognition – IJERT,” Attendance Monitoring System Using Face Recognition – IJERT, Apr. 24, 2018. https://www.ijert.org/attendance-monitoring-system-using-face-recognition
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
IT20200206 Mallawarachchi S.M.A.
Specialization : Information Technology
Emotion Detection
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Background
Lectures can not focus always to emotion condition of every students.
Emotion detection technology is increasingly being used in education to support teaching and learning
By integrating an emotion recognition system into a student monitoring system, teachers can learn a lot about their students' emotional states.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Summery
Research Paper [1] : Online education has become increasingly popular due to its convenience, but its efficiency has been questioned due to lack of communication and feedback between teachers and students. This work proposes a framework combining a face expression recognition (FER) algorithm with online courses platforms to analyze and classify 8 kinds of emotions.
Research Paper [2] : A real-time automatic emotion recognition system is developed incorporating novel salient facial features for classroom assessment using a deep learning model. The selected emotional states are happiness, sadness, and fear, and tested against gender, department, lecture time, seating positions, and difficulty of a subject.
Research Paper [3] : Facial expression recognition using multi-task convolutional neural network (MTCNN) can be used to support emotional learning in e-learning systems. The MTCNN model has an accuracy of 93.53% for recognizing students' facial emotions under obscuration.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Summery
Research Paper [4] : Facial expression recognition is an important area in intelligent e-learning systems, but it requires an awareness of students' emotions. This paper introduces a new system composed of three main steps: preprocessing, features extraction and classification, which tested with 8-12 year olds and reached state-of-the-art results.
Research Paper [5] : The e-learner's intelligent emotion detection system can detect the negative mood of students in an u-learning environment, providing a continuous feedback mechanism to instructor for enhancing content and keeping topics updated.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Gap
Technologies and methods | Research Paper [1] | Research Paper [2] | Research Paper [3] | Research Paper [4] | Research Paper [5] | Proposed System |
Capture multiple emotions | | | | | | |
Use for physical classes | | | | | | |
CNN / ANN / RCNN Method | | | | | | |
Usage Web Application | | | | | | |
CCTV Camera Accessibility | | | | | | |
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Existing Products
Emotimeter
Emotimeter can detect emotions from facial expressions using cutting edge machine learning technologies
Emotion detection systems are used in modern cars to detect drivers emotions when they drive. It captures only one specific emotion like whether the driver is sleepy or not
This project aims to classify a group’s perceived emotion as Positive, Neutral or Negative
Driver's drowsiness Detection
The Emotion Detector
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Problem
Classroom lecturing environment is different from online lecturing environment, because the number of students is large, and the teacher cannot pay attention to each student’s emotional condition and always give feedback while considering the course schedule
How can we overcome this problem?
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Specific and sub objectives
enhance the performance of students at the academic level.
Improve understanding of how student emotion can be monitored and managed to promote positive academic outcomes.
Get meaningful feedback about their students.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
System Diagram
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Integrated development environment (IDE)
PyCharm or Anaconda
Image processing
Python - handle algorithms
OpenCV framework
Algorithms
Back end
Database
Firebase / MongoDB
Technologies to be used
(Selection of the algorithms will be finalized when implementing based on the best approach)
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
• Identify student's emotions using camera.
• Real-time emotion detection.
• Multi-modal emotion detection
• The system should allow teachers to customize the emotion detection settings.
• High accuracy and reliability in emotion detection and recognition
• High scalability and availability in handling large numbers of students and classes
• Fast response time and low latency in processing images and data
Personal specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Work Breakdown Chart
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
References
[1] Hindawi, W. Wang, K. Xu, H. Niu, and X. Miao, “Emotion Recognition of Students Based on Facial Expressions inOnline Education Based on the Perspective of Computer Simulation,” Emotion Recognition of Students Based on Facial Expressions in Online Education Based on the Perspective of Computer Simulation, Sep. 11, 2020. https://www.hindawi.com/journals/complexity/2020/4065207/
[2] S. Fakhar et al., “Smart Classroom Monitoring Using Novel Real-Time Facial Expression Recognition System,” MDPI, Nov. 27, 2022. https://www.mdpi.com/2076-3417/12/23/12134
[3] Hindawi and Q. Yuan, “Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification,” Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification, Aug. 08, 2022. https://www.hindawi.com/journals/cin/2022/6453499/
[4] “Emotion Recognition in E-learning Systems,” Emotion Recognition in E-learning Systems | IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/document/8525872
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
IT20191788 Wijesiriwardana H.G.N.D
Specialization : Information Technology
Student Behavior Detection
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Background
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Teachers face challenges in detecting student behavior, such as time-consuming classroom observation.
Educators can identify and support students who are struggling, leading to improved outcomes and a more effective learning environment.
A more efficient solution is needed to analyze student behavior accurately and provide actionable insights to educators.
Research Summery
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Research Paper [3] : This paper proposes a novel method for hand-raising detection in real classroom environments, based on region-based, fully convolutional networks (R-FCN). It uses an adaptive templates selection algorithm and a feature pyramid to capture detail and semantic features, achieving 90% accuracy.
Research Paper [2] : An intelligent model is proposed to analyze the behavior of online learners and motivate them towards the learning process, providing an adaptive e-learning system. Classification algorithms used for this research.
Research Paper [1] : This research aims to create a self-sufficient agent that can offer information to both teachers and pupils in the online classroom to understand how the student has been listening to the whole class and to analyze and present a report based on the student's behavior. It used Open CV framework.
Research Summery
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Research Paper [4] : This paper develops an improved machine learning method to identify single and compound features of human behavior, with different algorithms such as CNN, RNN, LSTM using wearable sensors
Research Paper [5] : This proposed system uses sensing technology to collect information about learning behavior, analyze concentration levels, and apply an AI algorithm to optimize performance.The system utilizes an artificial bee colony (ABC) algorithm to optimize the system performance to help teachers immediately understand the and learning status of their students.
Research Gap
| Research Paper [1] | Research Paper [2] | Research Paper [3] | Research Paper [4] | Research Paper [5] | Proposed System |
CNN/ANN/RCNN Method | | | | | | |
Use of Web Application | | | | | | |
Camera Usage | | | | | | |
Monitor in Physical Classrooms | | | | | | |
Detect Multiple Behavior Actions | | | | | | |
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Existing Products
Hero
Kidz Behaviour Tracker
Behavior Tracker
Behavior Diary
These products required to input data manually.
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Research Problem
Identifying behavior for a particular student requires a teacher who has worked in a real class environment. This process is not difficult in a class with few students, but is difficult for a class with a large number of students because of it consumes too much human effort.
How can we overcome this problem?
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Specific and sub objectives
Enhance academic performance by providing real-time insights into critical metrics that traditional monitoring methods cannot.
Improve understanding of student behavior to promote positive academic outcomes and ensure the well-being of students in physical classroom environments.
Provide lecturers with actionable insights and meaningful feedback about their students to improve the quality of lectures and enhance the learning experience.
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
System Diagram
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Integrated development environment (IDE)
PyCharm or Anaconda
Image processing
Python - handle algorithms
OpenCV framework
Algorithms
Back end
Database
Firebase / MongoDB
Technologies to be used
(Selection of the algorithms will be finalized when implementing based on the best approach)
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
Personal specification Requirement
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Work Breakdown Chart
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
References
[1] A. L. R. Reddy, K. A. Gandhi, S. T. Jaffer, and M. K. Srilekha, “Student Live Behaviour Monitoring During Virtual Class using Artificial Intelligence - IOPscience,” Student Live Behaviour Monitoring During Virtual Class using Artificial Intelligence - IOPscience, Sep. 01, 2022. https://iopscience.iop.org/article/10.1088/1742-6596/2335/1/012026
[2] K. Abhirami, M. K., and @TechScience_TSP, “Student Behavior Modeling for an E-Learning System Offering Personalized Learning Experiences,” Tech Science Press, Jan. 01, 2021. https://www.techscience.com/csse/v40n3/44583
[3] “Hand-raising gesture detection in real classrooms using improved R-FCN,” Hand-raising gesture detection in real classrooms using improved R-FCN - ScienceDirect, May 14, 2019. https://www.sciencedirect.com/science/article/abs/pii/S0925231219306939?via%3Dihub
[4] J. Zhu, S. B. Goyal, C. Verma, M. S. Raboaca, and T. C. Mihaltan, “Machine Learning Human Behavior Detection Mechanism Based on Python Architecture,” MDPI, Sep. 02, 2022. https://www.mdpi.com/2227-7390/10/17/3159
[5] “Developing a sensor-based learning concentration detection system | Emerald Insight,” Developing a sensor-based learning concentration detection system | Emerald Insight, Feb. 25, 2014. https://doi.org/10.1108/EC-01-2013-0010
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
IT20122850
Perera S.S.A
Specialization : Information Technology
Student Analyzing and Feedback
IT20122850 | Perera S.S.A | TMP-23-227
Background
IT20122850 | Perera S.S.A | TMP-23-227
The suggested system has several benefits over conventional student monitoring methods, including the potential to give many students help and real-time feedback.
The suggested system's design, implementation, and evaluation are discussed in the study, which also highlights how well it analyzes student behavior and emotional states and produces customized feedback.
The system aims to provide reliable analysis of student behavior and emotional states, generate generalized feedback, and enhance student monitoring conditions in physical education classrooms.
Research Summary
IT20122850 | Perera S.S.A | TMP-23-227
The research paper collect student's consumption, life and learning data through the campus all in one card system. [1]
Analyzing physical health of college students using statistical and LMC models [3]
This study analyzed data collected over two academic years about students' marks and daily seating position in engineering school classrooms with 5-66 students. [2]
The system was tested in five online courses in multiple majors, and the results emphasize the importance of monitoring online lectures for satisfactory students' participation. [4]
This paper proposes rDNN and KmDNN models for consumer behavior prediction using deep learning. [5]
The paper focuses on three aspects of deep neural network models: theory, construction and implementation, and improvement. [6]
Research Gap
Technologies and methods | Research Paper [1] | Research Paper [2] | Research Paper [3] | Research Paper [4] | Research Paper [5] | Proposed System |
Student emotions and behaviors data collection | | | | | | |
Use for physical classes | | | | | | |
Student Learning Attention Pattern Recognition & Analyzer | | | | | | |
Usage Web Application / Mobile Application | | | | | | |
Psychology domain thory | | | | | | |
IT20122850 | Perera.S.S.A. | TMP-23-227
Research Gap
IT20122850 | Perera S.S.A | TMP-23-227
In a large physical classroom tracking the attention and focus levels of every students can be a challenge and they have no proper way to track the students attention level.
Will notify the student periodically regarding their attentiveness to the lecture.
Will capture the students distractive combination behaviors and emotion such as Mobil usage there by helping them stay focused on the lecture.
With the feedback obtains from the study lecturer can optimize his strategies to keep their attention.
Existing Products
DyKnow
ClassDojo
ClassDojo is a simple classroom management using positive feedback and parent communication
Dyknow is the best monitoring solution for schools.
Research Question
This is particularly challenging in large classroom settings where it is difficult for lecturers to pay attention to individual students' emotions and behaviors.
IT20122850 | Perera S.S.A | TMP-23-227
How can teacher monitor students engagement in large classrooms and adapt their teaching approach to meet individual needs?
Specific and sub objectives
Optimized algorithm to provide feedback to students periodically.
Analyze the distractive activities of students
Use of deep learning models to train students'emotion n behaviour activities of students and capture subject, teaching method and presentation mode.
IT20122850 | Perera S.S.A | TMP-23-227
Analyze and suggest suitable in class for students.
System Diagram
IT20122850 | Perera S.S.A | TMP-23-227
Integrated development environment (IDE)
PyCharm or Anaconda
Python - handle algorithms
OpenCV framework
Algorithms
Back end
Database
Firebase / MongoDB
Technologies to be used
(Selection of the algorithms will be finalized when implementing based on the best approach)
IT20122850 | Perera S.S.A | TMP-23-227
System, personal, and software specification requirements
Personal specification Requirement
• Processor -Intel Corei7 9th Generation
• GPU -NVIDIA GEFORCE GTX 1660 TI 6GB
• RAM – 16GB
• Storage – 256Gb NVME SSD
Software specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
Work Breakdown Chart
IT20122850 | Perera S.S.A | TMP-23-227
References
ICuixiang Guo, Junwu Suo, Chunguang Xu, Xinhua Yang, Liping Zhang, "Data Analysis of Physical Fitness Monitoring Based on Mathematical Models", Mathematical Problems in Engineering, vol. 2021, Article ID 8353391, 9 pages, 2021. https://doi.org/10.1155/2021/8353391 [3]
Xiaoying Shen, Chao Yuan, "A College Student Behavior Analysis and Management Method Based on Machine Learning Technology", Wireless Communications and Mobile Computing, vol. 2021, Article ID 3126347, 10 pages, 2021. https://doi.org/10.1155/2021/3126347 [1]
Ibáñez, V., Pérez, S., Silva, J., & Tamarit, S. (2019). Statistical analysis of students’ behavioral and attendance habits in engineering education. Educational Sciences: Theory and Practice, 19(4), 48 - 64. http://dx.doi.org/10.12738/estp.2019.4.004 [2]
IMaier, M.-I; Czibula, G.; One¸t-Marian, Z.-E. Towards Using Unsupervised Learning for Comparing Traditional and Synchronous Online Learning in Assessing Students’ Academic Performance. Mathematics 2021, 9, 2870. https://doi.org/10.3390/ math9222870 [4]
IYuan Zhang, Aiqiang Wang, Wenxin Hu, "Deep Learning-Based Consumer Behavior Analysis and Application Research", Wireless Communications and Mobile Computing, vol. 2022, Article ID 4268982, 7 pages, 2022. https://doi.org/10.1155/2022/4268982 [5]
Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Díez-Pastor, J.F.; Rodríguez-Arribas, S.; Marticorena-Sánchez, R.; Ji, Y.P. Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques. Appl. Sci. 2021, 11, 2677. https://doi.org/10.3390/app11062677 [6]
IT20122850 | Perera S.S.A | TMP-23-227
Any Questions
Thank You !