Advanced Student Monitoring System for physical classes using pattern recognition and behavioral analysis
2023-227
Meet Our Team
IT20189594
Rathnayka R. K. A. R.
IT20200206
Mallawarachchi S.M.A.
IT20191788
Wijesiriwardana H.G.N.D
IT20122850
Perera S.S.A.
Introduction
The system will incorporate facial recognition technology and computer vision algorithms to identify students, detect emotions and behaviors, and analyze the data to provide feedback to lecturers on how to improve their teaching strategies.
Research Question
Taking attendance in a large classroom can be a tedious and time-consuming task for teachers.
Traditional methods of taking attendance, can be inaccurate and easily manipulated.
Difficult to understand student behavior in large classrooms.
Teachers can't evaluate their performance due to the lack of student feedback.
Objectives
Develop a reliable face recognition algorithm that accurately identifies students in a classroom setting.
Efficiently store and manage and prostudent attendance data.
Develop a user-friendly interface for teachers to access the attendance data and analyze student behavior.
Optimize system to use in a classroom setting, where multiple students may be present in the camera's field of view.
Security measures to ensure the privacy and security of the stored student data.
Overall System Diagram
Commercialization
Improve and optimize the system larger class rooms and other potential markets such as educational institutions, corporate offices, and government agencies
Mobile app integration
Cloud based service, with a subscription based or one time payment options
Estimated Budget
Description | Total Amount (Rs.) |
Charges for software tools | 4000 |
Cost for hardware products | 8000 |
Internet and Wi-Fi charges | 6000 |
Other charges | 2000 |
Total estimated cost only for behavior detection system | 20000 |
Gantt Chart
IT20189594 Rathnayka R. K. A. R.
Specialization : Information Technology
Student Attendance Marking
with Face Recognition
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Background
Traditional attendance tracking methods are time-consuming, unreliable, and prone to errors, leading to inaccurate attendance records.
The proposed system uses real time facial recognition technology and deep learning algorithms to accurately track and record attendance in real-time.
The system is expected to improve attendance tracking accuracy, reduce administrative workload, and provide valuable insights into student engagement and attendance patterns.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Research Problem
Traditional attendance taking methods are tedious, time-consuming and prone to human error.
Additionally, can be easily fooled by students who sign in for their absent peers, use fake identification cards.
Therefore, there is a need for a more accurate and efficient attendance tracking system that can overcome these limitations.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Specific and sub objectives
Develop a reliable face recognition algorithm to identify students in a classroom.
Identify students with their names and track individual students time.
Integrate the algorithm with a database system to efficiently store and manage attendance data.
Preprocess data before sending to main component.
Optimize the algorithm for classroom environment.
Evaluate the system's accuracy and effectiveness through real-world testing and experiments.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
System Diagram
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Facial landmarks used to compare and recognize faces
Development Tools
Back end
Database
Technologies to be used
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Functional Requirement
Non-Functional Requirement
• Detect and recognize students’ faces using a camera and maintain database of enrolled students
• Record and store students’ attendance data in a secure and accessible format.(eg : excel sheet, pdf)
• Handle missing students, unrecognized faces, or faulty cameras
• High accuracy and reliability in face detection and recognition
• High scalability and availability in handling large numbers of students and classes
• Fast response time and low latency in processing images and data
Personal specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
Current Progress 50%
Collect data set for facial recognition
Train a model check the accuracy.
Implement the backend
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Completion of the project - Expected Result
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Completion of the project - Code breakdown
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Loading student face data
Completion of the project - Code breakdown
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Detecting face from a frame
Recognizing face
What is expected for PP2
Calculate time duration and mark attendece based on that
Develop an interfce to display detailed information about student attendance
Create a custom dataset to test the application.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
User Interface
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Work Breakdown Chart
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
References
https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html
https://numpy.org/doc/stable/
http://dlib.net/
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
https://www.youtube.com/watch?v=SIZNf_Ydplg
https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.researchgate.net%2Ffigure%2Fdentification-of-facial-landmarks-using-Dlib-a-Facial-landmarks-b-The-position-and_fig2_343699139&psig=AOvVaw3TAYGss86aeFC6Tx500rcz&ust=1684861492133000&source=images&cd=vfe&ved=0CBAQjhxqFwoTCJi6_Ki0if8CFQAAAAAdAAAAABAD
https://docs.opencv.org/4.x/d6/d00/tutorial_py_root.html
IT20200206 Mallawarachchi S.M.A.
Specialization : Information Technology
Emotion and Behavior Detection
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Background
Lectures can not focus always to emotion condition of every students.
Emotion detection technology is increasingly being used in education to support teaching and learning
By integrating an emotion recognition system into a student monitoring system, teachers can learn a lot about their students' emotional states.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Problem
Classroom lecturing environment is different from online lecturing environment, because teacher cannot pay attention to each student’s emotional condition and always give feedback while considering the course schedule.
How can we overcome this problem?
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Specific and sub objectives
Capture the emotions status of a particular student.
Develop a reliable face recognition algorithm to identify students in a classroom.
Optimize the algorithm for classroom environment.
Evaluate the system's accuracy and effectiveness through real-world testing and experiments.
Capture the head position of a student.
Preprocess the output data before send to the main component.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
System Diagram
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Integrated development environment (IDE)
PyCharm or Anaconda
Image processing
Python - handle algorithms
OpenCV framework
Algorithms
Back end
Database
Firebase / MongoDB
Technologies to be used
(Selection of the algorithms will be finalized when implementing based on the best approach)
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
• Identify student's emotions using camera.
• Real-time emotion detection.
• Multi-modal emotion detection
• The system should allow teachers to customize the emotion detection settings.
• High accuracy and reliability in emotion detection and recognition
• High scalability and availability in handling large numbers of students and classes
• Fast response time and low latency in processing images and data
Personal specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Work Breakdown Chart
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Current Progress 50%
Collect data set for students emotions and facial expressions
Data set: https://www.kaggle.com/datasets/msambare/fer2013
Train a model check the accuracy.
Implement the backend
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Completion of the project
Emotion detection system running process.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Completion of the project
Recognized emotions before preprocessing
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Completion of the project
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Completion of the project
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Backends for future implementation. dlib library is using now.
Completion of the project
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Import video footage and capturing emotions using DeepFace model
What is expected for PP2
Preprocess output data on emotion detection
Implement a model to capture head possession (Student behavior)
Create a manual dataset to test the application.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
References
[1] Hindawi, W. Wang, K. Xu, H. Niu, and X. Miao, “Emotion Recognition of Students Based on Facial Expressions inOnline Education Based on the Perspective of Computer Simulation,” Emotion Recognition of Students Based on Facial Expressions in Online Education Based on the Perspective of Computer Simulation, Sep. 11, 2020. https://www.hindawi.com/journals/complexity/2020/4065207/
[2] S. Fakhar et al., “Smart Classroom Monitoring Using Novel Real-Time Facial Expression Recognition System,” MDPI, Nov. 27, 2022. https://www.mdpi.com/2076-3417/12/23/12134
[3] Hindawi and Q. Yuan, “Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification,” Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification, Aug. 08, 2022. https://www.hindawi.com/journals/cin/2022/6453499/
[4] “Emotion Recognition in E-learning Systems,” Emotion Recognition in E-learning Systems | IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/document/8525872
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
IT20191788 Wijesiriwardana H.G.N.D
Specialization : Information Technology
Enhance Low Resolution Footage
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Background
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Student monitoring systems, the quality of video footage plays a major role in ensuring accurate analysis and decision-making
However, the implementation of high-quality cameras in real-world classroom settings often faces significant challenges due to budget constraint
Therefore we need a cost-effective solution that can enhance the visual quality of low-resolution footage
Research Gap
| Research Paper [1] | Research Paper [2] | Research Paper [3] | Research Paper [4] | Research Paper [5] | Proposed System |
SRGAN Method | | | | | | |
Video Upscaling Enhancement | | | | | | |
Camera Usage | | | | | | |
Implement In Physical Classroom Environment | | | | | | |
Integrate with Student Monitoring System | | | | | | |
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Research Problem
How to enhance low resolution footage with image upscaling
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Specific and sub objectives
Develop an innovative machine learning-based solution that can enhance low-resolution footage, significantly improving the scaling
Implement and integrate the SRGAN model into the student monitoring system to enhance the visual quality of low-resolution footage
Due to budget constraint of using high resolution cameras, we need a cost-effective solution that enhances the visual quality of low-resolution footage
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
System Diagram
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Model Architecture
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Integrated development environment (IDE)
Back end
Database
Technologies to be used
(Selection of the technologies can be changed when implementing based on the best approach)
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
Personal specification Requirement
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Process of image upscaling output
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Input
Output
Current Progress 50%
Requirement Gathering
•Finding Technologies.
•Configure development environment.
•Dataset : Image Super Resolution (from Unsplash) - https://www.kaggle.com/datasets/quadeer15sh/image-super-resolution-from-unsplash
Design Concepts
•Design Drafts.
•Design Structure.
•Follow Tutorials.
Start implementation
•Start Model Implementation
•Implent on Image Upscaling
Test the model
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
What is expected for PP2
Train model using custom data set.
Implement on Video Enhancement.
If I get acceptable results for my component, integrate with main system.
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
Work Breakdown Chart
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
References
[1] C. Ledig et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” arXiv.org, Sep. 15, 2016. https://arxiv.org/abs/1609.04802v5
[2] N. Tovar, S. (Seok-C. Kwon, and J. Jeong, “Image Upscaling with Deep Machine Learning for Energy-Efficient Data Communications,” Electronics, vol. 12, no. 3, p. 689, Jan. 2023, doi: 10.3390/electronics12030689.
[3] L. Galteri, L. Seidenari, T. Uricchio, M. Bertini, and A. Del Bimbo, “Preserving Low-Quality Video through Deep Learning,” IOP Conference Series: Materials Science and Engineering, vol. 949, no. 1, p. 012068, Nov. 2020, doi: 10.1088/1757-899x/949/1/012068.
[4] M. H. Maqsood, R. Mumtaz, I. U. Haq, U. Shafi, S. M. H. Zaidi, and M. Hafeez, “Super Resolution Generative Adversarial Network (SRGANs) for Wheat Stripe Rust Classification,” Sensors, vol. 21, no. 23, p. 7903, Nov. 2021, doi: 10.3390/s21237903.
[5] T. Sharmila and L. M. Leo, “Image upscaling based convolutional neural network for better reconstruction quality,” 2016 International Conference on Communication and Signal Processing (ICCSP), Apr. 2016, Published, doi: 10.1109/iccsp.2016.7754236.
IT20191788 | Wijesiriwardana H.G.N.D | TMP-23-227
IT20122850
Perera S.S.A
Specialization : Information Technology
Monitoring Student's Concentration levels
IT20122850 | Perera S.S.A | TMP-23-227
Background
IT20122850 | Perera S.S.A | TMP-23-227
The suggested system has several benefits over conventional student monitoring methods, including the potential to give many students help and real-time feedback.
The suggested system's design, implementation, and evaluation are discussed in the study, which also highlights how well it analyzes student behavior and emotional states and produces customized feedback.
The system aims to provide reliable analysis of student behavior and emotional states, generate generalized feedback, and enhance student monitoring conditions in physical education classrooms.
Research Problem
How to Collect emotions and facial expressions data in physical classroom?
How to perform customized separate model for each students emotions and facial expression?
How to measure a concentration level?
IT20122850 | Perera S.S.A | TMP-23-227
Novelty
Developing a model using emotions and behaviors to measure students concentration levels and participations.
IT20122850 | Perera S.S.A | TMP-23-227
Specific and sub objectives
The main objective of this research project is to create a system that uses machine learning to track students participation and concentration level in actual classroom settings.
To implement the system in a physical classroom environment.
Analyze the distractive activities of student.
IT20122850 | Perera S.S.A | TMP-23-227
System Diagram
IT20122850 | Perera S.S.A | TMP-23-227
Development Tools
Back end
Database
Technologies to be used
IT20122850 | Perera S.S.A | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
• The system should be able to collect data from a variety of sources, such as cameras and wearable technology.
• The data must be clean and appropriate for machine learning analysis before the system should preprocess it.
• The system should be able to evaluate the data using machine learning techniques and deliver immediate feedback on the level of student interest and focus
•The system has to be simple to use and easy to use.
• The system must be secure and protect student privacy.
• There should be less downtime and the system should be dependable.
Personal specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
IT20122850 | Perera S.S.A | TMP-23-227
Collected Data set
IT20122850 | Perera S.S.A | TMP-23-227
Current Progress 50%
Collect data set for students emotions and facial expressions.
Research on deep learning moddels to train the model with an algorithm.
Research on automated measurement of affective parameters.
IT20122850 | Perera S.S.A | TMP-23-227
Implement the backend
Model Training
IT20122850 | Perera S.S.A | TMP-23-227
Model Training
IT20122850 | Perera S.S.A | TMP-23-227
Model Training
IT20122850 | Perera S.S.A | TMP-23-227
Model Training
IT20122850 | Perera S.S.A | TMP-23-227
Completion of the project
Implement a Algorithm
IT20122850 | Perera S.S.A | TMP-23-227
Completion of the project
Implement a model
IT20122850 | Perera S.S.A | TMP-23-227
What is expected for PP2
Train model using local emotions, facial expression data set and binary values.
Train model without images and use binary data as emotions weight.
Train model using students videos, data set to track student participations.
IT20122850 | Perera S.S.A | TMP-23-227
Implement the frontend
User Inteface
IT20122850 | Perera S.S.A | TMP-23-227
Work Breakdown Chart
IT20122850 | Perera S.S.A | TMP-23-227
References
ICuixiang Guo, Junwu Suo, Chunguang Xu, Xinhua Yang, Liping Zhang, "Data Analysis of Physical Fitness Monitoring Based on Mathematical Models", Mathematical Problems in Engineering, vol. 2021, Article ID 8353391, 9 pages, 2021. https://doi.org/10.1155/2021/8353391 [3]
Xiaoying Shen, Chao Yuan, "A College Student Behavior Analysis and Management Method Based on Machine Learning Technology", Wireless Communications and Mobile Computing, vol. 2021, Article ID 3126347, 10 pages, 2021. https://doi.org/10.1155/2021/3126347 [1]
Ibáñez, V., Pérez, S., Silva, J., & Tamarit, S. (2019). Statistical analysis of students’ behavioral and attendance habits in engineering education. Educational Sciences: Theory and Practice, 19(4), 48 - 64. http://dx.doi.org/10.12738/estp.2019.4.004 [2]
IMaier, M.-I; Czibula, G.; One¸t-Marian, Z.-E. Towards Using Unsupervised Learning for Comparing Traditional and Synchronous Online Learning in Assessing Students’ Academic Performance. Mathematics 2021, 9, 2870. https://doi.org/10.3390/ math9222870 [4]
IYuan Zhang, Aiqiang Wang, Wenxin Hu, "Deep Learning-Based Consumer Behavior Analysis and Application Research", Wireless Communications and Mobile Computing, vol. 2022, Article ID 4268982, 7 pages, 2022. https://doi.org/10.1155/2022/4268982 [5]
Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Díez-Pastor, J.F.; Rodríguez-Arribas, S.; Marticorena-Sánchez, R.; Ji, Y.P. Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques. Appl. Sci. 2021, 11, 2677. https://doi.org/10.3390/app11062677 [6]
IT20122850 | Perera S.S.A | TMP-23-227
Any Questions
Thank You !