Advanced Student Monitoring System for physical classes using pattern recognition and behavioral analysis
2023-227
Meet Our Team
IT20189594
Rathnayka R. K. A. R.
IT20200206
Mallawarachchi S.M.A.
IT20191788
Wijesiriwardana H.G.N.D
IT20122850
Perera S.S.A.
Mr. Jeewaka Perera
Supervisor
Our Supervisor
Introduction
The system will incorporate facial recognition technology and computer vision algorithms to identify students, detect emotions and behaviors, and analyze the data to provide feedback to lecturers on how to improve their teaching strategies.
Research Question
Taking attendance in a large classroom can be a tedious and time-consuming task for teachers.
Traditional methods of taking attendance, can be inaccurate and easily manipulated.
Difficult to understand student behavior in large classrooms.
Teachers can't evaluate their performance due to the lack of student feedback.
Objectives
Develop a reliable face recognition algorithm that accurately identifies students in a classroom setting.
Efficiently store and manage and prostudent attendance data.
Develop a user-friendly interface for teachers to access the attendance data and analyze student behavior.
Optimize system to use in a classroom setting, where multiple students may be present in the camera's field of view.
Security measures to ensure the privacy and security of the stored student data.
Research Components
Emotion and behavior capture
Monitoring Student's Concentration levels
Student Attendance Marking with Face Recognition
Enhance low resolution footage
Overall System Diagram
Work Breakdown Chart
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Camera position
option A
option B
Completion of the project
Commercialization
Improve and optimize the system larger class rooms and other potential markets such as educational institutions, corporate offices, and government agencies
Desktop app integration
Cloud based service, with a subscription based or one time payment options
Poster
Research Paper Acceptance
IT20191788 Wijesiriwardana H.G.N.D
Specialization : Information Technology
Enhance Low Resolution Footage
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Background
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Student monitoring systems, the quality of video footage plays a major role in ensuring accurate analysis and decision-making
However, the implementation of high-quality cameras in real-world classroom settings often faces significant challenges due to budget constraint
Therefore we need a cost-effective solution that can enhance the visual quality of low-resolution footage
Research Gap
| Research Paper [1] | Research Paper [2] | Research Paper [3] | Research Paper [4] | Research Paper [5] | Proposed System |
SRGAN Method | | | | | | |
Video Upscaling Enhancement | | | | | | |
Camera Usage | | | | | | |
Implement In Physical Classroom Environment | | | | | | |
Integrate with Student Monitoring System | | | | | | |
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Research Problem
How to enhance low resolution footage with image upscaling
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
With the normal footage student faces may be not clear. If we enhance the video it helps to accurate analysis of our system.
Novelty
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Cost-Effective Enhancement: In educational settings, budget constraints often limit the high-quality cameras. Our component offers a cost-effective solution by improving the visual quality of existing low-resolution footage.
Contribution to Educational Technology: By enhancing low-resolution video, our research contributes to the field of educational technology, demonstrating the feasibility and effectiveness of improving student monitoring systems with advanced computer vision techniques.
Specific and sub objectives
Develop an innovative machine learning-based solution that can enhance low-resolution footage, significantly improving the scaling
Implement and integrate the SRGAN model into the student monitoring system to enhance the visual quality of low-resolution footage
Due to budget constraint of using high resolution cameras, we need a cost-effective solution that enhances the visual quality of low-resolution footage
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Enhance the video for better analysis.
System Diagram
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Model Architecture
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Integrated development environment (IDE)
Back end
Database
Technologies used
(Selection of the technologies can be changed when implementing based on the best approach)
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
Personal specification Requirement
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
What I have done in PP1
Requirement Gathering
•Finding Technologies.
•Configure development environment.
•Dataset : Image Super Resolution (from Unsplash) - https://www.kaggle.com/datasets/quadeer15sh/image-super-resolution-from-unsplash
Design Concepts
•Design Drafts.
•Design Structure.
•Follow Tutorials.
Start implementation
•Start Model Implementation
•Implent on Image Upscaling
Test the model
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Train the model and implement with cuda technology
Enhance the video
Test the component
Integrated with main system
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
What I have done in PP2
Implement with front end
Test the system with student segmentation
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
What I have done for final
Test the system with full system
Test component in classroom setting
Project Evidence
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Process of image upscaling output
Project Evidence
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Completion of the project
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Project Evidence
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Input
Output
Project Evidence
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
Input
Output
Project Evidence
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
References
[1] C. Ledig et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” arXiv.org, Sep. 15, 2016. https://arxiv.org/abs/1609.04802v5
[2] N. Tovar, S. (Seok-C. Kwon, and J. Jeong, “Image Upscaling with Deep Machine Learning for Energy-Efficient Data Communications,” Electronics, vol. 12, no. 3, p. 689, Jan. 2023, doi: 10.3390/electronics12030689.
[3] L. Galteri, L. Seidenari, T. Uricchio, M. Bertini, and A. Del Bimbo, “Preserving Low-Quality Video through Deep Learning,” IOP Conference Series: Materials Science and Engineering, vol. 949, no. 1, p. 012068, Nov. 2020, doi: 10.1088/1757-899x/949/1/012068.
[4] M. H. Maqsood, R. Mumtaz, I. U. Haq, U. Shafi, S. M. H. Zaidi, and M. Hafeez, “Super Resolution Generative Adversarial Network (SRGANs) for Wheat Stripe Rust Classification,” Sensors, vol. 21, no. 23, p. 7903, Nov. 2021, doi: 10.3390/s21237903.
[5] T. Sharmila and L. M. Leo, “Image upscaling based convolutional neural network for better reconstruction quality,” 2016 International Conference on Communication and Signal Processing (ICCSP), Apr. 2016, Published, doi: 10.1109/iccsp.2016.7754236.
IT20191788 | Wijesiriwardana H.G.N.D | 2023-227
IT20189594 Rathnayka R. K. A. R.
Specialization : Information Technology
Student Attendance Marking
with Face Recognition
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Background
Traditional attendance tracking methods are time-consuming, unreliable, and prone to errors, leading to inaccurate attendance records.
The proposed system uses real time facial recognition technology and deep learning algorithms to accurately track and record attendance in real-time.
The system is expected to improve attendance tracking accuracy, reduce administrative workload, and provide valuable insights into student engagement and attendance patterns.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Research Problem
Traditional attendance taking methods are tedious, time-consuming and prone to human error.
Additionally, can be easily fooled by students who sign in for their absent peers, use fake identification cards.
Therefore, there is a need for a more accurate and efficient attendance tracking system that can overcome these limitations.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Novelty
Detecting and Identifyingmultiple students faces and mark attendance.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Tracking eye gaze of each student to measure focus status
Specific and sub objectives
Develop a reliable face recognition algorithm to identify students in a classroom.
Identify students with their names and track individual students time.
Integrate the algorithm with a database system to efficiently store and manage attendance data.
Preprocess data before sending to ui.
Optimize the algorithm for classroom environment.
Evaluate the system's accuracy and effectiveness through real-world testing and experiments.
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
System Diagram
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Facial landmarks used to compare and recognize faces
What we have done previously in PP1
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Collect data set for facial recognition
Can detect multiple faces at a time
Implement the backend
Can detect and recognize multiple faces and mark attendence
Integrate component into the system
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Create a custom dataset to test the application.
Test component in real world settings eg:- classroom
What we have done previously in PP2
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Integrate with front end result UI
What we have done for final
Implementation of eye gaze tracker to measure focus state
Test component in classroom setting
IT20200206 Mallawarachchi S.M.A.
Specialization : Information Technology
Emotion and Behavior Detection
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Background
Lectures can not focus always to emotion condition of every students.
Emotion detection technology is increasingly being used in education to support teaching and learning
By integrating an emotion recognition system into a student monitoring system, teachers can learn a lot about their students' emotional states.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Research Problem
Classroom lecturing environment is different from online lecturing environment, because teacher cannot pay attention to each student’s emotional condition and always give feedback while considering the course schedule.
How can we overcome this problem?
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Novelty
Capturing emotions and head movements with multiple students.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Specific and sub objectives
Capture the emotions status of a particular student.
Develop a reliable face recognition algorithm to identify students in a classroom.
Optimize the algorithm for classroom environment.
Evaluate the system's accuracy and effectiveness through real-world testing and experiments.
Capture the head position of a student.
Preprocess the output data before send to the main component.
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
System Diagram
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
What we have done in PP1
Collected data set for students emotions
Data set: https://www.kaggle.com/datasets/msambare/fer2013
Can detect multiple emotions.
Implemented the backend
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Preprocess output data on emotion detection - Lable the data
Implement a model to capture head possession (Student behavior)
Integrated with the main system.
What we have done in PP2
What we have done for final
Develop user friendly interfaces for desktop application (Front end)
Integrate front end with back end
IT20189594 | Rathnayka R. K. A. R. | TMP-23-227
Fine tune system, reduce limitations
Test in real world classroom environment
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
Emotion Feature Extraction Output
Output 138 features
References
[1] Hindawi, W. Wang, K. Xu, H. Niu, and X. Miao, “Emotion Recognition of Students Based on Facial Expressions inOnline Education Based on the Perspective of Computer Simulation,” Emotion Recognition of Students Based on Facial Expressions in Online Education Based on the Perspective of Computer Simulation, Sep. 11, 2020. https://www.hindawi.com/journals/complexity/2020/4065207/
[2] S. Fakhar et al., “Smart Classroom Monitoring Using Novel Real-Time Facial Expression Recognition System,” MDPI, Nov. 27, 2022. https://www.mdpi.com/2076-3417/12/23/12134
[3] Hindawi and Q. Yuan, “Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification,” Research on Classroom Emotion Recognition Algorithm Based on Visual Emotion Classification, Aug. 08, 2022. https://www.hindawi.com/journals/cin/2022/6453499/
[4] “Emotion Recognition in E-learning Systems,” Emotion Recognition in E-learning Systems | IEEE Conference Publication | IEEE Xplore. https://ieeexplore.ieee.org/document/8525872
IT20200206 | Mallawarachchi S.M.A. | TMP-23-227
IT20122850
Perera S.S.A
Specialization : Information Technology
Monitor Student's Concentration levels
IT20122850 | Perera S.S.A | TMP-23-227
Background
IT20122850 | Perera S.S.A | TMP-23-227
The suggested system has several benefits over conventional student monitoring methods, including the potential to give many students help and real-time feedback.
The suggested system's design, implementation, and evaluation are discussed in the study, which also highlights how well it analyzes student behavior and emotional states and produces customized feedback.
The system aims to provide reliable analysis of student behavior and emotional states, generate generalized feedback, and enhance student monitoring conditions in physical education classrooms.
Research Problem
How to Collect emotions and behaviors data in physical classroom?
How to perform customized separate model for each students emotions and behaviors?
How to measure a concentration level?
IT20122850 | Perera S.S.A | TMP-23-227
Novelty
Developing a model using emotions and behaviors to measure students concentration levels and participations.
IT20122850 | Perera S.S.A | TMP-23-227
Specific and sub objectives
The main objective of this research project is to create a system that uses machine learning to track students participation and concentration level in actual classroom settings.
To implement the system in a physical classroom environment.
Analyze the distractive activities of student.
IT20122850 | Perera S.S.A | TMP-23-227
June 2023 – Field Visit for Data Gathering
Day 01 - Field visit
Day 02- Discuss the teachers
Day 03- Filed visit
Day 04- Collecting the data
Day 05- Collecting the data
Day 06- Collecting the data
Day 07- Collecting the data
Data & Evaluation
IT20122850 | Perera S.S.A | TMP-23-227
Data & Evaluation
IT20122850 | Perera S.S.A | TMP-23-227
Consent form
IT20122850 | Perera S.S.A | TMP-23-227
Linlk: https://docs.google.com/forms/d/e/1FAIpQLSda8o8nxOLl_mW31XYXXaHwyfk07VxQHwKTuVBQiLaOnglfAw/viewform
Collected Data set
IT20122850 | Perera S.S.A | TMP-23-227
Data collection
IT20122850 | Perera S.S.A | TMP-23-227
Evidence for completion of Analytics
IT20122850 | Perera S.S.A | TMP-23-227
How to verify datasets
How to verify datasets
How to verify with research papers
IT20122850 | Perera S.S.A | TMP-23-227
Trabelsi, Z.; Alnajjar, F.; Parambil, M.M.A.; Gochoo, M.; Ali, L. Real-Time Attention Monitoring System for Classroom: A Deep Learning Approach for Student’s Behavior Recognition. Big Data Cogn. Comput. 2023, 7, 48. https://doi.org/10.3390/bdcc7010048
Afroze, Sadia & Hoque, Moshiul. (2020). Classification of Attentional Focus Based on Head Pose in Multi-object Scenario. 10.1007/978-3-030-33585-4_35.
Krithika L.B, ; Lakshmi Priya GG, (2016). Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric. Procedia Computer Science, 85(), 767–776. doi:10.1016/j.procs.2016.05.264
Results
IT20122850 | Perera S.S.A | TMP-23-227
Results
IT20122850 | Perera S.S.A | TMP-23-227
Development Tools
Back end
Database
Technologies used
IT20122850 | Perera S.S.A | TMP-23-227
System, personal, and software specification requirements
Software specification Requirement
Functional Requirement
Non-Functional Requirement
• The system should be able to collect data from a variety of sources, such as cameras and wearable technology.
• The data must be clean and appropriate for machine learning analysis before the system should preprocess it.
• The system should be able to evaluate the data using machine learning techniques and deliver immediate feedback on the level of student interest and focus
•The system has to be simple to use and easy to use.
• The system must be secure and protect student privacy.
• There should be less downtime and the system should be dependable.
Personal specification Requirement
• Stakeholders – Students, Lectures (Teachers)
• Dataset – Captured student’s videos/photos
• Institutes – School, University
IT20122850 | Perera S.S.A | TMP-23-227
Model Training
IT20122850 | Perera S.S.A | TMP-23-227
Model Training
IT20122850 | Perera S.S.A | TMP-23-227
What we have done previously in PP1
Collect data set for students emotions and head movements
Research on deep learning models to train the model with an algorithm.
Research on automated measurement of affective parameters.
IT20122850 | Perera S.S.A | TMP-23-227
Implement the backend
Train model using local emotions, head movements data set and binary values.
Train model with csv and use binary data as emotions weight.
Test trained model using inputs from emotion and face detection components
IT20122850 | Perera S.S.A | TMP-23-227
Implement the backend
Integrate component into the system
What we have done previously in PP2
Create a full dataset values in a csv file.
What we have done for final
Display final result output graphically in a UI
Integrate front end with back end
Test in real world classroom environment
Generate final report output into a csv file
IT20122850 | Perera S.S.A | TMP-23-227
User Inteface
IT20122850 | Perera S.S.A | TMP-23-227
References
ICuixiang Guo, Junwu Suo, Chunguang Xu, Xinhua Yang, Liping Zhang, "Data Analysis of Physical Fitness Monitoring Based on Mathematical Models", Mathematical Problems in Engineering, vol. 2021, Article ID 8353391, 9 pages, 2021. https://doi.org/10.1155/2021/8353391 [3]
Xiaoying Shen, Chao Yuan, "A College Student Behavior Analysis and Management Method Based on Machine Learning Technology", Wireless Communications and Mobile Computing, vol. 2021, Article ID 3126347, 10 pages, 2021. https://doi.org/10.1155/2021/3126347 [1]
Ibáñez, V., Pérez, S., Silva, J., & Tamarit, S. (2019). Statistical analysis of students’ behavioral and attendance habits in engineering education. Educational Sciences: Theory and Practice, 19(4), 48 - 64. http://dx.doi.org/10.12738/estp.2019.4.004 [2]
IMaier, M.-I; Czibula, G.; One¸t-Marian, Z.-E. Towards Using Unsupervised Learning for Comparing Traditional and Synchronous Online Learning in Assessing Students’ Academic Performance. Mathematics 2021, 9, 2870. https://doi.org/10.3390/ math9222870 [4]
IYuan Zhang, Aiqiang Wang, Wenxin Hu, "Deep Learning-Based Consumer Behavior Analysis and Application Research", Wireless Communications and Mobile Computing, vol. 2022, Article ID 4268982, 7 pages, 2022. https://doi.org/10.1155/2022/4268982 [5]
Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Díez-Pastor, J.F.; Rodríguez-Arribas, S.; Marticorena-Sánchez, R.; Ji, Y.P. Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques. Appl. Sci. 2021, 11, 2677. https://doi.org/10.3390/app11062677 [6]
IT20122850 | Perera S.S.A | TMP-23-227
Demonstration
Any Questions
Thank You !