2 of 13

Deaf Learning System 3D Visionary

Team Members and Supervisors

Team Members

MAHBOOB ALI (FC-104)

SUMMER ABBAS (FC-102)

MUAHAMMAD TAYYAB (FC-091)

Supervisor

MR. SUBHAN ARIF

Co-Supervisor

MR. SHEHROZ TARIQ

Project Title

3 of 13

Introduction

According to WHO's statistics, approximately 466 million people are deaf worldwide, which is about 6.1% of the world's population
With 432 million adults and 34 million children affected.
More than 900 million people will suffer from hearing loss by 2050.
Around 10 million people are deaf in Pakistan.
The objective of the Deaf Learning System 3D Visionary project is to transform the learning experience and engagement of deaf individuals with educational material.
Utilizing state-of-the-art 3D visualization technologies, the project strives to establish an immersive and user-friendly learning platform tailored specifically to the requirements of the deaf community.

4 of 13

Problem Definition

Sign language education is a significant challenge for deaf-mute individuals in Asian regions, especially in Pakistan due to a lack of specialized resources and inclusive learning environments. As a result, the deaf community suffers from educational disparities and marginalization.

5 of 13

A Visit to Deaf School

6 of 13

Literature Review

Sr. No	Article No / APP/ System	Year	Features	Strength	Limitations
1	Enhancing interaction and learning experience for deaf students through sign language translator [1]	2024	Use of ADDIE, translate words into sign language, Voice detection from a video and converts it into sign language, An LMS type static learning system.	The voice detection from a video and its conversion it sign language.	Cannot convert voice into 3D motion.
2	Indonesian Sign Language (SIBI) Learning Media Application Based on Deep Learning Technology for Deaf Children [2]	2023	Uses TensorFlow Lite, Images/objects as input and convert into Indonesian Sign Language in real-time.	Its average accuracy is about 100%, Real-time conversion.	If cannot recognize certain image or object, in case of failure recommends the Indonesian Sign Language dictionary.
3	Machine learning based AI generated sign language translator [3]	2023	Use LSTM model, Takes text/voice as input and provides sign’s image according to ISL.	Focuses on converting voice to sign language	Less user satisfaction, complexities in voice recognition.
4	Hand Talk [4]	2023	Real time voice/text conversion into 3D animation.	Real time detection of voice and text.	No static learning system for beginners specially, for American Sign Language.
5	ISL Journey [5]	2022	A visual dictionary with every sign used in the lessons, practice quizzes, grammar and culture tips.	A visual dictionary with every sign of ISL.	No real time conversion of text or voice into sign language, no 3D animation

7 of 13

Literature Review

Sr. No	Article No / APP/ System	Year	Features	Strength	Limitations
6	AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove [6]	2021	Uses CNN model, uses special gloves to recognize the gestures and its conversion to text and voice	Its average accuracy is about 86%.	Dependent on specialized gloves, cannot convert text/voice into 3D motion.
7	EduSign Academy [7]	2021	Specially designed for the academic purposes, Videos for beginners according to the specific course or degree.	Special courses for deaf learners.	No real time detection of text or voice into 3D Motion hand Gesture.
8	An intelligent system to help deaf students learn Arabic Sign Language [8]	2021	Use of ArSL model, Converts voice to Arabic Sign Language, Takes gesture images as input and converts into voice.	Its average accuracy is about 99%, System keeps itself up-to-date according to new dataset.	Cannot convert text to 3D motion and vice versa, Only applicable for Arabic Language.
9	PSL [9]	2020	The Pakistan Sign Language in short video tutorials in Urdu.	Videos for deaf persons beginners to learn PSL.	No real time detection, No 3D Motion Visualization.
10	Deaf Sign Language [10]	2019	Contains categories of static content like characters, counting, animal names etc.	System uses only pictures, Static data for basic learning for deaf person.	Static data, no 3D Motion, no real time detection.

8 of 13

Modules of our Deaf Learning System

Text to Gesture

This module converts input text into clear, expressive gestures.

Voice to Gesture

This module recognizes spoken words and audio as text and then converts into corresponding signs and gestures.

Gesture to Text

This module converts gestures used by deaf user into text.

Gesture to Voice

This module recognizes the gestures and signs used by deaf user as speech and later converts into speech.

Basic Content Learning System

This module provides a comprehensive learning system containing categories like English alphabets, Fruit names, Vegetables name etc.

9 of 13

Text to Gesture

Voice to Gesture

NLP Model

How are you

10 of 13

Gesture to Text

Gesture to Voice

Image/Video Processing

Model

Image/Video

Processing Model

Nice to meet you

11 of 13

Summary of Proposed System Modules

12 of 13

References

[1]	S. F. ,. M. Z. Dian Nugraha, "Enhancing interaction and learning experience for deaf students," Indonesian Journal of Electrical Engineering and Computer Science ( pp. 1730~1738), 2024.
[2]	H. A. F. R. H. R. S. S. Martianda Anggraeni, "Indonesian Sign Language (SIBI) Learning Media Application Based on Deep Learning Technology for Deaf Children," Journal of Technology and Informatics (JoTI) 5.1, 2023.
[3]	I. S. J. J. N. G. Archana PG, "MACHINE LEARNING BASED AI GENERATED SIGN LANGUAGE," International Research Journal of Modernization in Engineering, Technology and Science (e-ISSN: 2582-5208), 2023.
[4]	[Online]. Available: https://play.google.com/store/apps/details?id=br.com.handtalk&hl=en&gl=US.
[5]	[Online]. Available: https://play.google.com/store/apps/details?id=com.toleio.india.
[6]	Z. Z. T. H. &. C. L. Feng Wen, "AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove," Nature Communications volume 12, 2021.
[7]	[Online]. Available: https://play.google.com/store/apps/details?id=in.edusign.elearning&hl=en&gl=US.
[8]	H. M. M. &. D. M. Elbourhamy, "An intelligent system to help deaf students learn Arabic Sign Language," Interactive Learning Environments 31.5, 2021.
[9]	[Online]. Available: https://play.google.com/store/apps/details?id=com.grappetite.fesf.psl.
[10]	[Online]. Available: https://play.google.com/store/apps/details?id=com.bhbharesh.DeafSignLanguage.

1 of 13