1 of 51

2023-361

WISE EYE: Elderly Monitoring System Using�Machine Learning

1

11/5/2023

2 of 51

Student Name

Student Reg.No

Sampath H.W.A.

IT20124830

Kavinda K. G. A.

IT20211714

Gunaratne B.S.

IT20208462

Wijethunga W. M. T. S

IT20216146

Supervisor : Ms. Vindhya Kalapuge

Co Supervisor : Ms. Geethanjali Wimalasena

Team Members

2

2

11/5/2023

3 of 51

Overview

  • What is the elderly monitoring system?
  • What are the smart elderly monitoring methods?
  • Unable to monitor elders, when caregivers away from home
  • Unidentified accidents causes for serious health issues
  • No proper way to analyze elders' activity levels in county like Sri Lanka

​

3

3

11/5/2023

4 of 51

Research Problem

  • Lack of technologies to monitoring elderly individuals
  • What are the elderly monitoring methods?
  • There is no real-time monitoring option
  • Unable to assess the routine of the elder accurately.

​

4

4

11/5/2023

5 of 51

What is Wise Eye?

The Wise Eye Elderly Monitoring System is an intelligent smart phone application with elderly monitoring devices to monitor the home alone elderly individuals, when the caregivers away from the home.

5

5

11/5/2023

6 of 51

Speciality of Wise Eye

First Time in Sri Lanka

Automatic device activation based on the elderly individual

Send emergency alerts to the caregiver’s mobile app

Low cost, simple and smart

6

6

11/5/2023

7 of 51

Research Objective

The main objective of this research is to,

​

  • The primary goal is to develop a system that can effectively monitor the safety of elderly individuals.
  • The research aims to create a seamless interface between the monitoring system and caregivers' mobile applications. This facilitates quick response and support, allowing caregivers to stay informed about the well-being and needs of the elderly individuals under their care.

7

7

11/5/2023

8 of 51

Automatic accident detection of elderly person with location

Automatically activation and

deactivation of the system by

identifying the situation

Voice and Sound recognition

Activity level prediction using

behavioral pattern analysis

Sub Objectives

8

8

11/5/2023

9 of 51

IT20124830 | Sampath H.W.A

Software Engineering

IT20124830 | Sampath H.W.A | 2023-361

ITXXXXXXXX | <<Student Name>> | <<Project ID>>

9

11/5/2023

10 of 51

Background & Problem

Sudden fall is the most common issue among senior people. In a country like Sri Lanka, there is no proper way to identify the accident of a elder when caregivers are away from the home. Unidentified accidents will impact to elder in dangerous ways. Why doesn’t Sri Lanka have proper elderly monitoring system?

​

  • Lack of knowledge and technologies
  • Elders don’t like to wear devices in their body
  • There is no real-time monitoring option
  • Unable to assess the routine of the elder accurately.

​

IT20124830 | Sampath H.W.A | 2023-361

10

10

11/5/2023

11 of 51

Main Objective

  • Implementation of a model which will be able to automatically accident detection of elderly person and send emergency alert to caregivers' mobile application.

Sub Objective

  • Collect and curate a dataset of elder falls, process and develop an AI model.
  • Get accurate accident detection by with the help of audio input.
  • Detect the elder accident location with the help of sensors.
  • Creating a mobile application to send emergency elder accident alerts to caregivers.

​

​

​

​

Objectives

IT20124830 | Sampath H.W.A | 2023-361

11

11

11/5/2023

12 of 51

Current Progress

IT20124830 | Sampath H.W.A | 2023-361

  1. Dataset

​

​

​

  • Collected adult's dataset from Kaggle, Kfall and some Sri Lankan elderly persons
  • Collected total 500 images
  • Classes : Fall, Not Fall

12

12

11/5/2023

13 of 51

Current Progress

IT20124830 | Sampath H.W.A | 2023-361

  1. Data Preprocessing

​

​

​

  • Manually separated data into 2 action classes.
  • Each frame down sampled and cropped to single fragment of constant length.

13

13

11/5/2023

14 of 51

Current Progress

IT20124830 | Sampath H.W.A | 2023-361

  • Loss and Accuracy Plots

​

​

​

Achieved 83.25% Accuracy

14

14

11/5/2023

15 of 51

Current Progress

IT20124830 | Sampath H.W.A | 2023-361

  • Final Outputs

​

​

15

15

11/5/2023

16 of 51

IT20211714 | KAVINDA K.G.A

​

Software Engineering

IT20211714|   Kavinda K.G.A  |   2023-361

ITXXXXXXXX | <<Student Name>> | <<Project ID>>

16

11/5/2023

17 of 51

Background

​

  • Elderly monitoring systems are designed to help caregivers and family members to monitor the well-being of elderly people who may be living alone.

​

​

  • Elderly monitors market which was USD 2.92 billion in 2022, is expected to reach USD 6.04 billion by 2030,.

​

​

​

​

IT20211714|   Kavinda K.G.A  |   2023-361

17

11/5/2023

18 of 51

Research Problem

​

  • The current systems should be turned on manually

​

​

  • It causes many problems like if the caretakers can forget to activate the system, it will not be useful.

​

​

  • The system should automatically deactivate when elder is not alone.

​

​

IT20211714|   Kavinda K.G.A  |   2023-361

18

11/5/2023

19 of 51

Objectives

​

  • Main Objective 
  • Automatic activation and deactivation of the system based on the scenario.

​

  • Sub Objective
  • Detect motion and take images.
  • Send the images taken from all devices to the central device and send to the backend .
  • Send the images to the ML model and based on the prediction activate and deactivate the system.

​

​

IT20211714|   Kavinda K.G.A  |   2023-361

19

11/5/2023

20 of 51

20

11/5/2023

Current Progress

Dataset

​

​

​

  • Collected adult's dataset from Google and some Sri Lankan elderly homes.
  • Collected total 100 images
  • Classes : Group, Alone

20

11/5/2023

21 of 51

Current Progress

IT20211714|   Kavinda K.G.A  |   2023-361

​

Choosing MobileNet

  • Lightweight and accuracy is high
  • Suitable for mobile devices
  • Faster to train and deploy.
  • Fast result predictions.

​

​

21

11/5/2023

22 of 51

Current Progress

IT20211714|   Kavinda K.G.A  |   2023-361

​

Model accuracy and loss...

​

​

​

Achieved 93.67% Accuracy

22

11/5/2023

23 of 51

IT20208462 | Gunaratne B.S

Software Engineering

IT20208462 | Gunaratne B.S | 2023-361

ITXXXXXXXX | <<Student Name>> | <<Project ID>>

23

11/5/2023

24 of 51

Voice and Sound Recognition

IT20208462 | Gunaratne B.S | 2023-361

Sinhala Automatic Speech Recognition System with Enhanced Elderly Voice Recognition

24

11/5/2023

25 of 51

Background

  • Sinhala is a low resource language.

​

  • Most of the technologies does not support for Sinhala.

​

  • Automatic Speech Recognition (ASR) is widely used technology.

​

  • There are some ASR systems for Sinhala as well.

IT20208462 | Gunaratne B.S | 2023-361

25

11/5/2023

26 of 51

Background (Continue…)

​

  • Computer literacy rate of Sri Lanka by age
    • 60 – 69 years: 7%

​

  • Computer literacy rate of Sri Lanka by Language Literacy
    • Sinhala : 40.4%
    • English : 76.3%

​

  • Giving voice commands/speaking is an easy and widely used method.

​

  • It reduces the gap between the new technologies and people.

IT20208462 | Gunaratne B.S | 2023-361

26

11/5/2023

27 of 51

Research Problem

  • There are no elderly monitoring systems support Sinhala voice commands.

​

  • The available Sinhala ASR systems are not enhanced for elderly voices.

​

  • There are not enough resources to train an elderly voices enhanced ASR system in Sinhala.

IT20208462 | Gunaratne B.S | 2023-361

27

11/5/2023

28 of 51

Objectives

  • Main Objective
    • Develop an Automatic Speech Recognition system, which is enhanced to elder voices and can classify the intent of what the user is saying.

IT20208462 | Gunaratne B.S | 2023-361

28

11/5/2023

29 of 51

Objectives (Continue…)

  • Sub Objectives
    • Train the ASR model using minimum amount of Sinhala voices.

​

    • Improve the accuracy of recognizing elderly voices.

​

    • Predicts the intent of what the user is saying.

​

    • Create a new corpus of Sinhala elderly voices.

IT20208462 | Gunaratne B.S | 2023-361

29

11/5/2023

30 of 51

30

11/5/2023

Current Progress

  • Collected elderly Sinhala voices from YouTube, and some manual recordings.
  • Processed the collected voices and transcribed them into text.

30

30

11/5/2023

31 of 51

Current Progress

  • Audio samples

IT20208462 | Gunaratne B.S | 2023-361

31

31

11/5/2023

32 of 51

Current Progress

  • Audio waveform and Spectrogram

IT20208462 | Gunaratne B.S | 2023-361

32

32

11/5/2023

33 of 51

33

11/5/2023

Current Progress

  • Loss and Accuracy Plots

​

​

​

Achieved 90.02%

Accuracy

33

33

11/5/2023

34 of 51

IT20216146 | Wijethunga W.M.T.S

Software Engineering

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

ITXXXXXXXX | <<Student Name>> | <<Project ID>>

34

11/5/2023

35 of 51

Background

  • What is a behavioral pattern analysis of an elderly person?
  • Why it is important to an elderly monitoring system?

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

35

11/5/2023

36 of 51

Research Problem

  • How does a caretaker detect their older person is in physically active status?

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

36

11/5/2023

37 of 51

Main Objective

Using video recordings, identify the elder's movements by analyzing the behavioral patterns and create a deep learning model to assess the activities.

​

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

37

11/5/2023

38 of 51

Sub Objectives

  • Develop a deep learning model to identify the motions and activities performed by the elderly person using the uploaded video.
  • Calculating activity level as a percentage according to the elder's behavioral pattern analysis.
  • Giving predictions according to behavioral patterns analysis.
  • Sending alerts to caretakers according to the predictions.

​

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

38

11/5/2023

39 of 51

Current Progress

  • Study different action recognition approaches.
    1. CNN (Convolutional Neural Network) ResNet50 Approaches
    2. Skeleton/Graph Based Approaches.(STGCN - Spatial Temporal Graph Convolutional Networks)
  • Select the best approaches.
  • Implement the selected approaches and test them.
  • Choose the best approach for adult action recognition.

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

39

39

11/5/2023

40 of 51

40

11/5/2023

Current Progress

Dataset

​

​

​

  • Collected adult's dataset from Kaggle, Roboflow and some Sri Lankan elderly homes
  • Collected total 965 images
  • Classes : Walking, Standing, Sitting & Sleeping

40

40

11/5/2023

41 of 51

Current Progress

  • Selection of the best approach. CNN (Convolutional Neural Network) ResNet50
  • Why ResNet50 ?
    • In Skeleton/Graph Based approaches data preprocessing stages are more complex and time consuming.
    • ResNet50 is suitable for activity recognition, extracting features, and classifying actions from images.

​

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

41

41

11/5/2023

42 of 51

Current Progress

  • Choose the best approach for adult action recognition.

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

42

42

11/5/2023

43 of 51

Current Progress

  • Loss and Accuracy plots of the model by epochs.

​

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

Achieved 87.75% Accuracy

43

43

11/5/2023

44 of 51

Current Progress

  • Apply Transfer Learning to the selected approach.

​

​

​

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

44

44

11/5/2023

45 of 51

Requirements

  • Functional Requirements

​

    • Identify elder’s activities and behavioral patterns.
    • Activity level prediction by using behavioral pattern analysis.
  • Non-Functional Requirements

​

    • Reliability
    • Security
    • Speed
    • Usability

​

IT20216146   |   Wijethunga W.M.T.S   |   2023-361

45

45

11/5/2023

46 of 51

Tools and Technologies

46

11/5/2023

47 of 51

Commercialization

  • Main target group – Caretakers
  • Promote application using social media platforms.
  • Develop a mobile application with two languages.(English/Sinhala)
  • Caretakers can monitor their elders when they are away from home.

​

47

47

11/5/2023

48 of 51

Future Progress

  • Further developments of mobile app and the device
  • Test bugs and fix
  • Improve the UI/UX
  • Improve the device functions
  • Finalize the device and app

48

48

11/5/2023

49 of 51

Demonstration

49

49

11/5/2023

50 of 51

50

50

11/5/2023

51 of 51

THANK YOU!

51

51

11/5/2023