A SECURED PLATFORM FOR PHARMACEUTICAL AND

MEDICAL CONSULTING SERVICES

A SECURED PLATFORM FOR PHARMACEUTICAL AND MEDICAL CONSULTING SERVICES

CDAP | Project Progress Presentation - 2

Project ID : 2022-164

2022 / 10 / 11 1

Perera N.D.A.

Jayasinghe J.P.N.H.

Gangegedara T.K.

IT19146416

Software Engineering

IT19210070

Software Engineering

IT19210216

Software Engineering

Our Team

Mr. Thusithanjana Thilakarthna

LECTURER

FACULTY OF COMPUTING | COMPUTER SCIENCE & SOFTWARE ENGINEERING

2022 / 10 / 11 2

Mr. Didula Chamara

​

LECTURER

FACULTY OF COMPUTING | COMPUTER SCIENCE & SOFTWARE ENGINEERING

Samiraja D.S.M.M.C.A.B.

IT19153582

Information Technology

Introduction

Research Problem

Objectives

System Overview

Subcomponents

Commercialization

Content

2022 / 10 / 11 5

INTRODUCTION

2022 / 10 / 11 6

Introduction of mHealth

​

• The usage of mobile phones is increased in present society.
• The value of handwritten prescriptions is well known in Sri Lanka, where they are still widely used by physicians.
• People face many problems due to illegible handwriting and the medical abbreviations of physicians.
• In this presentation, we propose the mHealth mobile application to overcome these aspects.

​

2022 / 10 / 11 7

How to identify a particular drug name with its description

Elder people are not flexible with interfaces

Sometimes pharmacists’ couldn’t identify the prescription

Can’t trust about secure of the data

Research Problem

2022 / 10 / 11 10

Our Objectives

2022 / 10 / 11 11

System Overview

2022 / 10 / 11 12

Perera N.D.A

IT19146416

Software Engineering

2022 / 10 / 11 13

Block-chain based network to secure and store medical records

2022 / 10 / 11 14

IT19146416 | Perera N.D.A | 2022-164

Introduction

​

• Personal health records (PHRs) have played a key role in enabling safer, more efficient, and consumer-driven healthcare systems.

​

• Although automated PHR solutions can enable individuals to manage their data efficiently.

​

• Blockchain is a promising technology that has the potential to reshape the way data is being controlled or managed in existing PHRs management systems

​

​

2022 / 10 / 11 15

IT19146416 | Perera N.D.A | 2022-164

Research Problem

Can a user trust to enter the patient details?

Can patients control their medical records?

2022 / 10 / 11 16

IT19146416 | Perera N.D.A | 2022-164

Main Objective

To develop a blockchain based network to secure and store medical records

2022 / 10 / 11 17

IT19146416 | Perera N.D.A | 2022-164

Specific Objectives

A mechanism is created for the pharmacists to store medicine data

Implement a proxy re-encryption schema to secure the patient details

2022 / 10 / 11 18

IT19146416 | Perera N.D.A | 2022-164

METHODOLOGY

2022 / 10 / 11 19

IT19146416 | Perera N.D.A | 2022-164

Content

2022 / 10 / 11 20

IT19146416 | Perera N.D.A | 2022-164

Proof of work

Tools &Technologies

Current Progress

Task to be Completed

Proof of work

• Blockchain.py

• DbController.py

IT19146416 | Perera N.D.A | 2022-164

2022 / 10 / 11 22

• Model.py

• MongoDB

IT19146416 | Perera N.D.A | 2022-164

2022 / 10 / 11 23

Tools &Technologies

Blockchain

Decentralized network

Python

2022 / 10 / 11 24

IT19146416 | Perera N.D.A. | 2022-164

AES algorithm

Current Progress

IT19146416 | Perera N.D.A | 2022-164

2022 / 10 / 11 25

• Implement a blockchain network
• Connect the blockchain system to the application
• Created a backup plan
• Create a proxy re-encryption schema

Task To Be Completed

IT19146416 | Perera N.D.A | 2022-164

2022 / 10 / 11 26

• Test the application with real users

​

Jayasinghe J.P.N.H.

IT19210070

Software Engineering

2022 / 10 / 11 29

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Use voice recognition to search medications and get relevant information

2022 / 10 / 11 30

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Introduction

​

• Human speech is the most natural mode of human communication. It provides information about linguistic content and para-linguistic states and traits.

​

• Technology-based health can be utilized to provide support to the healthcare system.

​

• Speech processing research is currently gaining interest to utilize computational paralinguistic analysis for the assessment of different health conditions.

​

​

​

2022 / 10 / 11 31

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Research Problem

Can elders or people with some reading difficulties use this application easily?

Do you know the usages of the medicine you use?

2022 / 10 / 11 33

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Main Objective

Voice recognition feature to search the medicines and their details

22022 / 10 / 11 34

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Specific Objectives

A reliable mechanism is created for the doctors to store medical data and access the historical data

​

Use text-to-speech feature for listening to the description

2022 / 10 / 11 35

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

METHODOLOGY

2022 / 10 / 11 36

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Content

2022 / 10 / 11 20

IT19146416 | Perera N.D.A | 2022-164

Proof of work

Tools &Technologies

Current Progress

Task to be Completed

Proof of work

• After training the model, test accuracy was 97%

2022 / 10 / 11 40

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

• After training the model, test accuracy was 97%

2022 / 10 / 11 40

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

2022 / 10 / 11 39

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

2022 / 10 / 11 40

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

2022 / 10 / 11 41

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Proof of Work

​

• Approach 1

Python , CNN, Librosa

​

​

​

• Approach 2

Google cloud API , IBM

​

​

​

2022 / 10 / 11 56

IT19210216 | Gangegedara T.K. | 2022-164

Technologies

Python

Librosa

CNN

TensorFlow

Keras

2022 / 10 / 11 42

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Google Cloud API

Current Progress

• Created a dataset using voice clips(.wav) of medicine names

​

• Trained the model after reprocessing the dataset and building a CNN model

​

• Take prediction after detecting the voice and Identify the medicine

​

• Get a usage of medicine after input a medicine as input

2022 / 10 / 11 43

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Task To Be Completed

2022 / 10 / 11 44

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

• Add more data and train the model
• Implement text to speech feature
• Get feedback from real-time users
• Test the task on application

Samiraja D.S.M.M.C.A.B.

IT19153582

Information Technology

2022 / 10 / 11 64

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Design Flexible Interfaces For Both Elder & Younger People According To User Age Groups

2022 / 10 / 11 65

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Introduction

​

• Most of the elder people avoid using mobile phones because they are not familiar and not flexible with them.

​

• Due to these aspects, they face many troubles.

​

• To overcome these issues in our proposed system mHealth, we suggest designing flexible interfaces for both elders and younger alike.

​

2022 / 10 / 11 66

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Research Problem

Elder people can’t clearly see the interfaces

Too many actions to do within a task

2022 / 10 / 11 67

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Main Objective

To Make sure that the application is flexible for both elder and younger users

2022 / 10 / 11 68

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Specific Objectives

Make sure the application is understandable for everyone

Use different types of login

2022 / 10 / 11 69

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

METHODOLOGY

2022 / 10 / 11 70

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Content

2022 / 10 / 11 20

IT19146416 | Perera N.D.A | 2022-164

Proof of work

Tools &Technologies

Current Progress

Task to be Completed

Proof of work

Create UI using Android Studio.

2022 / 10 / 11 73

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Use the ML model and train the dataset..

2022 / 10 / 11 74

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Technologies

Python

Firebase

Android studio

Keras

Deep face

2022 / 10 / 11 75

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Current Progress

2022 / 10 / 11 76

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Task To Be Completed

2022 / 10 / 11 77

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Gangegedara T.K.

IT19210216

Software Engineering

2022 / 10 / 11 47

IT19210216 | Gangegedara T.K. | 2022-164

Identify medicines and read prescriptions by detecting through a mobile camera

2022 / 10 / 11 48

IT19210216 | Gangegedara T.K. | 2022-164

Introduction

​

• In Sri Lanka most physicians use handwritten prescriptions when prescribing medicines.

​

• Sometimes it’s very difficult to read prescriptions for both patients and pharmacists.

​

• Cause physicians use illegible handwriting and medical abbreviations.

​

• In the proposed system mHealth, it detects medications and prescriptions through the mobile camera and identifies the medicines.

​

​

2022 / 10 / 11 49

IT19210216 | Gangegedara T.K. | 2022-164

Research Problem

Difficult to read handwritten prescriptions

Lack of knowledge about medicines

Can’t get pharmacy details

Cause too many diseases by getting wrong medicines

2022 / 10 / 11 50

IT19210216 | Gangegedara T.K. | 2022-164

Main Objective

Provide accurate medicine descriptions by identifying the medicine box or reading prescriptions through the camera

2022 / 10 / 11 51

IT19210216 | Gangegedara T.K. | 2022-164

Specific Objectives

Get all the pharmacy details

Get to know about various medications

Get the nearest pharmacy that medicine is available

2022 / 10 / 11 52

IT19210216 | Gangegedara T.K. | 2022-164

METHODOLOGY

2022 / 10 / 11 53

IT19210216 | Gangegedara T.K. | 2022-164

Content

2022 / 10 / 11 20

IT19146416 | Perera N.D.A | 2022-164

Proof of work

Tools &Technologies

Current Progress

Task to be Completed

Proof of Work

​

• Approach 1

Tesseract, OCR

​

​

​

• Approach 2

Python, CNN model, OpenCV

​

​

​

2022 / 10 / 11 56

IT19210216 | Gangegedara T.K. | 2022-164

2022 / 10 / 11 57

IT19210216 | Gangegedara T.K. | 2022-164

​

• Create the CNN model , train the dataset and get the accuracy around 78%.

2022 / 10 / 11 58

IT19210216 | Gangegedara T.K. | 2022-164

Technologies

Python

CNN

Keras

Open CV

2022 / 10 / 11 59

IT19210216 | Gangegedara T.K. | 2022-164

Flask

Current Progress

​

• Dataset creation using medicine prescription images.
• Train the CNN model and get the accuracy
• Detect the printed and handwritten text & identify the medicine.
• Display the medicine

​

2022 / 10 / 11 60

IT19210216 | Gangegedara T.K. | 2022-164

Task To Be Completed

• Get the medicine description and pharmacy details
• Test the mobile application
• Get feedback from real users
• Add more data and train the model

2022 / 10 / 11 61

IT19210216 | Gangegedara T.K. | 2022-164

Commercialization

• Customized logo and specific brand name
• Target Audience –

Pharmacists, Doctors, Patients & Persons.

​

• Free Application

Free access to the application

in the first 3 months.

​

• Advertisement

Publish a video trailer of our

product in social media platforms.

2022 / 10 / 11 80

Any Questions?

2022 / 10 / 11 81

THANK YOU !

2022 / 10 / 11 82