A SECURED PLATFORM FOR PHARMACEUTICAL AND

MEDICAL CONSULTING SERVICES

A SECURED PLATFORM FOR PHARMACEUTICAL AND MEDICAL CONSULTING SERVICES

CDAP | Project Progress Presentation

Project ID : 2022-164

2022 / 02 / 02 1

Our Supervisor

Mr. Thusithanjana Thilakarthna

LECTURER

FACULTY OF COMPUTING | COMPUTER SCIENCE & SOFTWARE ENGINEERING

2022 / 02 / 02 2

Our Co-Supervisor

Mr. Didula Chamara

​

LECTURER

FACULTY OF COMPUTING | COMPUTER SCIENCE & SOFTWARE ENGINEERING

2022 / 02 / 02 3

Our Team

Perera N.D.A.

Jayasinghe J.P.N.H.

Gangegedara T.K.

IT19146416

Software Engineering

IT19210070

Software Engineering

IT19210216

Software Engineering

Samiraja D.S.M.M.C.A.B.

IT19153582

Information Technology

2022 / 02 / 02 4

Introduction

Research Problem

Objectives

System Overview

Subcomponents

Commercialization

Content

2022 / 02 / 02 5

INTRODUCTION

2022 / 02 / 02 6

Introduction of mHealth

​

• The usage of mobile phones is increased in present society.
• The value of handwritten prescriptions is well known in Sri Lanka, where they are still widely used by physicians.
• People face many problems due to illegible handwriting and the medical abbreviations of physicians.
• In this presentation, we propose the mHealth mobile application to overcome these aspects.

​

2022 / 02 / 02 7

Older users and physically disabled persons can’t use a mobile application

Elder people are not flexible with interfaces

Sometimes pharmacists’ couldn’t identify the prescription

Can’t trust about secure of the data

Research Problem

2022 / 02 / 02 10

Our Objectives

• Identify medication or read a prescription detecting through mobile camera

​

• Use voice recognition for identifying medicine

​

• Secure the application

​

• Make the application flexible for elder and younger people alike

2022 / 02 / 02 11

System Overview

2022 / 02 / 02 12

Perera N.D.A

IT19146416

Software Engineering

2022 / 02 / 02 13

Ethereum block-chain based network to secure and store medical records

2022 / 02 / 02 14

IT19146416 | Perera N.D.A | 2022-164

Introduction

​

• Personal health records (PHRs) have played a key role in enabling safer, more efficient, and consumer-driven healthcare systems.

​

• Although automated PHR solutions can enable individuals to manage their data efficiently.

​

• Blockchain is a promising technology that has the potential to reshape the way data is being controlled or managed in existing PHRs management systems

​

​

2022 / 02 / 02 15

IT19146416 | Perera N.D.A | 2022-164

Research Problem

Can a user trust to enter the patient details?

Can patients control their medical records?

2022 / 02 / 02 16

IT19146416 | Perera N.D.A | 2022-164

Main Objective

To develop an Ethereum block-chain based network to secure and store medical records

2022 / 02 / 02 17

IT19146416 | Perera N.D.A | 2022-164

Specific Objectives

Implement a trusted reputation-based re-encryption oracle to securely fetch and share patients’ medical data

A reliable mechanism is created for the doctors to store medical data and access the historical data

2022 / 02 / 02 18

IT19146416 | Perera N.D.A | 2022-164

METHODOLOGY

2022 / 02 / 02 19

IT19146416 | Perera N.D.A | 2022-164

Content

2022 / 05 / 26 20

IT19146416 | Perera N.D.A | 2022-164

System Diagram

Achievements

Technologies

Current Progress

Task to be Completed

Requirements

Risk Mitigation

System Diagram

2022 / 05 / 26 21

IT19146416 | Perera N.D.A | 2022-164

Proof of work

• Remix IDE

• Encryption / Decryption

IT19146416 | Perera N.D.A | 2022-164

2022 / 05 / 26 22

• Blockchain UI

• Metamask Wallet

• Encryption/Decryption UI

IT19146416 | Perera N.D.A | 2022-164

2022 / 05 / 26 23

Tools &Technologies

Blockchain

Decentralized network

Ethereum Blockchain System

Remix IDE

Smart Contracts

2022 / 05 / 26 24

IT19210216 | Gangegedara T.K. | 2022-164

Metamask Wallet

Proxy Re-encryption Schema

Current Progress

IT19146416 | Perera N.D.A | 2022-164

2022 / 05/ 26 25

• Implement blockchain network
• Connect the Ethereum blockchain system to the application
• Created meta mask wallet
• Create proxy re-encryption schema

Task To Be Completed

IT19146416 | Perera N.D.A | 2022-164

2022 / 05 / 26 26

• Develop a Proof of Work mechanism
• Give access control to the relevant user
• View medical records and comparison
• Integrate the application

Requirements

Functional Requirements

• Should secure all the patient’s medical records
• Doctors can access the patient’s records and they can store the details
• Patients can see their records but couldn’t modify the records

​

Non-functional Requirements

• Accuracy
• Reliability
• Availability
• Privacy

2022 / 05 / 26 27

IT19146416 | Perera N.D.A | 2022-164

Risk Mitigation

IT19146416 | Perera N.D.A | 2022-164

2022 / 05 / 26 28

• Couldn’t purchase more forces to the meta mask wallet

Jayasinghe J.P.N.H.

IT19210070

Software Engineering

2022 / 02 / 02 29

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Use voice recognition to search medications and get relevant information

2022 / 02 / 02 30

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Introduction

​

• Human speech is the most natural mode of human communication. It provides information about linguistic content and para-linguistic states and traits.

​

• Technology-based health can be utilized to provide support to the healthcare system.

​

• Speech processing research is currently gaining interest to utilize computational paralinguistic analysis for the assessment of different health conditions.

​

​

​

2022 / 02 / 02 31

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Background

Why do healthcare Apps need such kind of feature?

• The prime reason to use speech for healthcare is that it can be easily available, collected, transmitted, and stored.
• Most importantly, various physical and mental diseases cause changes in human speech, which are measurable with the help of speech technology.

​

How our research varies from others?

• Most of the applications use texts as the search inputs.
• Existing or suggested systems only use speech-to-text, and they do not have a text-to-speech feature to read a description.

2022 / 02 / 02 32

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Research Problem

Can elders or people with some reading difficulties use this application easily?

Do you know the usages of the medicine you use?

2022 / 02 / 02 33

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Main Objective

Voice recognition feature to search the medicines and their details

2022 / 02 / 02 34

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Specific Objectives

Get to know about various medications

Use text-to-speech feature for listening to the description

2022 / 02 / 02 35

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

METHODOLOGY

2022 / 02 / 02 36

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Content

2022 / 05 / 26 37

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

System Diagram

Achievements

Technologies

Current Progress

Task to be Completed

Requirements

Risk Mitigation

System Diagram

2022 / 05 / 26 38

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Proofs

• After training the model, test accuracy was 97%

2022 / 05 / 26 39

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

• After training the model, test accuracy was 97%

2022 / 05 / 26 40

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

• Make prediction and test the module

2022 / 05 / 26 41

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Technologies

Python

Librosa

CNN

TensorFlow

Keras

2022 / 05 / 26 42

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Current Progress

• Collected User requirements.

​

• Created a dataset using voice clips(.wav) of medicine names.

​

• Preprocessed the dataset and build a CNN model

​

• trained the model.

​

• Take predictions and detect the words from audio clips.

2022 / 05 / 26 43

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Task To Be Completed

• Creation of mobile app.

​

• Improve accuracy.

​

• Get related medicine details with translation.

​

• read the description with text-to-speech feature.

2022 / 05 / 26 44

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Requirements

Functional Requirements

• Should search medicines by voice as input
• Should read the searched description after tapping the text-to-speech button
• Should read the medicine name after recognized by image processing

​

Non-functional Requirements

• Accuracy
• Reliability
• Availability
• Usability

2022 / 05 / 26 45

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Risk Mitigation

• Sometimes we are unable to detect words properly when we try to detect unclear voice clips.

​

• Had to create a dataset manually according to requirements and collecting voice clips because could not find an accurate dataset.

2022 / 05 / 26 46

IT19210070 | Jayasinghe J.P.N.H. | 2022-164

Gangegedara T.K.

IT19210216

Software Engineering

2022 / 05 / 26 47

IT19210216 | Gangegedara T.K. | 2022-164

Identify medicines and read prescriptions by detecting through a mobile camera

2022 / 05 / 26 48

IT19210216 | Gangegedara T.K. | 2022-164

Introduction

​

• In Sri Lanka most physicians use handwritten prescriptions when prescribing medicines.

​

• Sometimes it’s very difficult to read prescriptions for both patients and pharmacists.

​

• Cause physicians use illegible handwriting and medical abbreviations.

​

• In the proposed system mHealth, it detects medications and prescriptions through the mobile camera and identifies the medicines.

​

​

2022 / 05 / 26 49

IT19210216 | Gangegedara T.K. | 2022-164

Research Problem

Difficult to read handwritten prescriptions

Lack of knowledge about medicines

Can’t get pharmacy details

Cause too many diseases by getting wrong medicines

2022 / 05 / 26 50

IT19210216 | Gangegedara T.K. | 2022-164

Main Objective

Provide accurate medicine descriptions by identifying the medicine box or reading prescriptions through the camera

2022 / 05 / 26 51

IT19210216 | Gangegedara T.K. | 2022-164

Specific Objectives

Get all the pharmacy details

Get to know about various medications

Get the nearest pharmacy that medicine is available

2022 / 05 / 26 52

IT19210216 | Gangegedara T.K. | 2022-164

METHODOLOGY

2022 / 05 / 26 53

IT19210216 | Gangegedara T.K. | 2022-164

System Diagram

Achievements

Technologies

Current Progress

Content

2022 / 05 / 26 54

IT19210216 | Gangegedara T.K. | 2022-164

Task to be Completed

Requirements

Risk Mitigation

System Diagram

2022 / 05 / 26 55

IT19210216 | Gangegedara T.K. | 2022-164

Achievements

​

• Method 1

CNN based Neural Network Model, TensorFlow , OpenCV

​

​

​

• Method 2

CNN, Tesseract , OpenCV

​

​

​

2022 / 05 / 26 56

IT19210216 | Gangegedara T.K. | 2022-164

In method 1 train a CNN model and get accuracy around 0.8

2022 / 05 / 26 57

IT19210216 | Gangegedara T.K. | 2022-164

In method 2 printed medicine or handwritten medicine detected through the webcam.

2022 / 05 / 26 58

IT19210216 | Gangegedara T.K. | 2022-164

Technologies

Python

CNN

Tesseract

Open CV

TensorFlow

2022 / 05 / 26 59

IT19210216 | Gangegedara T.K. | 2022-164

OCR

Current Progress

• Understand & collect user requirements.
• Dataset creation using medicine prescription images.
• Identify the medicine and list down as CSV file.
• Detect the printed and handwritten text & identify the word from a webcam.

2022 / 05 / 26 60

IT19210216 | Gangegedara T.K. | 2022-164

Task To Be Completed

• Creatin of mobile app.
• Improve accuracy.
• Get related medicine details.
• Get nearest pharmacy and track the location using map.

2022 / 05 / 26 61

IT19210216 | Gangegedara T.K. | 2022-164

Requirements

Functional Requirements

• Detect the medicine correctly & accurately.
• Get correct information about the medicine.

Non-functional Requirements

• Correctness
• Performance
• Accuracy

2022 / 05 / 26 62

IT19210216 | Gangegedara T.K. | 2022-164

Risk Mitigation

• An accurate dataset couldn’t be found & had to create a dataset manually considering the requirements and collecting prescriptions.

​

• Unable to identify some unclear prescribed medicines through the detector.

2022 / 05 / 26 63

IT19210216 | Gangegedara T.K. | 2022-164

Samiraja D.S.M.M.C.A.B.

IT19153582

Information Technology

2022 / 05 / 26 64

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Design Flexible Interfaces For Both Elder & Younger People According To User Interaction

2022 / 05 / 26 65

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Introduction

​

• Most of the elder people avoid using mobile phones because they are not familiar and not flexible with them.

​

• Due to these aspects, they face many troubles.

​

• To overcome these issues in our proposed system mHealth, we suggest designing flexible interfaces for both elders and younger alike.

​

2022 / 05 / 26 66

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Research Problem

Elder people can’t clearly see the interfaces

Language may not understandable for everyone

Too many actions to do within a task

2022 / 05 / 26 67

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Main Objective

To Make sure that the application is flexible for both elder and younger users

2022 / 05 / 26 68

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Specific Objectives

Make sure the application is understandable for everyone

Use different languages

Use different types of login

2022 / 05 / 26 69

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

METHODOLOGY

2022 / 05 / 26 70

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

System Diagram

Achievements

Technologies

Current Progress

Content

2022 / 05 / 26 71

Task to be Completed

Requirements

Risk Mitigation

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

System Diagram

2022 / 05 / 26 72

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Achievements

Create UI using flutter and flutter lab.

2022 / 05 / 26 73

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Detect the person from the webcam and identify age and gender correctly.

2022 / 05 / 26 74

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Technologies

Python

Figma

Flutter

Open CV

Flutter Lab

2022 / 05 / 26 75

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Current Progress

• Train the model and get the accuracy successfully
• Identify the age groups and gender
• Front-end implementation
• Change the UI features according to the age groups

​

2022 / 05 / 26 76

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Task To Be Completed

• Finalize the user interfaces in the mobile application
• Test the application using real-time users

​

​

2022 / 05 / 26 77

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Requirements

Functional Requirements

• Identify the correct age group.
• UI and language changes as per the user requirements.

Non-functional Requirements

• User friendly
• Flexible
• Correctness

2022 / 05 / 26 78

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Risk Mitigation

​

• Difficult to get accurate age in some persons.
• When interface designing it requires more space in RAM for Android studio.

​

2022 / 05 / 26 79

IT19153582 | Samiraja D.S.M.M.C.A.B. | 2022-164

Commercialization

• Customized logo and specific brand name
• Target Audience –

Pharmacists, Doctors, Patients & Persons.

​

• Free Application

Free access to the application

in the first 3 months.

​

• Advertisement

Publish a video trailer of our

product in social media platforms.

2022 / 05 / 26 80

Any Questions?

2022 / 05 / 26 81

THANK YOU !

2022 / 05 / 26 82