Product Development:�IoT Patient Monitoring System

​

Priscilla Cheng, Nicholas Eu, Austin Lazaro

EE180D Networked Embedded Systems Design

Product Development Mission Background

• Lightweight, scalable system to monitor patient motion and activity
• Posture identification, fall risk detection
• Room ID verification
• Current solutions involve pressure/depth sensors
• can be very inaccurate (prone to false alarms)
• cumbersome to install/move
• Results in an expensive and unscalable system

​

​

Product Development Mission Background

• Utilize IoT and Edison/9DOF sensor to create low power, plug and play solution.
• Great benefit to hospital employees and management, as well as hospital patients
• May be expanded to workplace, home monitoring
• In line with the emergence of small wearables

Design Overview: Technical Challenges

• Portability and mounting stability
• Accuracy of the accelerometer, gyro data (problems with drift)
• Communication and coordination between Edisons
• IR emitter power source (Breakout Board vs. 9V battery)
• Calculating rotation angle using gyroscope data
• Integration between IR sensing code,fall risk detection code, and GUI
• Effective, easy to use/read GUI
• Scalability to multiple users

System Implementation: Progress to Date

​

• User position is determined by calculating and analyzing the pitch, roll, and yaw of the wearable sensors.
• standing up, sitting up, lying face down, lying face up, lying on left, lying on right
• Motion/rotation is detected through integration of angular velocity.
• For our implementation, we are only concerned about the change in yaw over a particular time interval, so drift was not too big of an issue for us.

System Implementation: Progress to Date

• Set up TCP connection between the upper body edison (master) and the lower body edison (slave)
• lower body edison sends individual posture to upper body
• upper body integrates lower body data into full posture
• Master edison sends posture as an integer to the IoT cloud and to GUI
• Now we’re just sending packets of information to python script directly
• Packets hold posture, fall risk, and room number
• Simple stop and wait protocol to avoid congestion
• Setup Python GUI canvas using Tkinter library to display current posture

Testing and Verification: Progress to Date

1. Patient position sensing / 9DOF sensor
1. Read outputs from x,y,z accelerometer data
2. Assign position to specific threshold values
3. Place 9DOF on shoulder, change positions, see if position correctly updates

​

2. Transmission from Edison to cloud

• Output accelerometer data on command line
• Monitor IoT dashboard chart
• Shake 9dof sensor
• Confirm that IoT dashboard chart has updated

​

​

Testing and Verification: Progress to Date

3. Communication between Edisons

• Set up TCP connection between the master and slave Edison
• Ensured connection was successfully established first
• Output data being received from the slave Edison

​

4. Obtaining data from cloud / displaying on GUI

• First display all data pulled from cloud using API
• Pull only most recent posture value
• Update posture on GUI

​

5. Sending data straight to GUI

• Set up TCP socket connection between upper body sensor and GUI script
• Display posture on GUI based on what is received from sensor
• Output raw values received and ensure GUI updates correctly

Testing and Verification: Progress to Date

6. IR Communication

• Switch to arduino breakout board because of noise issues on GPIO block
• Decide on sampling rate and output modulated signal from IR emitter
• Signal is organized into packets: preamble | message | postamble
• Output received bits on receiver side and ensure packet is correctly read

7. Detecting patient rotation

• First refine gyroscope integration algorithm until accurate rotation readings are achieved on the table
• Have someone wear edisons and simulate more realistic human motion/rotation - ensure accuracy of readings is preserved
• Determine whether or not rotation warrants a “fall risk” warning

8. Detecting patient fall

• Have subject wear edisons and simulate falls multiple times
• Observe behavior of gyroscope sensor data to determine features that are typical for falls
• Adjust algorithm to detect these features in order to detect falls

Testing and Verification: Progress to Date

9. IPC: 9dof sensor and IR receiver

• Coordinate communication between two separate programs
• IR receiver
• Posture monitoring, fall risk detection
• Run IR receiver program in background and have it write messages from IR communication to a file
• Run posture monitoring/fall risk program; ensure it can read from the shared file and process information

10. Putting it all together: GUI, posture monitoring, and IR beacons

• Start all programs and ensure all basic functions work independently
• Place IR receiver in front of IR beacon; ensure room ID is displayed on GUI
• Simulate fall risk scenario and actual fall: ensure these events are captured on the GUI

Final System Mission Progress Report

• Determining patient fall-risk through sequences of postures and rotation
• New goal was established during this quarter
• Calculation of rotation via pitch and yaw angles
• Pitch/yaw angles calculated via accelerometer data
• Tested on person to determine appropriate rotation threshold
• Saved history of postures, and combined with rotation, to develop algorithm to identify a fall-risk
• Introduced IR sensing into system
• Achieved goal set out in Quarter 1
• Changed objective to identify room or patient ID
• Preamble and postamble added to bit sequence for security
• 3 state system: read preamble -> read sequence -> read postamble
• Used timer to read individual bits at sampling rate

​

Final System Mission Progress Report

• Refine UI of GUI program to be more user friendly
• Achieved goal set out in Quarter 1
• Added pictures for clarity
• Introduced Text widget to see raw standard output
• Detect actual falls for patients
• Simple (and slightly crude) algorithm developed after fall risk was finished
• Can detect falls from a standing position by looking for spikes in gyro sensor data
• Spikes in gyro data asserts a flag;
• if flag is set at the beginning of next 5-second interval, determine whether or not it was a fall by studying the posture sequence

Demo: Basic Posture/Fall Risk Detection

Demo: Fall Detection

Demo: Patient Identification Beacons

Demo: Patient Identification Beacons

Final System Development

• Approach
• Determined each group member’s schedule and availability
• Met together twice a week, outside of lab, to discuss individual progress and continued to work towards goals during that time
• Weekly updates via FB Chat on Sunday, see what each group member has accomplished
• Uploaded files to Google Drive so members could see and work on them and committed changes to Github

Priscilla

• Sensors/Hardware
• Develop algorithm for patient posture sensing, fall detection, and activity monitoring using 9dof sensor data
• Implement robust algorithm for detecting all combinations of patient motion
• Develop reliable test procedures that account for all patient behavior
• Assist in implementing communication protocol within the system

Austin

• Networking/connectivity
• ensure nodes in network are capable of data transfer (node to cloud, node to node)
• Implement graceful startup and shutdown of connections
• ensure reliability of nodes under varying connection conditions
• fault handling, slow connections, etc
• security of data transfer
• optimal networking protocols
• Assist in all other aspects of development (GUI, IR sensing, posture monitoring)

Nick

• User Interface/IR hardware
• Develop GUI program in Python
• Determine user interactions with the program
• Determine layout of program and functionality
• Setup IR emitter and receiver circuits
• Determine optimal positioning of IR emitters
• Examine and interpret output of IR receiver

Resources and References

Resources and References consulted:

�https://github.com/enableiot/iotkit-api/wiki/Api-Home�https://github.com/enableiot/iotkit-lib-python/blob/master/iotkitclient/account.py�http://www.tutorialspoint.com/python/python_gui_programming.htm

http://web.cs.wpi.edu/~claypool/courses/525-S01/samples/setitimer.c

http://www.x-io.co.uk/open-source-imu-and-ahrs-algorithms/

https://www.semanticscholar.org/paper/Wagyromag-Wireless-sensor-network-for-monitoring-Olivares-Olivares/0e2a33141f2e23c57e32bde3f13f451b3820922e/pdf

​

​

Conclusion: Future Development Opportunities

• Test with more Edisons and IR beacons
• Verify goal of implementing scalable system
• Differentiation between multiple edisons
• GUI can be extended to interact with many edisons and distinguish between them
• Associate patients with the edisons they’re wearing and display on GUI
• Increased functionality of IR beacons
• Full Duplex communication
• Increased range and portability
• More user-friendly GUI with more interaction with the display
• Multiple tabs/screens for each edison/patient
• Capture history of patients’ movements and display on command
• Map of rooms?

Conclusion: Future Development Opportunities

• Extend from posture monitoring to orientation monitoring
• Use extracted Euler angles to display true orientation of patient
• Explore the calculation of quaternions instead of Euler angles?
• Add some way to signal patient when fall risk is detected
• Buzzer or light to notify them to lie back down
• Eliminate need for timer to keep track of posture sequences
• Better way to harness sensors to body