Camera-based interface to track hand function
Congyi Zhu1 , Sasha Portnova2, Adria Robert Gonzalez3, Emily Boeschoten3, Kat Steele4, Heather Feldner3
Departments of 1Human Centered Design & Engineering, 2Computer Science & Engineering, 3Rehabilitation Medicine, 4Mechanical Engineering at the University of Washington, Seattle
Assessment Time
Many people in the US require hand therapy rehabilitation as they develop arthritis, carpal tunnel syndrome, or experience a traumatic hand or wrist injury.
Hand function assessments are an important aspect of therapy and rehabilitation that are laborious and require skill. Clinicians have limited time to provide therapy and document assessment results.
Patients may benefit from the presentation of results in a clear visual format.
GOAL: to develop a simple interface for hand therapy patients and/or clinicians to perform efficient hand function assessments and track results over time
MOTIVATION
We asked clinicians, who are involved in providing hand therapy, to answer questions about their experiences with function assessments in their current practices. Response analysis yielded the emergence of the following themes:
CLINICIANS’ OUTLOOK
INTERFACE DEVELOPMENT
LOOKING AHEAD
Goniometer
Dynamometer
Pinch Gauge
RANGE OF MOTION
NUMBERS FOR VISUALIZATION
COMPARISON TO NORMS
IMPORTANCE OF PROGRESS TRACKING
Webcam-Based
MediaPipe Hand
Finger Range-of-Motion
Step-by-Step Instructions
Visualization of Results
Finalize the interface in Unity
Show-and-tell with clinicians
Feedback incorporation
Open-source application release
Future app support
We will also be looking into long-term sustainability plans for interface support, incorporating tracking of other joints (e.g., wrist, shoulder, elbow), as well as expanding to other populations, such as neurological injuries and burns.