In-Sensor Movement Variability Tracking

Swapnil Sayan Saha, Krishna Chaitanya Palle, Mahesh Chowdhary

STMicroelectronics Inc.

Movement variability

2

Variability in motion patterns effected by neuromusculoskeletal system

prevents recreating same movement trajectory

Indicator of mobility, pathological conditions, and psychological disorders

Movement variability in tennis serve

1

2

3

4

Movement variability during 4 cycles

of walking

Requirements for tracking movement variability

3

Ubiquitous for activities of daily living

Low cost, low-power,

and portable

Not dependent on specialized infrastructure in self-care settings

Must not need user intervention

In-sensor movement variability tracking

4

Our intelligent sensor processing unit can guide the user to monitor repetition of motion patterns using 0.2 mA current, 6.9 kB of memory, and 6 seconds of training data per trajectory

Intelligent sensor processing unit

5

ISPU core: 8-kilogate RISC STRED processor running at 5-10 MHz

Supports standard C code and neural network execution

Can execute 30 separate algorithms running concurrently using interrupts

Consumes 5x lower current and 2000x lower memory than microcontrollers

Principal Component Extraction

6

Training: collect as many repetitions of a trajectory for maximal information gain and noise removal

Inference: provide feedback with minimal delay (inference buffer size, N < training buffer size, M)

Template from 10

seconds of data

Template from 3 seconds of data with maximal variance

Bicep Curl

Lateral Raise

Template Creation and Storage

7

Extract the gravity vector swing, g from roll, Φ and pitch, θ calculated from accelerometer samples, a

Select the two swing axes u with maximal variance and create a quantized image template P

Store u, P, and motion counter as an array of structures

Bucketing and quantization

Template pixel values

Gravity swing calculation

Roll and pitch calculation

Template Matching

8

The universal image quality index, Q is used to match stored templates, e with current activity template, f (if axis variance information cannot do a classification)

Movement Variability Measure

9

Flatten e and f, then divide into D temporal chunks of size, E

For each chunk, d present the movement variability measure, R as a heatmap

a₃ is equivalent to Q. q is the min-max normalization function

Scoring scheme example: high score for ideal match; low score for bad match; medium score for deviations or corrections

Variability Tracking Example

10

(a) Ideal match (b) speed mismatch (too fast) (c) correction (d) deviation

Accuracy and Resource Usage

11

96.7% test accuracy for 6-class activity recognition across 30 gym activity sessions*

* wrist-mounted wireless sensor; 12 participants; proctor-guided ground truth

6.88 kB data memory; 20 kB program memory (all on sensor)

0.21 mA current consumption (peak 0.34 mA)

Summary

12

The solution runs on low-cost inertial sensor using extremely small amount of power,

memory, and compute resources

Only a few seconds of data are needed for training without requiring calibration, user intervention, or drift correction

Heatmap provides interpretable and actionable insights to a patient

for self-care/rehabilitation applications

Applications: rehabilitation, sports biomechanics, gaming, movie

production, and disease diagnosis

© STMicroelectronics - All rights reserved.�ST logo is a trademark or a registered trademark of STMicroelectronics International NV or its affiliates in the EU and/or other countries. For additional information about ST trademarks, please refer to www.st.com/trademarks. �All other product or service names are the property of their respective owners.