The Playbook: Digital Clinical�Measures
Introducing the essential guide for successful remote monitoring across clinical research, clinical care, and public health.
FOCUS: Driving adoption
1
1
Source: Playbook team analysis
A heart-beat is a heart-beat.
Sleep is sleep.
The Playbook / Quick Start Guide / Our goal
Digital clinical measures should:
2
2
Source: Playbook team analysis
The Goal:
To build through collaboration a broadly accepted, shared foundation for developing and deploying digital clinical measures across clinical research, clinical care, and public health.
3
The Playbook / Quick Start Guide / The goal
3
3
Source: Playbook team analysis
Benefits of digital clinical measures span research, care, and public health
To accelerate
clinical research
To enhance
clinical care
To power
public health
4
The Playbook / Quick Start Guide / Benefits
4
4
Source: “Visible vs. Invisible Data” chart designed by Evidation Health, re-worked by Elektra Labs, Playbook team analysis
The Playbook / Quick Start Guide / Benefits
Measuring health using digital sensing products offers a more holistic view of a person’s lived experience
Data collected from traditional visits to hospitals, clinics, and sites
Data collected during �everyday life
Visible Data Points�(episodic)
Invisible Data Points
(continuous)
5
5
Source: Playbook team analysis
The Playbook can help you answer questions such as:
The Playbook / Quick Start Guide / Your questions answered
How do I know what technology product is right for my patients / participants / population?
How do I know what to measure when considering a digital measurement product?
How do I know / can I show that a given digital clinical measure is ‘validated’?
I’m waiting for regulations to catch up with innovations in remote monitoring. How can I help?
I know what I want to measure and the technology to use: Now how do I make it work in practice?
I want to use digital clinical measures ethically. What are the most important considerations?
6
6
NOTE: We intentionally do not use the word “devices” as this is considered a “Term of Art” within regulatory circles and not all connected sensor technologies for remote monitoring will be classified as a medical “device” per the FDA and other regulators.
Source: Playbook team analysis
The Playbook focuses on connected sensor products used outside of the clinic
In Scope
Connected sensor products
Out of Scope
Digital Therapeutics (DTx) - software-only products that intervene
Communication technologies (e.g., video conferencing)
Survey-based assessments that do not require sensors (e.g, electronic patient-reported outcomes)
The Playbook / Quick Start Guide / Scope
7
7
7
Source: Playbook team analysis
We don’t expect you to learn all of this in one go! This is a marathon. Not a sprint. This is a set of tools to use along your journey.
PRO TIP
Take your time
8
The Playbook / Quick Start Guide / How to use
8
8
Source: Playbook team analysis
Build the foundation �for digital clinical measures
Measures
What do you want to measure? Why?
Operations
What’s needed �to deploy remotely and at scale?
Technologies
What are the right tools for the job?
Then customize by context of use...
...to accelerate Clinical Research.
...to enhance Clinical Care..
...to power Public Health..
The Playbook / Quick Start Guide / Building the foundation
9
9
The Playbook / Quick Start Guide / Order matters
You can avoid common, order-related pain points if you follow sequentially the steps we outline in The Playbook!
PRO TIP
Order Matters
Source: Playbook team analysis
10
10
10
The Playbook / Quick Start Guide / Order matters
Measures
“Parkinson’s patients tell us that being able to walk independently is important, so we’re interested in measuring activity”
Following The Playbook flow, you’ll avoid order related pain points
Technologies
“Using a smartwatch with accelerometer”
Operations
“Deploying 250 smartwatches across 15 countries”
OPTIMAL FLOW
Technologies
“I saw a cool Apple watch at a Conference”
Operations
“I ordered 250 of them! Shipping to our patients.”
Measures
2 years later reviewing results: “Discard data. No clinically relevant signals yet in Parkinson’s”
COMMON ISSUE #1
Operations
“We’re deploying 250 smartwatches across 15 countries”
Technologies
“Patients are having Bluetooth connection issues; security incident. Compliance is down”
Measures
“We weren’t able to gather any usable data to construct a measure”
COMMON ISSUE #2
Source: Playbook team analysis
11
11
Source: Playbook team analysis
The Playbook builds a shared foundation for developing and deploying digital clinical measures using a step-wise approach:
The Playbook / Quick Start Guide / Order matters
Measures
Technologies
Operations
Opportunities for collaboration across industry include:
Promoting a culture of ethics to ensure equity and justice
Setting and developing standards for digital measures
Developing benchmarks to compare digital measures (e.g., algorithms)
Participating in the policy and regulatory process (e.g. public comments)
Evaluate the risk/benefit to ensure safety and efficacy (e.g., validation (V3), utility & usability, security, data rights)
Plan for the jobs to be done during deployment (e.g., purchasing, distribution, monitoring, data analysis)
12
12
Source: Playbook team analysis
The Playbook builds a shared foundation for developing and deploying digital clinical measures using a step-wise approach:
Opportunities for collaboration across industry include:
Promoting a culture of ethics to ensure equality and justice
Setting and developing standards for digital measures
Developing benchmarks to compare digital measures (e.g., algorithms)
Participating in the policy and regulatory process (e.g. public comments)
Measures
Technologies
Operations
Evaluate the risk/benefit to ensure safety and efficacy (e.g., complete validation (V3), utility & usability, security, data rights)
Plan for the jobs to be done during deployment (e.g., purchasing, distribution, monitoring, data analysis)
The Playbook / Quick Start Guide / Measures
13
13
PRO TIP
Clinical meaningfulness of the measure must always be established first
This applies regardless of whether a digital clinical measure is being used to support clinical research, clinical care or public health.
The Playbook / Quick Start Guide / Measures
Source: https://www.nature.com/articles/s41746-020-0260-4, Playbook team analysis
14
14
14
Begin by using the V3 framework to evaluate whether a digital measurement product is fit-for-purpose
Verification evaluates sample-level sensor outputs
Analytical validation evaluates the performance of an algorithm to convert sensor outputs into physiological metrics using a defined data capture protocol in a specific subject population
Clinical validation evaluates whether the physiological metric acceptably identifies, measures, or predicts a meaningful clinical, biological, physical, functional state, or experience, in the stated context of use and specified population
The Playbook / Quick Start Guide / Measures
Source: https://www.nature.com/articles/s41746-020-0260-4, Playbook team analysis
15
15
15
The Playbook builds a shared foundation for developing and deploying digital clinical measures using a step-wise approach:
The Playbook / Quick Start Guide / Technologies
Source: Playbook team analysis
Opportunities for collaboration across industry include:
Promoting a culture of ethics to ensure equality and justice
Setting and developing standards for digital measures
Developing benchmarks to compare digital measures (e.g., algorithms)
Participating in the policy and regulatory process (e.g. public comments)
Measures
Technologies
Operations
Evaluate the risk/benefit to ensure safety and efficacy (e.g., complete validation (V3), utility & usability, security, data rights)
Plan for the jobs to be done during deployment (e.g., purchasing, distribution, monitoring, data analysis)
16
16
DRAFT FOR PUBLIC COMMENT
Does the tool measure what it claims to measure? Is the measurement appropriate for the target population?
Does the manufacturer build safety by design? Is there a Disclosure Policy? Software Bill of Materials?
Who has access to the data and when? Is the privacy policy publicly accessible?
How is the tool worn? Battery life? Available technical support?
What’s the net benefit versus price? Is cost a one-time or subscription model?
V3: Verification, Analytical Validation and Clinical Validation
Security
Data Rights, Privacy, & Governance
Utility and �Usability
Economic �Feasibility
An evaluation framework for fit-for-purpose digital measurement products
The Playbook / Quick Start Guide / Technologies
Source:https://www.nature.com/articles/s41746-020-0237-3, Playbook team analysis
17
17
17
Source: Playbook team analysis
The Playbook builds a shared foundation for developing and deploying digital clinical measures using a step-wise approach:
The Playbook / Quick Start Guide / Operations
Opportunities for collaboration across industry include:
Promoting a culture of ethics to ensure equality and justice
Setting and developing standards for digital measures
Developing benchmarks to compare digital measures (e.g., algorithms)
Participating in the policy and regulatory process (e.g. public comments)
Measures
Technologies
Operations
Evaluate the risk/benefit to ensure safety and efficacy (e.g., complete validation (V3), utility & usability, security, data rights)
Plan for the jobs to be done during deployment (e.g., purchasing, distribution, monitoring, data analysis)
18
18
Source: Playbook team analysis
Four stages of operational considerations when deploying remote monitoring
The Playbook / Quick Start Guide / Operations
Post Go-Live
Close Out
1
2
3
4
Procure digital measurement products
Prepare product-level ecosystem
Go Live
‘Close out’ processes look different across research, care and public health, though exist across all contexts
Acquire access to the needed technologies
Patients start using digital measurement products
19
19
Digital measurement products are, by definition, connected to the internet. They are a type of internet of (medical) things (IoMT) product.
With IoMT, you almost never deploy and forget. These deployments are not static. Lessons from the world of traditional IoT can inform healthcare deployments.
PRO TIP
Operational considerations pull from internet of (medical) things (IoMT) concepts
The Playbook / Quick Start Guide / Operations
Source:https://www2.deloitte.com/global/en/pages/life-sciences-and-healthcare/articles/medtech-internet-of-medical-things.html, https://www.avnet.com/wps/portal/us/resources/article/3-steps-to-iot-deployment/, Playbook team analysis
20
20
20
Source: Playbook team analysis
The Playbook builds a shared foundation for developing and deploying digital clinical measures using a step-wise approach:
The Playbook / Quick Start Guide / Collaborative opportunities
Opportunities for collaboration across industry include:
Promoting a culture of ethics to ensure equity and justice
Setting and developing standards for digital measures
Developing benchmarks to compare digital measures (e.g., algorithms)
Participating in the policy and regulatory process (e.g. public comments)
Measures
Technologies
Evaluate the risk/benefit to ensure safety and efficacy (e.g., complete validation (V3), utility & usability, security, data rights)
Operations
Plan for the jobs to be done during deployment (e.g., purchasing, distribution, monitoring, data analysis)
21
21
Source: Playbook team analysis
You should evaluate whether and how the connected sensor products deployed may be received and check assumptions about the use of technology that may not be shared across the study or patient population.
PRO TIP
Be intentional about instilling fairness and equity
The Playbook / Quick Start Guide / Ethics
22
22
22
Source: Playbook team analysis
Benchmarking is essential to building confidence in digital clinical measures
Benchmarking digital clinical measures addresses a number of challenges, including:
The Playbook / Quick Start Guide / Benchmarking
23
23
What is a standard?
Digital medicine standards 101
Source: https://medium.com/digital-medicine-society-dime/standards-for-digital-medicine-1a44b743c347, Playbook team analysis
24
The Playbook / Quick Start Guide / Standards
24
24
Source: https://www.dimesociety.org/index.php/knowledge-center/federal-comment-toolkit, Playbook team analysis
PRO TIP
The federal rulemaking process in the U.S. is designed to include opportunities for public participation.
You can directly participate in the rulemaking process by submitting comments to public dockets.
The Playbook / Quick Start Guide / Policy making
25
25
25
Source: Playbook team analysis
You’ve built the foundation... �
Measures
What do you want to measure? Why?
Operations
What’s needed �to deploy remotely and at scale?
Technologies
What are the right tools for the job?
Now customize by context of use.
...to accelerate Clinical Research.
...to enhance Clinical Care..
...to power Public Health..
The Playbook / Quick Start Guide / Customize by context of use
26
26
Source: Playbook team analysis
Considerations for clinical research also follow the measures, technologies, and operations framework
Design & Feasibility
Start-up Study
Run Trial
Preparing for Regulatory Submission
The Playbook / Quick Start Guide / Customize by context of use / Clinical research
1
2
3
4
Measures
Technologies
Operations
27
27
Source: Playbook team analysis
Considerations for using remote monitoring in clinical care
The Playbook / Quick Start Guide / Customize by context of use / Clinical care
28
The Playbook touches on the following key issues to drive the adoption of safe and effective digital clinical measures in clinical care.
Addressing reimbursement challenges
Reconciling remote monitoring and clinical workflows
Optimizing responses to information generated by remote �monitoring
Developing guidelines for implementation of remote monitoring, by traditional and new healthcare organizations
28
28
Promoting ‘fit-for-purpose’ remote monitoring in public health requires attention to three areas
29
Source: Playbook team analysis
Measures are equally meaningful to all members of the population and occur with sufficient prevalence to be valuable for informing response initiatives
A technology driven approach is
Technology driven approaches to remote monitoring are operationalized to uphold principles of equity and justice
1
Appropriate
2
Suitable at scale
3
Trusted
The Playbook / Quick Start Guide / Customize by context of use / Public health
29
29
Source: Creative Commons, Playbook team analysis
How to share and adapt The Playbook in your work
Under the following terms:
30
The Playbook / Quick Start Guide / License
You are free to:
This license is acceptable for Free Cultural Works. The licensor cannot revoke these freedoms as long as you follow the license terms.
The Playbook is licenced under: �Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
30
30