Digital Medicine: Clinical Validity, Human Factors and Ethical Research
Andy Coravos
CEO, Elektra Labs
Sage Bionetworks: Mobile Health App Developer Workshop
September 12, 2019
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
class Career
def current_experience
CEO/co-founder @ Elektra Labs
Member, Harvard-MIT Center for Regulatory Sciences
Board of Directors, Digital Medicine Society (DiMe)- 501c3
end
def past_experience
Entrepreneur in Residence, US Food and Drug Administration (FDA)
Software Engineer @ Akili Interactive
Healthcare Private Equity @ KKR
Management Consultant @ McKinsey & Company
end
end
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
My sister Chrissy and me, showing off a difficult surgery.
Notice the three (!) carefully-wrapped casts and head bandage.
Today’s lightning talk
Overview to digital medicine �A look at the wearables, ingestibles and biosensors that are collecting digital specimen data
Deep Dives: Eat Your Greens�Security, ethical, legal, and regulatory considerations when using digital specimen data
An Evaluation Framework for Connected Technologies�Do we need a “nutrition label” for tech?
Elektra Labs | September 2019 | @elektralabs
Before we start,
a few definitions.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
New to digital medicine?
We published a book (with cartoons!)
Download a copy: https://www.karger.com/Article/Abstract/500413
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
There are four types of Clinical Outcome Assessments (COAs)
COAs can become digitized to collect data at home. When referring to digital COAs, it’s traditional to include an “e” → eCOA, ePRO
But what happens when we use a mobile or connected technology? Are there ‘mobileOAs’?
Source: WIRED November 2018, https://www.wired.com/story/prescription-video-games-and-vr-rehab/
In Nov 2018, WIRED published an op-ed based on the digital medicine framework.
Contained a landscape analysis of software and algorithms that:
… and a perspective on how to bring these products to market safely, effectively and ethically.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Measure
Diagnose
Treat
With sensors + algorithms to create objective measurements
With advanced algorithms to support the clinician
With novel software- based therapies that may augment or substitute a drug
Software and algorithms have a wide range of applications
E.g., Digital biomarkers, clinical decision support
Digital diagnostics
Digital therapeutics
To develop these products, we’ll need to build safe and clinically-validated algorithms.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Digital biomarkers are consumer-generated physiological and behavioral measures collected through connected digital tools
Source: Rock Health, 2015 (https://rockhealth.com/reports/the-emerging-influence-of-digital-biomarkers-on-healthcare/)
Source: Carlos Rodarte, Karger Digital Biomarkers, September 2017. (https://www.karger.com/Article/FullText/479951)
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Source: npj Digital Medicine, March 2019 https://www.nature.com/articles/s41746-019-0090-4
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
A glossary of terminology and uses of biomarkers and endpoints in biomedical research, medical product development and clinical care
Although not explicitly listed in the BEST framework, a “digital biomarker” is a biomarker collected through digital means, often used in a remote (at-home) setting
Source: FDA-NIH BEST Framework, https://www.ncbi.nlm.nih.gov/books/NBK326791/
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Measure
Diagnose
Treat
With sensors + algorithms to create objective measurements
With advanced algorithms to support the clinician
With novel software- based therapies that may augment or substitute a drug
Software and algorithms have a wide range of applications
E.g., Digital biomarkers, clinical decision support
Digital diagnostics
Digital therapeutics
To develop these products, we’ll need to build safe and clinically-validated algorithms.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Source: Coravos A, Khozin S, Mandl KD. Developing and adopting safe and effective digital biomarkers to improve patient outcomes. NPJ Digit Med. 2019;2(1), https://www.nature.com/articles/s41746-019-0090-4
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Philips Pagewriter Touch Interpretive EKG Machine: $15k
In 2014, AliveCor brought the EKG home...
Meet Kardia Mobile. Your personal EKG: $99. FDA-Cleared.
Take a medical-grade EKG in just 30 seconds. Results are delivered right to your smartphone.
… and since then, the FDA has cleared multiple “software-as -a-medical-device” (SaMDs)
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Measure
Diagnose
Treat
With sensors + algorithms to create objective measurements
With advanced algorithms to support the clinician
With novel software- based therapies that may augment or substitute a drug
Software and algorithms have a wide range of applications
E.g., Digital biomarkers, clinical decision support
Digital diagnostics
Digital therapeutics
To develop these products, we’ll need to build safe and clinically-validated algorithms.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Four Years Later...
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
So, what can these
tools enable?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Mobile technologies are enabling new clinical investigation designs like Decentralized Clinical Trials (DCTs)
Source: Khozin S, Coravos A. Decentralized Trials in the Age of Real-World Evidence and Inclusivity in Clinical Investigations. Clin Pharmacol Ther. 2019; https://ascpt.onlinelibrary.wiley.com/doi/full/10.1002/cpt.1441
Regulatory Considerations
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Digital tools are not making it easy to adhere to historical distinctions between the intervention and measurement/endpoint collection
Contains Software
that collects a digital biomarker
Parkinson’s Medication
Collects a measurement
Smartphone-based tapping test
with a responsive intervention (e.g., variable dosing)
Insulin Pump
Akili Interactive Labs Project:EVO for ADHD
The TOVA test (e.g., change in Attention Performance Index)
Software’s Purpose
Clinical Trial Example
Intervention
Endpoint data collected by
Continuous Glucose Monitor (CGM)
Alters the treatment / intervention
Is the treatment / intervention
Trial protocol ...
with a digital therapeutic
Rise of digital therapeutics increases available treatment options for physicians
An insulin pump with software that responds/doses based on the CGM reading
Opportunity for a clinician to send a patient home to behavioral and psychological measures remotely
Digital tools are blurring the line between measuring, diagnosing, and intervening
How does the US ensure that the products brought to market are safe and effective?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
US Regulatory Agencies
Different but complementary authorities
Source: Mobile Health: Industry Overview and Evolving Regulatory Framework, CERSI, https://cersi.stanford.edu/cersi_mhealth_course
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
US Food and Drug Administration (FDA)
Source: Mobile Health: Industry Overview and Evolving Regulatory Framework, CERSI, https://cersi.stanford.edu/cersi_mhealth_course
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Federal Trade Commission (FTC)
Mission
Source: Mobile Health: Industry Overview and Evolving Regulatory Framework, CERSI, https://cersi.stanford.edu/cersi_mhealth_course
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Source: Mobile Health: Industry Overview and Evolving Regulatory Framework, CERSI, https://cersi.stanford.edu/cersi_mhealth_course
Both the FTC and FDA oversight is focused on consumer protection
oversee promotion & advertising
oversee promotion & advertising with a public health perspective
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
A “device” is a
Term of Art
at the FDA
(Try to minimize using the term “device”
unless the product is actually a device.)
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Is my product a “device”?
Talk with your regulator and lawyer!
The next example is metaphorical rather than factual.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Device?
Not a device?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Device?
Not a device?
Trick question.
It’s all about what the manufacturer claims the product can do.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
The exact same product can be developed and marketed either as a “device” (and thus, regulated) or not as a “device” (and unregulated) simply through a change of words, and no change in hardware or code.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
There are many agencies that may oversee health tech products, and there are also many gaps in the current regulatory system.
Oversees human subjects testing, though many healthy-lifestyle devices fall out of agency’s purview (not a “device”)
Oversees connectivity and net neutrality (e.g., regulating access to the internet)
Police unfair and deceptive practices; main enforcement for security and privacy - small agency
Only recently started proposed rulemaking for Internet of things
Oversees information that’s used in background testing and other social evaluations
Consumer Product Safety Commission
Source: The ‘Internet of Bodies’ Is Here. Are Courts and Regulators Ready? (WSJ, Nov 2018, Andrea M. Matwyshyn) https://www.wsj.com/articles/the-internet-of-bodies-is-here-are-courts-and-regulators-ready-1542039566
Ok. Enough regulatory speak.
What do I need to know if a digital tool is “fit for purpose”?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Cybersecurity considerations
Learn more about the FDA-led initiative at WeHeartHackers.org
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
All three of these people are in the Workshop today.
+3 bonus points if you can find the other two.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Source: https://www.jmir.org/2019/3/e12568/
Our healthcare system has strong protections for patients' biospecimens, like blood or genomic data, but what about our digital specimens?
Your anonymized data is not.
[2] https://www.nytimes.com/2018/10/23/business/media/fever-advertisements-medicine-clorox.html
[4] https://hbr.com/2015/02/theres-no-such-thing-as-anonymous-data
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Need to consider unintended consequences when algorithms are trained for some populations… and underperform on others
{Data rights and
governance}
>
{Data ownership}
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Can we develop a new “informed consent” for how we handle interact with these digital platforms?
What “social contract” are we agreeing to when you have my data / digital specimen?
Will our technological revolution bring us personalized medicine?
...or the biggest surveillance system we’ve ever had to deal with?
Most likely both.
How can YOU participate in the US rulemaking process?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Serve a “Tour of Duty”
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Submit a comment to the public docket
Comments for the FDA AI/ML white paper are due June 3, 2019!
Whenever an agency is proposing either brand-new regulations or changes to existing ones, they must do it in two phases.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Reasons to submit a public comment
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Kick around ideas with colleagues to improve the regulatory paradigm. Our society needs new models.
For example, co-authored this op-ed with Irene Chen.
Using ‘clinical trials’ frameworks to teach us about AI and algorithm development:
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Clinicians have professional societies to support their development, e.g., ASCO.
What exists for those who practice and develop digital medicine products?
Members from government agencies have teamed up with software engineers, security researchers and more to launch...
Learn more about the 501(c)3 Digital Medicine (DiME) Society at DiMeSociety.org.
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
The Digital Medicine Society (DiMe) launched in March 2019
How can I know if these tools are trustworthy?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
Evaluation Philosophies
We use Duke-FDA CTTI “Mobile Technologies” evaluation framework as the backbone of the analysis
Novel Endpoints, Launched June 2017
Mobile Technologies, Launched July 2018
[1] ttps://www.ctti-clinicaltrials.org/projects/novel-endpoints
[2] https://www.ctti-clinicaltrials.org/projects/mobile-technologies
Digital Measures Atlas Overview
Browse fit-for-purpose connected technologies for remote capture of digital biomarkers in research and clinical care
Choose from 900+ measures
Choose from 40+ therapeutic areas
Compare 690+ consumer and research grade products made by 350+ organizations
Ready to join our community?
Sign-up to create your custom reports and subscribe to the Atlas.
The Atlas employs a “fit-for-purpose” approach
to evaluate connected technologies
VERIFICATION AND VALIDATION (V&V)
HUMAN FACTORS AND USABILITY�
SECURITY PRACTICES�
DATA RIGHTS AND GOVERNANCE�
Who has access to the data and when? Third-party transfer? End user license agreement analysis.
Follow clear security practices for connected tools?
What’s the look and feel of the product? Battery life? Waterproof? Adherence?
Are the data trustworthy and meaningful?
A deep dive on these four components is available in the Appendix.
The Atlas employs a “fit-for-purpose” approach to
evaluate connected technologies
VERIFICATION AND VALIDATION (V&V)
HUMAN FACTORS AND USABILITY�
SECURITY PRACTICES�
DATA RIGHTS AND GOVERNANCE�
Who has access to the data and when? Third-party transfer? End user license agreement analysis.
Follow clear security practices for connected tools?
What’s the look and feel of the product? Battery life? Waterproof? Adherence?
Are the data trustworthy and meaningful?
A deep dive on these four components is available in the Appendix.
A
Think of it like a nutrition label, balancing four components based on needs in a clinical trial.
Over past 12 months, our team has developed research around these four concepts -- and published with industry-leading collaborators
VERIFICATION AND VALIDATION (V&V)�FDA and Boston Children's co-authors; Nature npj Digital Medicine�https://www.nature.com/articles/s41746-019-0090-4
HUMAN FACTORS AND USABILITY�Former FDA co-author, Clinical and Translational Science (CTS)�https://ascpt.onlinelibrary.wiley.com/doi/full/10.1111/cts.12602
SECURITY PRACTICES�DEF CON co-authors; Journal of Medical Internet Research (JMIR)�https://www.jmir.org/2019/3/e12568/
DATA RIGHTS AND GOVERNANCE�Duke Forge co-author, Lancet Digital Health
https://www.thelancet.com/journals/landig/article/PIIS2589-7500(19)30001-9/fulltext
Publication pending for April 29, 2019
Questions?
Sage Bionetworks: Mobile Health App Developer Workshop | September 2019 | @AndreaCoravos | @elektralabs
We’re hitting an inflection point in digital medicine. Now’s the time to shape the healthcare community.
Let’s build an intentional future that we want, and not an accidental one.
Want to continue the conversation? Shoot me your questions via Twitter @andreacoravos