��Diabetic Retinopathy Screening Point-of-Care Artificial Intelligence -DRES-POCAI: �AI - Clinical Intervention at�San Ysidro Health����Real World AI Series�AIM-HI | HAIP Symposia ��August 8, 2024��

Edgar Diaz, MD

(Presenter)

Director of Research

Sharon Velasquez, MD

Associate Chief Medical Officer

Oliver Solis, OD

SYHC Optometrist

Sonia Tucker, MD, MBA (Presenter)

VP Population Health

Fatima Munoz, MD, MPH (Presenter)

Associate VP Health Support Services

• Diabetic Retinopathy (DR) is the leading cause of blindness among working-aged adults affecting 9.6 million people in the U.S.*
• DR prevalence rate is higher among
• Ages 55-79 (28.66%)
• Male (28.04%)
• Black, non-Hispanic (33.81%) and Hispanic (28.04%)

​

​

Percentage of 2021 US Resident Population With Diabetic Retinopathy (DR), by Stage and Age Group (CDC)

*CDC 2022, U.S. Department of Health and Human Services

• DR is mostly asymptomatic
• Routine screening is critical

for early diagnosis and timely

treatment to reduce the risk

of vision impairment, making

blindness preventable.

• Diabetes care clinical guidelines include routine eye exams – Low DR screening rate (35-60%)
• Individual level barriers (e.g., dilated eye exam, insurance, transportation)
• Structural (e.g., lack of access to eye specialist)

​

​

​

• Integrating point-of-care (POC) artificial intelligence (AI) technology for DR screenings in San Ysidro Health represents a unique and innovative opportunity to address unmet medical needs in diabetic eye care, enhancing existing our clinical practices.

​

• Federally-Qualified Health Center (FQHC) committed to providing high-quality, compassionate, and accessible health care services for the entire family.

​

​

• Originally established along the border in 1969 by Our Founding Mothers—seven women in search of medical services for their children.

​

​

• Leading safety-net provider of critical primary care services and second largest community health center network in San Diego County.

​

​

​

​

2023 Patient Profile:

• 160,000+ patients served, generating 600,000+ visits
• Clinic (in-person) visits: 419,369 / Telehealth (virtual) visits: 195,471
• 85% of patients lived at or below 200% of the Federal Poverty Level
• 29% were children (ages of 0-17 years)
• 28% were seniors (ages 55+ years)

​

Patient Care: clinical treatment for patients

• Medical
• Behavioral Health
• Dental
• Specialty (such as Acupuncture, Rheumatology and Chiropractic)
• Ancillary (Laboratory, Radiology, Home Health)

​

Patient Experience: clinical support for patients

• Case Management| Health Education
• Outreach
• Nutrition
• Social Services|Family Support Programs
• Research

Health Center Locations

• San Ysidro Health serves medically underserved minority population with:
• High rates of diabetes ~12%
• Low DR screening rates ~47.2%

Patient Profile at Intervention Sites

Aim 1 (Co-Design)

To refine and operationalize a multi-component AI clinical intervention with physicians, clinical staff, and patients introducing autonomous DR screening integration in primary care.

​

• 1-hour 7 virtual group interviews were conducted using brainwriting premortem (Gilmartin, 2019) to gather insights prior to implementing point-of-care AI-powered technology for screening diabetic retinopathy with: 
• 12 San Ysidro Health patients
• 11 Providers and clinic staff
• $50 USD gift card
• English and Spanish sessions (3/13 - 4/24/2024).

​

• Interviewer guide included:
• Possible reasons why this new screening program might fail
• Potential solutions to address those failures.
• Findings, solutions, and implementation plans from the interviews have been adopted for the research stage (R), some will be adopted following the research stage and during full SYH implementation (I), and others were deemed not necessary to address at either stage.

​

Manuscript Working title: Co-Designing an Artificial Intelligence Screening Program for Diabetes Retinopathy in a Federally Qualified Health Center

(Nicole Stadnick, PhD, MPH)

• 24 failure themes
• 62 solutions to address the perceived potential failures of the screening program.

- 45 (73%) solutions were

adopted prior to implementation

- 12 (19%) for later adoption

- 5 (8%) will not be adopted.

Concerns Addressed Now

​

1. Patient Hesitancy towards New Technology and AI
2. Language Barriers and Cultural Sensitivity
3. Staff Confusion and Role Delineation
4. No-Show Rates and Late Arrivals
5. Space and Exam Room Limitations
6. Referral Process and Follow-Up Concerns
7. Anxiety Over Receiving Negative or Complex Results
8. Provider Bottleneck from Screening Delays
9. Confusion About Comprehensive Yearly Eye Exams
10. Patient Preference for Exams by a Provider
11. Scheduling Challenges Due to Work and Transportation
12. Clinic Wi-Fi and Power Concerns

​

"They're so used to having provider engagement. You know…We saw it through the pandemic. We were able to overcome the telecommunication and the various modalities of contacting our patients but ultimately, they love provider engagement. So even though they're going to see their PCP, will they feel that loss of a specialized provider?"

​

“You’re depending on a robot, and as we can see, even with our phones, they can hack you, you know?”

​

"Does this consist of having a regular exam, and this is another exam on top of that?”

A total of 62 solutions were suggested by interviewees to address their perceived potential failures of the screening program.

"Artificial intelligence is a … label that's out in the communities. But most of our communities are kind of a little skeptical about it.“

"Artificial intelligence is a … label that's out in the communities. But most of our communities are kind of a little skeptical about it.“

"A lot of patients here are elderly patients and they [do] not really like AI … they're gonna be using a new technology, a new machine and everything, and it's like “What are you gonna do [to] me? I already have [an] eye doctor I don't want to go to."

1. Patient Hesitancy towards New Technology and AI 

2. Patient Preference for Exams by a Provider

​

​

​

​

​

3. Power Failure

"They're so used to having provider engagement. You know…We saw it through the pandemic. We were able to overcome the telecommunication and the various modalities of contacting our patients but ultimately, they love provider engagement. So even though they're going to see their PCP, will they feel that loss of a specialized provider?"

“If it did go out [or] if the lights were out, it would[n’t] do anything"

Concerns Not Addressed Now

​

1. Lack of Clinical Decision-Making from the EyeArt Machine

• Concerns: Lack of clinical decision-making.
• Reason for not addressing: EyeArt is only designed to provide analysis on presence of DR that must be read and acted upon by the patient’s primary care physician or eye specialist.

2. Access to Retinal Specialists for Uninsured Patients

• Concerns: Difficulties accessing follow-up specialty care.
• Reason for not addressing: Current study patients will have the insurance necessary to receive the appropriate care.
• Potential future solutions: Leverage community programs like Project Access, explore partnerships with retinal specialists, seek grants or funding for uninsured patients, Educate patients on insurance and available resources.

DRES-POCAI seeks to identify and integrate clinical best practices for diabetes care by enhancing the delivery of services and maximizing the use of technology within the primary care setting to support diagnostic-decision making via AI technology

​

“The right care, at the right time, at the right place”

EyeArt sensitivity (safety) superior to that of dilated eye examinations

Bhaskaranand M, Ramachandra C, Bhat S, et al. The Value of Automated Diabetic Retinopathy Screening with the EyeArt System: A Study of More Than 100,000 Consecutive Encounters from People with Diabetes. Diabetes Technol Ther. 2019;21(11):635-643. doi:10.1089/dia.2019.0164

Lee AY, Yanagihara RT, Lee CS, et al. Multicenter, Head-to-Head, Real-World Validation Study of Seven Automated Artificial Intelligence Diabetic Retinopathy Screening Systems. Diabetes Care. 2021;44(5):1168-1175. doi:10.2337/dc20-1877

Heydon P, Egan C, Bolter L, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol. 2021;105(5):723-728. doi:10.1136/bjophthalmol-2020-316594

* with enrichment adjustment

^ⱡAll study endpoints met with p <0.0001

Ipp, Eli, David Liljenquist, Bruce Bode, Viral N. Shah et al. “Pivotal Evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-Threatening Diabetic Retinopathy.” JAMA Network Open 2021; 4 (11): e2134254.

​

Lim, Jennifer Irene, Carl D. Regillo, SriniVas R. Sadda, Eli Ipp et al. “Artificial Intelligence Detection of Diabetic Retinopathy: Subgroup Comparison of the EyeArt System with Ophthalmologists’ Dilated Examinations.” Ophthalmology Science 2023;

​

Pivotal prospective clinical trial with 942 participants at 15 US centers including primary care.

EyeArt performance evaluated against rigorous ETDRS reference standard from Wisconsin Reading Center.

Diabetic Retinopathy Screening Point-of-Care Artificial Intelligence

Integration of new POC Autonomous DR Screening (EyeArt AI system) into primary care workflow

2. Integration of EyeArt AI and SYH-EHR Systems (Epic & Eyenuk System). Results available in patient’s chart and referral can be placed

3. Patient Education and Engagement. Evidence-based strategies to promote diabetes self-management.

AI at the Point of Care: 

• EyeArt Actionable Report in Less Than 30 Seconds
• Patients Can Receive Validated Results While On-site
• Minimize Risk Of Delay In Patient Follow-up/Treatment

Auto referral for patients based on AI results:

• vtDR positive: referred to a retina specialist (with preferential scheduling, if possible)
• mtmDR positive: referred to a retina specialist
• Ungradable: referred to an optometrist for a dilated eye exam (usual care)

1. DR Screening & Equipment

2) EHR Integration – Pre-screening process

3) Screening Process

The "View Image" hyperlink will open the attached result PDF in a popup window. 

• The authorizing provider will receive a result InBasket message to review the results. 

• In Chart Review

4) EHR Integration – DR Screening Results

Sample Report

5) EHR Integration – Automated referral process

DRES-POCAI Intervention

​

To evaluate the implementation and effectiveness of a multicomponent AI clinical intervention using a patient-level randomized controlled Clinical Trial.

Aim 2: To evaluate the implementation and effectiveness of a multicomponent AI clinical intervention using a patient-level randomized controlled trial (RCT).

H2.1 greater screening rates of DR (10% increase).

H2.3 greater (a) knowledge, (b) attitudes, (c) self-efficacy and (d) patient satisfaction DRES-POCAI.

H2.2 (a) ↑ # complete referrals to the eye specialist, (b) ↑ established early stages Dx of DR, and (c) ↓ DR complications due to early treatment.

Pre-Randomization

Intervention

Randomization

​

Intervention Group

(n=424)

​

Usual Care Group

(n=424)

1. Eligibility

​

​

• Screening and recruitment

​

​

• Consent Process

​

​

• Baseline survey

​

​

​

Intervention Components:

1. POC-AI DR screening during medical visit

​

• Immediate DR screening results available in EHR and PCP provides patient care plan (repeat in 12 months or referral to eye specialist)

​

• Patient education and care management

Usual Care:

1. Visit with PCP and referral for DR with eye specialist
2. Patient education and care management

Data Collection

Extracted from EPIC (3 months post enrollment):

1. Demographic/clinical data
2. POC-AI results/completed screening
3. Eye specialist referrals for DR diagnosis
4. DR diagnosis

Extracted from EPIC (3 months post-enrollment):

1. Demographic/clinical data
2. DR completed screening
3. DR diagnosis

Inclusion criteria:

​

1. Established and active patients of SYH-CV and KC (having had a medical appointment in the last 18 months);
2. 22 years of age or older;
3. Medical appointment scheduled during the intervention period.
4. have not had a retinal exam in the last 11 months

​

Exclusion criteria:

1. Have a prior diagnosis of DR, macular edema, or retinal vascular occlusion;
2. Have persistent visual Impairment in one or both eyes;
3. History of ocular injections, laser treatment of the retina, or intraocular surgery (excluding cataract surgery);
4. Diagnosis of mental or neurodegenerative disease that prevents self-consent
5. Pregnancy at the time of enrollment

Dynamic Eligibility Criteria

• Not having a DRS in the last 11 months
• Patients with a PCP appointment on the next two weeks

​

Dynamic Eligibility Criteria

• Not having a DRS in the last 11 months
• Patients with a PCP appointment on the next two weeks

​

422 Patients with appointments in the next 2 weeks,

F=259 (61.37%), M=163 (38.63%)

White= 70 ( 16.59%), BAF=20 (4.74%); His=335 (79.38%)

English or Spanish=419 (99.29%)

​

360 Due for Diabetic Retinopathy Screening (only 90 with last date)

230 In-Person visits in the next 2 weeks

F=140 (60.87%), M=90 (39.13%);

White= 39 (16.96%), BAF=13 (5.65%); His=178 (77.39%)

English or Spanish=230 (100%)

• In the last 2 weeks, our staff randomized 1 person into the Diabetic Retinopathy (DR) Artificial Intelligence (AI) screening (intervention) group and 1 person into the Retinal Screening Usual Care (UC) (control)

​

• The participant in the Intervention group had an Ungradable result on both eyes after 3 attempts, and our team confirmed that a referral to an eye specialist was created after their medical visit

​

• The participant in the Usual Care group was advised to schedule a Retinal Screening with an Eye specialist, and our staff confirmed that the participant had a scheduled appointment to meet with the specialist

​

• Primary effectiveness outcome — DR screening completion

​

• Secondary effectiveness outcome — DR stages diagnosis

​

• Exploratory analysis — identify knowledge, attitudes, and self-efficacy factors associated with DR screening uptake.

​

• Reach (% of patients who participate and their characteristics) and Implementation (consistency of implementation across clinics) documented using the RE-AIM Model Dimension Checklist self-efficacy

SYH EHR - Eyenuk Integration - validation

​

7/29/2024

• Evidence-based strategies adapted for the clinical environment.

​

• Alignment to Patient Centered Medical Home (PCMH): enhancement of current patient care clinical workflows, quality improvement, and meaningful use of EHR.

​

• Potential cost savings from reducing unnecessary visits to eye specialists and early diagnosis will reduce patients' lifetime treatment costs.

​

• Findings from SYH’s research intervention will help identify best practices for scalability and capacity building, such as data privacy, algorithm accuracy, and ongoing evaluation to optimize the impact of AI/ML technology for diagnostic decision-making in real-world settings.

THANK YOU!