�A Machine Learning Framework for Predicting Frequent Emergency Department Users �Using Claims Data

Summer (Xia) Hu

Margret Bjarnadottir

Sean Barnes

Bruce Golden

University of Maryland, College Park

​

1

POMS Conference

May 06, 2016, Orlando, Florida

Background: Frequent Emergency Department (ED) Usage�

2

• Higher risks
• Prolonged wait times
• Higher abandon rates
• Higher rates of dissatisfaction

• Higher rates of medical errors
• Lower productivity and morale
• Reduced ability responding to mass casualty incidents

Negative Effects – ED Overcrowding

• Much higher payment

(compared with regular

PC visits)

High ED Usage put stress on the

ED system as well as the payer!

Background: Frequent ED Users & ED Jumpers

3

• Frequent ED users constitute 21% of the member population
• Yet they account for 78% of ED visits

Frequent ED Users

(members with ≥ 4 ED

visits in a single year)

• Types of ED Users

Non-frequent ED Users

(members with < 4 ED

visits in a single year)

• Facts from Claims Data

ED Jumpers

Year 1 Year 2

Objective

4

• Predict potential frequent ED users and ED Jumpers

based on their claims records from the previous year.

​

• Identify characteristics of frequent users and Jumpers.

​

5

Method Overview

• User Segmentation

User segmentation via Clustering

• Prediction – Frequent ED Users and ED Jumpers

• Descriptive Analysis

Data & Preprocessing

• Raw Data Summary

​

​

​

​

​

• Data Processing:

​

​

​

​

​

6

​

Eligible Enrollment

(≥350 days of enrollment for

2 consecutive years)

​

Patient-based Yearly Profiles

​

Information

Uniqueness

(unique gender? Birth year?)

​

​

Feature Extraction

​

​

Claim Aggregation

​

Med

Pharmacy

Lab

MH

Dental

• Five datasets
• Four years (Jan 09 – Dec. 12)

– Transform claim-based records to patient-based yearly profiles

Claim-based Raw Data

• Result in 164,402 member files, 439 features per year

7

Final Data

• Feature Overview

• Number of Result Year’s ED Visits

• On average most members have very few ED visits.

Descriptive Analysis:�Influence of Number of ED Visits from Observation Year ��

• Linear trend between the number of ED visits in two consecutive years

• Number of ED Visits from Observation year, as expected, explains Result Year’s ED visits the best

Among frequent ED users, 43% of them stay as frequent ED users in the Result Year

9

Predict Frequent ED Users: Supervised Machine Learning

• Data Setup

• Unbalanced Binary Classification Problem: (21%) (79%)
• Six machine learning algorithms:

– Logistic regression (with or without regularization), Naïve Bayes, Decision Tree (CART),

Boosted Tree (C5.0), Random Forest.

​

​

​

Data

Frequent/ Non-frequent ED User

in 2^nd Year?

439 Features from 1^st Year for each eligible member

Predict

X

Y

Training Set (70%)

Validation Set (15%)

Test Set

(15%)

Data Partition

• Supervised Machine Learning

10

Predict Frequent ED Users

Best Machine Learning Model (C5.0) v.s. Baseline Model

Models:

• Baseline: Use the number of ED visits from the observation year as estimation for next year’s number of ED visit
• Six machine learning models

Performance Metric:

• Detection Accuracy rates

– percentage of correct predictions of 2^nd Year top ED users selected by each model on unseen validation set

​

• The best machine learning model improved detection accuracy of Frequent ED Users by 7.5%

11

Top Influential Features from Machine Learning

in Predicting Frequent ED Users

12

Predict ED Jumpers

• ED Jumpers

• Prediction

• The best machine learning model improved detection accuracy of frequent ED Users by 53%

Machine Learning Models:

LDA, MDA, QDA, FDA, RDA

Baseline Model: Use

1) # of ED claims

2) # of outpatient claims from the 1^st year

as estimation for next year’s number of ED visit.

13

ED Jumpers: User Segmentation via Clustering

• Clustering among ED Jumpers
• Mixture model clustering (based on BIC) on Training set

​

Summary of ED Jumper Clusters

14

Feature Summary of ED Jumpers: User Segmentation via Clustering

Cluster 1

Cluster 6

Future Work

​

• Two-year Analysis (Current Focus):
• Analyze thresholds that define frequent ED users

– How do the coefficients and performance change as a function of the thresholds?

​

• Explore the relationship of mental health utilization on ED utilization and other outcome variables

​

• Multi-year Analysis (Next Step):
• Create multi-year models

– use multiple years of claims history to predict future ED usage levels

​

• Analyze the influence of the allowable gap on the prediction performance

– How accurately can we predict ED performance with only partial-year enrollment?

​

• Explore the “stickiness” of frequent ED use

– How consistent are frequent ED users from one year to the next?

15

​

​

​

​

​

​

​

​

​

​

​

​

Summer (Xia) Hu

University of Maryland, College Park

xhu64@umd.edu

16

Questions and Comments?

​

Appendix

17

i) General diagnosis category

Each ED visit

iii) Clinical Classifications Software (CCS)

ii) NYU ED Visit

Severity Algorithm

Three grouping algorithms

Preventable

ED Visits

Non-preventable

ED Visits

Based on historical

probability distribution

Others

Visit types

• Non-Emergent
• Primary care treatable
• ED needed yet preventable

• MH-related
• Alcohol-related
• Substance abuse-related
• Injury
• Unclassified

– Classify reasons for each ED visit by three grouping algorithms

Example of feature extraction

Appendix

Influence of Age on Frequent ED Users

Appendix

Influence of Age on Super ED Users

Conclusions:

20

Appendix

Machine Learning Result – Frequent ED User

Appendix

Key Characteristics

• Machine learning algorithms did not perform very well in detecting jumpers

22

Appendix

ED Jumpers

23

Appendix

ED Jumpers (continues)

24

Feature Summary of ED Jumpers: User Segmentation via Clustering