Ernest Chianumba

Graduate Researcher & Master’s Student,

Data Science, Montclair State University.

Optimizing Large Language Models for ICU Readmission Prediction

A Bristol Myers Squibb Science Scholars Initiative

​

Clinical datasets often reflect biases that favor majority populations, leading to predictive models that inadequately serve underrepresented groups such as Black and Hispanic populations. This exacerbates healthcare disparities and limits equitable outcomes.

​

This project focuses on optimizing Large Language Models (LLMs) to improve predictions for ICU readmission, using clinical trial demographic data combined with medical publications. By leveraging advanced machine learning techniques, we aim to mitigate bias, enhance personalization, and promote equity in critical healthcare decisions.

​

Aligned with Bristol Myers Squibb’s mission to transform lives through science, this work contributes to reducing disparities and improving outcomes for underrepresented populations in ICU care.

INTRODUCTION

PRESENTATION OVERVIEW

START

PRESENTATION OVERVIEW

END

System Flowchart

Data Sources:

• Use ClinicalTrials.gov and PubMed APIs to link publications with the clinical trials they reference.
• Extracted abstracts from PubMed and patient records from MIMIC-IV database.
• Queried using NCT IDs, PMIDs, and participants/patients' information.

​

Focus Areas:

• Retrieved 19,707 abstracts linked to 23,370 clinical trials.
• Ensured diverse and representative data for underrepresented groups.

Core Activities:

• Removed duplicates, invalid entries and noise from the dataset.
• Handled irregular formats and missing values to ensure dataset integrity.
• Used oversampling techniques to balance datasets to ensure identical abstracts lengths for all models.

Impact:

• Ensured clean, reliable input for preprocessing and model fine-tuning and training.

Core Activities:

• Fine-tuned a neutral model using combined datasets.
• Same for demographic-specific BERT models (Blacks, Hispanics, Whites).
• Generated embeddings of drugs administered and diagnoses from MIMIC-IV to train/test Predictive models.

Impact:

• Built models trained on unique linguistic and contextual patterns of underrepresented groups.

Data Collection

​

Preprocessing

Model Training

Database Tables

PostgreSQL Database schema

Sample SQL Querying

Black – 1,105

Filtering publications that referenced clinical trials with at least 50% Black/White/Hispanic participants out of 19,707 Abstracts extracted

​

Hispanic – 579

White – 5,935

Sample SQL Querying

Filtering clinical trials with at least 25% Blacks & 25% Hispanics participants. – 11 Trials out of 23,370

BERT Models’ Token Prediction Performances

BERT Models’ Training

Abstracts count before and after Augmentation and Oversampling to ensure equal number of demographic-specific datasets for fine-tuning BERT models.

Prediction Models Used and Why:

• LightGBM - Handles large datasets efficiently, Robust to imbalanced data and Excels with mixed data types.
• MultiLayer Perceptron (MLP) - Captures complex, non-linear relationships, especially in conjunction with embeddings

Predictive Models’ Train/Test Data From MIMIC-IV

Abstracts Data From PubMed for Fine-Tuning LLMs

Training Features

Predicted Feature

Predictive Models’ Performances on Blacks-Filtered Subset

Predictive Models’ Performances on Hispanics-Filtered Subset

Predictive Models’ Performances on Whites-Filtered Subset

Future Work:

• Intersectional Analysis: Study subgroups like Black women vs. Hispanic women or Black adolescents vs. Hispanic adolescents for deeper insights, etc.
• Feature Importance: Use ablation studies or SHAP to identify key predictors of ICU readmission.
• Beyond Demographics: Integrate socioeconomic and environmental factors to improve predictions.

​

​

Limitations:

• Limited Trial Participation: Underrepresentation of minorities in clinical trials limits data for building equitable predictive models. Unwillingness to participate in clinical trials.
• Data Integration Complexity: Ensuring consistent and accurate merging of structured and unstructured datasets poses challenges.
• High Computational Needs: Training and fine-tuning advanced models require significant computational resources, affecting scalability and accessibility.

FUTURE WORK AND LIMITAIONS

Conclusion:

• Advancing Healthcare Equity: Address disparities in ICU readmission outcomes by delivering equitable predictive analytics for underserved groups.
• Real-World Clinical Impact: Provide a modular and adaptable architecture for future applications in clinical decision-making and patient-centered care.
• BMS Mission Alignment: Drive innovative use of AI to fulfill BMS's commitment to delivering impactful healthcare solutions.

CONCLUSION AND ACKNOWLEDGEMENT

Key Supporters:

• Bristol Myers Squibb, for their generous support through the Science Scholars Program.
• Prof. Hao Liu – School Mentor, Ayanbola Elegbe – BMS Mentor
• Michael Little, Pedro Smith, Yusuf Oni, Linda Obenauer-Kutner, Joseph Valente, Songyan Zheng – Interview Panel & Advisors
• MSU Data Science Lab for the datasets and database resources.
• Friends, Blessing Austin-Gabriel, Pankaj Somkuwar, Anand Gopeekrishnan, who provided support and insight throughout the project.

Ready and excited to contribute to Bristol Myers Squibb's mission to discover, develop and deliver innovative medicines that help patients prevail over serious diseases.

Thank You!