HSMA6 Project

Improving Ambulance Care: Fast Feedback of Quality Care Indicators

​

James Wise – Senior Management Information Analyst

Phil King - Management Information Analyst

​

Introductions

• James –
• Responsible for management information and reporting for SCAS Commercial services inc. Patient Transport, HR and Finance
• Enthusiasm for improvement led to Python experimentation, testing off-shelf machine learning tools, and running MSc student projects
• Phil –
• Avoided further education and embarked on 20-year career working with clinical data at South Central Ambulance Service
• Daily user of SQL to extract and analyse large datasets from multiple data sources, resulting in the creation of web-based visualisations using Qlik and adhoc spreadsheets
• Involved in the design of electronic patient record systems, at the data and presentation layers (and a little bit of the middle)
• Prior to HSMA, only executed one Python script, successfully. Ever.

The Question

To support clinical auditors of Ambulance incidents, can we make an AI/ML tool to quickly analyse the large amounts of free text information recorded in Clinician notes?

​

The Problem

​

• Ambulance clinicians are recording information on the electronic patient record (ePR) in free text instead of using the available buttons.
• The current data extraction method (SQL) is limited in scope on how it analyses the free text. It can look for phrases but not positive or negative statements or combinations of strings. It is heavily reliant on buttons to capture this information.
• Each month the Clinical Auditor will manually review ~500 records for the Out of Hospital Cardiac Arrest Outcome (OHCAO, or ‘Och-ah’) dataset. The purpose is to increase the accuracy of the data ahead of submission to NHSE. Each quarter this includes the ‘Care Bundle’ which is a collection of 6 measures:

​

​

• The Clinical Auditor is a single point of failure; it can take up to 3 days to complete a review; and has limited documentation to support a wider review base.

​

Method – Data Extraction

• Extracted 2,533 (6 months) of OHCAO data which has already been reviewed and confirmed as cardiac arrests. It focuses on two care bundle elements known to be captured regularly in the free text:
• 12 lead ECG
• Oxygen Administration
• Only two of the months will have had the Care Bundle information reviewed, meaning they should be more accurate than those which were not
• For each record, 8 free text fields were extracted in full
• Test our results to show if any improvements could be made over human-reviewed data

Why and how?

• Why?
• We want to use SCAS’ rich dataset with Machine Learning and AI to greatly improve our service to patients and our colleagues
• Many uses for natural language data;
• Improve clinical auditing & staff training
• Save call-centre staff time filling in personal details
• Mine staff feedback for themes and suggestions
• Scan social media for good and bad sentiment
• Etc…
• Not replacing colleagues; enabling them
• Securely, ethically processed, in-house
• Open-source tools
• HSMA gave us the foundations to build on

​

​

​

The approach

• Write code to match patterns in free-text; 12-lead, Oxygen using “Named Entity Recognition”
• Pull out the sentence where the pattern was found
• Work out if the thing was done or not, e.g. “No Oxygen given”, and label the data
• Use this data to train a “Neural Network” model to do an even better job

The cahlleng3s of cl!nic@l txt

• Why write “12 lead” when you can write “twelve lead”, or “12-led”?
• “I’m hungry! Let’s eat Grandma.”
• Negation is more common, and harder to identify, than you think
• Not trivial problems to solve – was hard to make an accurate data set to train a model with

​

Project outcome

• After a lot of challenges – delivered a working way to search real clinical notes for phrases and whether they happened or not
• > 55,000 records scanned in <30min
• 19,186 records with 12-lead button press - 6,829 extra occurrences in free-text (36% increase)
• Something we can build on
• Improve our training data
• “Manufacture” clean data
• “Tune” our model / custom pipeline
• Eventually, “Personalise” a Large Language Model, like Chat-GPT, on SCAS data

Next steps

• Demonstrate to the Clinical Directorate for feedback and comments
• Highlight the pros and cons of using such a method in relation to automating reviews on free text
• Suggest upscaling to other phrases within the OHCAO care bundle with support of Clinical Auditor
• Fine tune the model over time and perform continuous improvements
• Scope the possibility of looking at the other Indicators (STEMI, Stroke, Falls) as well as adhoc internal measures as set out in the Trusts Clinical Audit Plan

Impact of HSMA on SCAS, and us

• SCAS
• Influenced by HSMAs to invest in AI hardware
• Specifically with our project in mind
• Independent production, Secure data
• Trickle-down effect to colleagues
• Links – e.g. NWAS boundaries project
• James
• Community – project/NHS Hackathon
• Applied skills in so many projects already
• Fun ☺
• Phil
• Greater understanding of data science applications and skills into the real world
• Increased confidence in my own ability to work with data
• Future proofed my career development

​