1 of 36

Quantifying Post-Operative Blood Loss

Team Surgience

2 of 36

Meet Our Talented Team

Sitara Rao

Jeremy Yuan

Neil Sundaram

2

Alexandra Miekisz

Cassie Hoppenrath

Richa Bhujbal

3 of 36

Key Milestones

3

Contacted relevant physicians

POC of UI design and development

Parsed ASAC dataset for a preliminary model

A successful working prototype for vital signs plot

4 of 36

Logistic Regression Pros vs Cons

4

PROS:

Easier to implement, interpret, and very efficient to train–training a model with this algorithm does not require high computation power
Makes no assumptions about distributions of classes in feature space
Easily extend to multiple classes and a natural probabilistic view of class predictions
Not only provides a measure of how appropriate a predictor(coefficient size) is, but also its direction of association (positive or negative)--so we can use logistic regression to find out the relationship between the features
Very fast at classifying unknown records
Good accuracy for many simple data sets and it performs well when the dataset is linearly separable
Can interpret model coefficients as indicators of feature importance

�CONS:

If the number of observations is lesser than the number of features, it may lead to overfitting
It constructs linear boundaries
Major limitation is the assumption of linearity between the dependent variable and the independent variables
It can only be used to predict discrete functions
Non-linear problems can’t be solved with logistic regression because it has a linear decision surface. Linearly separable data is rarely found in real-world scenarios
Requires average or no multicollinearity between independent variables
Tough to obtain complex relationships; more powerful and compact algorithms such as Neural Networks can easily outperform this algorithm

LR: After discussing with a number of professionals 9/10 times the regression model would be preferred over any other machine learning or artificial intelligence algorithm. Most of the time you are delivering a model to a client or need to act based on the output of the model and have to speak to the why. It is relatively easy to explain a linear model, its assumptions, and why the output is what it is. Trying to do that with a neural network would be not only exhausting but extremely confusing to those not involved in the development process

5 of 36

Machine Learning Model Results

5

Log Regression

KNN

Only two outcomes in our case: Fluids (1) and no fluids (0)
Want to find the probability threshold of “Fluids” vs “No Fluids”
Output is a number between 0 and 1, which is our predicted probability that output is 1. Our threshold is where we draw the line to decide probability.

KNN assumes that similar things will exist in close proximity based on variables, in our case patients needing fluids will be similar to other patients needing fluids and vice-versa
KNN finds the “distance” between data points and classifies them as Fluids (1) or No Fluids (0)

6 of 36

Neural Network Pros vs Cons

6

PROS:

Neural networks are flexible and can be used for both regression and classification problems. Any data which can be made numeric can be used in the model, as a neural network is a mathematical model with approximation functions.
Neural networks are good to model with nonlinear data with a large number of inputs; for example, images. It is reliable in an approach of tasks involving many features. It works by splitting the problem of classification into a layered network of simpler elements.
Once trained, the predictions are pretty fast.
Neural networks can be trained with any number of inputs and layers.
Neural networks work best with more data points.

�CONS:

Neural networks are black boxes, meaning we cannot know how much each independent variable is influencing the dependent variables.
It is computationally very expensive and time consuming to train with traditional CPUs.
Neural networks depend a lot on training data. This leads to the problem of overfitting and generalization. The mode relies more on the training data and may be tuned to the data.

NN: Regression is a method dealing with linear dependencies, neural networks can deal with nonlinearities. So for data with some nonlinear dependencies, neural networks should perform better than regression.

In some applications, neural networks fits better than another model such as linear regression. And it usually occurs when there are nonlinearities involved. Though, it is important to evaluate before other aspects. For example: a linear regression model will have less parameters to estimate than a neural networks for the same set of input variables. Then, neural networks will require a larger dataset for its optimization in order to get its benefit of generalization and nonlinear mapping. So, if we do not have enough data, despite existing nonlinearities involved, a linear regression model may be better adjusted.

�

7 of 36

Machine Learning Model Results

7

Log Regression

KNN

8 of 36

Machine Learning Model Results

8

Log Regression

KNN

9 of 36

Summary & Next Steps

For next semester…

Visit Operating Room to develop a pseudo dataset from model training

Develop a working POC to prove that ML learning can be applied

Consult various surgeons on UI and interface preferences

A better way for surgeons to utilize real time hemodynamic status to minimize postoperative complications

Current alternatives of hemodynamic monitoring are lacking

ML models are accurate but are underutilized
Massimo provides supplemental data to the surgeon but is not comprehensive

Traditional methods of quantifying blood loss are inaccurate

UI Health has moved towards monitoring hemodynamic status instead of measuring blood loss

9

10 of 36

Machine Learning Model Results

10

Log Regression

KNN

11 of 36

Current Practices

11

Estimation of blood volume from visual inspection of pads, sponges, and suction containers.

Huge influence from factors such as professional experience, gender, and age. ^[10]
Inaccuracies have a median of around 30% of true blood volume. ^[11]

Visual Estimation

Indirect measurement, weighing blood-contaminated surgical material and comparing with dry material weight.

Dilution and rinsing are two main factors that lead to large inaccuracies. ^[11]
Measurement is also extremely time-consuming.

Gravimetric

Mainly used in obstetrics, use of calibrated blood collection bag during vaginal delivery

Improved accuracy when used in combination with visual estimation, but studies still show serious deviations from true blood loss. ^[11]

Direct Measurement

Estimation using mathematical formulas, takes into account hemodynamic factors and gender, age, height, etc.

Studies show that mathematical formulas show a tendency to overestimate blood loss. ^[11]

Calculated Blood Loss

Under or late transfusion: increased mortality rates in high blood loss surgeries^[1]and poorer neurological outcomes. ^[2]

Over transfusion (too much blood): higher risk of postoperative bacterial infection^[5]

12 of 36

Concept Solution: Data Collection

12

Fluid Status

Total body fluid volume

Cardiac Output

Urine Output

Serum Levels

Hemoglobin

Lactate

Albumin

Other Influences

Sex, age, BMI

Medications

Pathology

Surgery Type & Length

Blood Gas

Arterial Blood Gas

Blood level pH

Pulse Oxygen

Oxygen Saturation

Oxygen Perfusion

Blood Pressure

Hypotension may indicate blood loss

Dependent Variable: Volume of blood that a physician would administer at that moment in time

13 of 36

User Needs

13

User Need Item	User Need	Rationale
1	Device must be sanitizable	Device will be used in operating room and must be sanitized prior
2	Device must be used for multiple days at a time	Patient recovery time may vary and we want to monitor hemodynamic status continuously
3	Device must be used by medical staff	Device interface must be easy and quick to use as wasting surgical time may be expensive
4	Device must rapidly give hemodynamic status of the patient when necessary	During critical procedures constant knowledge of patient status is necessary to ensure safety
5	Device must be portable for post-operative monitoring	Device should function outside OR to help prevent post-op complications

14 of 36

Design Requirements

14

Item	Design Criteria Item	Quantifiable Requirement	Units	Rationale	Verification Method
A	2	Device must have be powered by a rechargeable battery with 9V to power Arduino	V	Device battery must be rechargeable for continued used and must be strong enough to power Arduino microcontroller	Multimeter
B	2	Device must have 8GB of memory	GB	Device must have enough memory to store and analyze patient data	Purchase of 8GB SD card
C	1	Device must meet at least an SAL of 10^-6 prior to operating room entry	SAL (sterility assurance level)	Device must be sterile prior to entering operating room	Once the number of microbes killed is determined, double the log reduction of this number and this gives you the SAL
D	4	Model must have a runtime of less than 30 seconds	seconds	Knowledge of hemodynamic status must be quickly retrievable during surgery	Time
E	1	Device must be able to connect to network	Latency	Patient data should be backed up on network	ping

15 of 36

What have we done so far?

We have conducted research on regulatory standards relevant to our device and its implementation.

15

Regulatory Standards

IEC 62304: This standard defines the lifecycle for software as a medical device (SaMD)

and

ISO 24291: This standard governs the use cases for machine learning in medicine that is used for clinical practice

Leading us to:

Why is this important?

These standards have allowed us to determine certain design requirements and processes necessary for model development

16 of 36

What have we done so far?

Our major focus so far has been finding data that allows for us to build and validate our model

16

Data Availability

The Anaesthetic Shoulder Arthroscopy Cases (ASAC) Dataset, this dataset includes patient data and time-stamped medical events and interventions including drug, vapour administration, repositioning of the patient etc for a total of 20 cases.

Leading us to:

What have we done?

So far we have worked towards parsing data from the XML format to allow us to utilize this data for model development

17 of 36

17

18 of 36

Understanding the Problem

18

Adverse outcomes from excessive perioperative bleeding and blood loss underestimation have been extensively documented.^{[1][2][3][4][5][6][7]}

Heart attack
Stroke
Renal Failure
Sepsis
Longer ICU and hospital stays
Death

Overestimation of perioperative blood loss can also be a problem. Increased fluid retention can lead to adverse outcomes post-operation as well.
Currently there is no method to accurately measure and monitor patient perioperative hemodynamic status

19 of 36

Model Performance & Comparison

19

Typical classification model used in datasets with linear data. Easy to implement, interpret, and very efficient to train.

Logistic Regression

LAUNCH

60%

Intelligent model that is capable of handling non-linear data. Can form complex interaction between variables and discover hidden patterns.

Neural Network

LAUNCH

??%

A linear model will be more interpretable while a non-linear model will be more powerful.

Outcome Focus

A NN cannot give descriptive statistics about how an independent variable affects the dependent variable. It is black box.

Future Applications

LR will perform better with simpler datasets while NN will perform better with larger / more complex datasets.

Training data

NN are more flexible and can be used for both regression and classification problems. Can also handle continuous data.

Flexibility

NN are significantly more computationally expensive when compared to LR.

Computational Power

WIP

Classification algorithm that finds the distance between the query and nearest (k) examples in data to find the most frequent label

Support Vector Machines

LAUNCH

??%

20 of 36

Post Operative Monitoring: A New Scenario

20

Fluid Estimation

Fluid resuscitation is an inexact science – which can be focused on restoring normal perfusions rather than replacing an estimated volume deficit. Visual estimation does not account for occult bleeding. [16]

Delayed Action

Visual symptoms of bleeding are a late manifestation and typically occur after the blood loss has already exceeded 15 percent of the patient’s total volume (class 1 haemorrhage). [16]

Return to Operating Room

In most cases, a return to the operating room will be necessary to control the bleeding, therefore there is a need for a device that primarily functions in the post operative setting but contains design requirements that satisfy an intraoperative space. [16]

What are some issues with today’s current practices?

21 of 36

Wards require only intermittent ”snapshots” of vital sign measurement (4-6 hours)

21

60%

50%

~30%

20%

Hypoxemia occurred in 20% of all patients recovering from non-cardiac surgery. 90% of these hypoxemic cases in which O2 saturation <90% for at least 1 hour [17]

15-45% of patients suffer from postoperative complications – with an overall mortality rate of 1-2% within a month after surgery [17]

Hypotension (<65mm Hg) was missed with 50% of postoperative monitoring routine “spot checks” when compared to continuous monitoring [17]

At least 60% of patients have one abnormal vital sign 1-4 hours before an acute cardiorespiratory arrest [17]

An inherent problem with the current periodic monitoring of vital signs in the ward is the lack of interpretation of subtle changes in vital signs or a pattern detection that may occur during early deterioration

22 of 36

"4 a.m. phenomenon"

22

Unplanned and potentially preventable ICU admissions as a results of lack of timely intervention in the absence of continuous ward monitoring [17]

Vital signs may be ”late” in the hypovolemic setting due to compensatory mechanisms (increased peripheral constriction)

Usage of current vital signs as proxy to status

Perturbations in patient status may go undetected in traditional spot checks

Invisible changes

23 of 36

What must our solution accomplish?

23

Surgience

Multi-functional Data

Data can be easily transformed to fulfill latent needs or other upcoming projects.

Uses outside of OR

Model can be scaled to catch undetected bleeds and postoperative hematological complications

Open-Source Algorithm

The final algorithm will be open to the public for additional applications, further refinements and research purposes

Important hematological factors are at the forefront for viewing by surgical teams

Portability

Interface is dynamic and can show necessary information on a smaller device that can follow the patient throughout the hospital

Interpretability

Transfusion volume as an output better helps surgeons gauge transfusion volume than a risk metric would

User Needs

Design Requirements

UI and Interface Design

24 of 36

Problem Statement

24

A continuous, smarter, and portable platform for surgical teams to monitor vital signs using machine learning-based pattern detection solutions to improve safety for post operative patients.

25 of 36

Prototype Demo

25

26 of 36

Sample Data Entry

26

INDEX	HADM_ID	AGE	EVENTS	FLUIDS
0	2400096	35	['2115-03-06 01:26:00', 'Heparin', 'Administered'], ['2115-03-06 03:14:00', 'Acetaminophen IV', 'Administered'], ['2115-03-06 03:14:00', 'Ketorolac', 'Administered'], [‘2115-03-06 05:23:00’,’EKG’]	1

27 of 36

Data Cleaning Process

27

Load data from tables into data frames.
Calculate age from anchor_year and anchor_age utilizing MIMIC formula.
Remove any null values and unneeded columns from data frames.
Transform data frames to dictionaries, with the hadm_id as the key.
Merge output events, procedure events, and emar dictionaries into events array
Utilizing the input events dataframe, we find all instances where blood was transfused and create a dictionary with hadm_id as the key. Create final dictionary.
If blood was transfused to the patient, then Fluids is set to ‘True’. Merge into final dictionary.
Write final dictionary to a CSV.

28 of 36

Log Regression Intro

28

Only two outcomes in our case: Fluids (1) and no fluids (0)
Want to find the probability threshold of “Fluids” vs “No Fluids”
Output is a number between 0 and 1, which is our predicted probability that output is 1. Our threshold is where we draw the line to decide probability.

29 of 36

Machine Learning Model Results

29

Log-loss is indicative of how close the prediction probability is to the corresponding actual/true value (0 or 1 in case of binary classification). The lower the log loss the better. A perfect model would have a log-loss score of 0, whereas ours definitely does not.

The parameter C is the penalty parameter.

A high value of C tells the model to give high weight to the training data, and a lower weight to the complexity penalty. A low value tells the model to give more weight to this complexity penalty at the expense of fitting to the training data. Basically, a high C means "Trust this training data a lot", while a low value says "This data may not be fully representative of the real world data, so if it's telling you to make a parameter really large, don't listen to it".

We tested our logistic model with 31 values of C ranging from 10^-9 to 10^6. The best value is the one that produces minimal loss in the training and validation sets, in our case that value is C=1

Best C-value for LR model: 1.000

Validation set log-loss at best C-value: 0.1323

acc for the model with the best c value = 0.9607185903442127

30 of 36

Machine Learning Model Results

30

Threshold

31 of 36

Machine Learning Model Results

31

32 of 36

ML Model Recap

32

Classification problem
Normalize the data
Perform feature engineering to enhance the model
One-hot encode the categorical variables
Remove or correct the outliers
Perform visualisations for numerical variables

33 of 36

Model Performance & Comparison

33

Typical classification model used in datasets with linear data. Easy to implement, interpret, and very efficient to train.

Logistic Regression

LAUNCH

60%

Intelligent model that is capable of handling non-linear data. Can form complex interaction between variables and discover hidden patterns.

Neural Network

LAUNCH

??%

A linear model will be more interpretable while a non-linear model will be more powerful.

Outcome Focus

A NN cannot give descriptive statistics about how an independent variable affects the dependent variable. It is black box.

Future Applications

LR will perform better with simpler datasets while NN will perform better with larger / more complex datasets.

Training data

NN are more flexible and can be used for both regression and classification problems. Can also handle continuous data.

Flexibility

NN are significantly more computationally expensive when compared to LR.

Computational Power

34 of 36

Tools for Business Value Analysis

34

$225k

$6k

Male User

Female User

List of considerations & assumptions

Model Focus

Model accuracy
Intelligence of model
Run time and computing power

Outcome Focus

Additional time in hospital (~$2k)
Reentry to ICU (~$6k)
Average cost of malpractice claim ($225k)

Neural Network

Logistic Regression

Model Accuracy

Model Intelligence

Computing Power

Interpretability

Complexity

Variable



?

35 of 36

Next Steps and Future Vision

0

With more time and processing power to comb through the datasets, we can pull out the most relevant features and figure out how to optimally represent them in our training data

Feature engineering

Refine the front end design prototype and backend modeling to include specific features to improve model accuracy.

Refine prototype

Collecting continuous data from patients would help us improve model accuracy greatly

Continuous Data Collection

Conduct additional research on available models and test our data on these model to determine which model has greatest level of accuracy

Perform more model comparisons

35

Our proposed solution is a machine learning model that allows for surgeons and other clinicians to accurately monitor and assess a patients’ perioperative and postoperative hemodynamic status.

36 of 36

References

Karkouti K, Wijeysundera D N, Yau T M et al. The independent association of massive blood loss with mortality in cardiac surgery. Transfusion. 2004; 44(10): 1453–1462.
Rajagopalan V, Chouhan R, Pandia M, et al. Effect of intraoperative blood loss on perioperative complications and neurological outcome in adult patients undergoing elective brain tumor surgery. Neuroscience Rural Practice. 2019; 10(4): 631-640.
Blajchman M A, Vamvakas E C. The continuing risk of transfusion-transmitted infections. N Engl J Med. 2006; 355(13): 1303–1305.
Triulzi D J, Vanek K, Ryan D H, Blumberg N. A clinical and immunologic study of blood transfusion and postoperative bacterial infection in spinal surgery. Transfusion. 1992; 32(06): 517–524.
Hill G E, Frawley W H, Griffith K E, Forestner J E, Minei J P. Allogeneic blood transfusion increases the risk of postoperative bacterial infection: a meta-analysis. J Trauma. 2003; 54(05): 908–914.
Palomo Sanchez JC, Jimenez C, Moreno Gonzalez E et al.Effects of intraoperative blood transfusion on postoperative complications and survival after orthotopic liver transplantation. Hepatogastroenterology. 1998;45(22):1026–1033.
Manning-Geist B L, Alimena S, Del Carmen M G et al.Infection, thrombosis, and oncologic outcome after interval debulking surgery: does perioperative blood transfusion matter. Gynecol Oncol. 2019;153(01):63–67.
Cowan T, Weaver N, Whitefield A, et al. The epidemiology of overtransfusion of red cells in trauma resuscitation patients in the context of a mature massive transfusion protocol. European Journal of Trauma and Emergency Surgery. 2021
Barr PJ, Donnelly M, Cardwell CR, Parker M, Morris K, Bailie KE. The appropriateness of red blood cell use and the extent of overtransfusion: right decision? Right amount? Transfusion. 2011; 51(8): 1684–94.
Kohlberg S, et al. Accuracy of visually estimated blood loss in surgical sponges by members of the surgical team. AANA. 2019; 87(4): 277-284
Gerdessen, L., Meybohm, P., Choorapoikayil, S. et al. Comparison of common perioperative blood loss estimation techniques: a systematic review and meta-analysis. J Clin Monit Comput. 2021; 35: 245–258
Malcherczyk D., Klasan A., Ebbinghaus A., et al. Factors affecting blood loss and blood transfusion in patients with proximal humeral fractures. Journal of Shoulder and Elbow Surgery. 2019; 28(6): e165-e174.
Sadique, Zia et al. “Cost-effectiveness of a cardiac output-guided haemodynamic therapy algorithm in high-risk patients undergoing major gastrointestinal surgery.” Perioperative medicine (London, England) vol. 4 13. 14 Dec. 2015, doi:10.1186/s13741-015-0024-x
Tengberg, L T et al. “Multidisciplinary perioperative protocol in patients undergoing acute high-risk abdominal surgery.” The British journal of surgery vol. 104,4 (2017): 463-471. doi:10.1002/bjs.10427
Hahn-Klimroth, M., Loick, P., Kim-Wanner, SZ. et al. Generation and validation of a formula to calculate hemoglobin loss on a cohort of healthy adults subjected to controlled blood loss. J Transl Med 19, 116 (2021). https://doi.org/10.1186/s12967-021-02783-9
Siparsky, Nicole. “Overview of Postoperative Fluid Therapy in Adults.” Uptodate, 14 Oct. 2014, https://www-uptodate-com.proxy.cc.uic.edu/contents/overview-of-postoperative-fluid-therapy-in-adults?search=overview+of+postoperative+fluid+&source=search_result&selectedTitle=1~150&usage_type=default&display_rank=1
TN;, Khanna AK;Ahuja S;Weller RS;Harwood. “Postoperative Ward Monitoring - Why and What Now?” Best Practice & Research. Clinical Anaesthesiology, U.S. National Library of Medicine, 23 July 2019, https://pubmed.ncbi.nlm.nih.gov/31582102/.
Peters, Philip G Jr. “Twenty years of evidence on the outcomes of malpractice claims.” Clinical orthopaedics and related research vol. 467,2 (2009): 352-7. doi:10.1007/s11999-008-0631-7
Leander, Erik. “2022 Medical Malpractice Insurance Rates: What the Data Tells Us.” Cunningham Group, 19 Nov. 2021, https://www.cunninghamgroupins.com/news/2022-medical-malpractice-insurance-rates-what-the-data-tells-us/.