Discussion about:

Mitigating selection bias in

organ allocation models

​

Led by Marina Mautner Wizentier

Erin M. Schnellinger, Edward Cantu III, Michael O. Harhay, Douglas E. Schaubel, Stephen E. Kimmel and Alisa J. Stephens-Shields

Background

Organ Allocation Models

• The Organ Procurement and Transplantation Network (OPTN) allocates deceased-donor organs in the U.S.

​

• The OPTN utilizes separate policies to govern the allocation of livers, kidneys, hearts, and lungs.

​

​

​

Lung Allocation Score (LAS)

• Lung transplantation is a highly effective treatment. However, since organs are scarce, wait-list mortality is high.

​

• Concerns about inequities in wait-list mortality led the DHHS to mandate the development of the LAS based on medical need rather than wait time.

​

• LAS is calculated using the predicted difference between transplant benefit and waitlist urgency, with transplant benefit defined as one-year post-transplant survival minus one-year waitlist survival, and waitlist urgency defined as one-year waitlist survival.

​

• The LAS aims to determine the number of days of life a person would gain over the next year if they receive transplant compared to if they do not receive transplant.

​

​

​

LAS score and selection bias

• Pre-transplant survival
• Dependent censoring: patients can be removed from the waiting list prior to 1 year of follow-up due to receipt of transplant, loss to follow-up, or other clinical reasons (e.g., the inability to withstand the transplant surgery). In each of these cases, the patients’ true one-year pre-transplant survival is unobserved.

​

• Post-transplant survival
• Survivor bias: Post-transplant survival models are applied to all wait-listed patients but are fitted using only transplanted patients. The post-transplant survival estimate excludes individuals who:
1. Did not survive on the waitlist long enough for a suitable donor organ to become available.
2. Did not have sufficient priority to actually receive the organ.

​

​

​

​

Motivation

• Selection bias can lead to inaccurate predictions for both the pre-transplant and post-transplant prediction models. Biased estimates of pre- and post-transplant survival in turn imply that the current prioritization of lung transplant recipients may be inaccurate.

​

• Although prior research has incorporated weights in the pre-transplant survival model, the models did not capture important geographic differences in patient selection and survival.

​

• No work, to their knowledge, had estimated weights to the post-transplant survival

​

• They developed a modified LAS using inverse probability weighting to improve the accuracy of the LAS by accounting for selection bias in the pre- and post-transplant survival models.
• Their work incorporates additional factors in the pre-transplant weights to better address selection bias in the pre-transplant survival model.
• They also developed new weights to address selection bias in the post-transplant survival model.

​

​

​

Methods

Data

• Publicly available pre- and post-lung transplant data from the United Network for Organ Sharing (UNOS). Inclusion criteria: Patients 18 years or older who were listed for single or bilateral lung transplant in the U.S.

​

• Development cohort: January 1, 2010, and December 31, 2013.

Testing cohort: January 1, 2016, and December 31, 2017.

​

• To avoid concerns about positivity violations associated with the likelihood of receiving a transplant, individuals who had clinical contraindications to receiving transplant, and individuals with both restrictive lung disease (diagnosis group D) and height less than five feet were removed.

​

• Data were divided into pre- and post-transplant subsets. The pre-transplant subset consisted of daily time intervals, and the post-transplant subset consisted of a single record per patient.

​

• This data structure allowed them to construct time-varying inverse probability of treatment weights (IPTW) and inverse probability of censoring weights (IPCW), which effectively circumvent survivor bias by “mapping” the survival probabilities obtained among the post-transplant group back to the full waitlist population.

Fitting the outcome models

• Pre-transplant survival model
• Weighted Cox proportional hazards model was fit to each patient’s baseline record in the pre-transplant data, weighted by each patient’s daily time-varying weight to estimate one-year pre-transplant survival accounting for dependent censoring.
• This approach incorporates information from time-varying covariates and time on the waitlist captured by the IPTW and IPCW models, while still retaining the same form of the outcome model as the current pre-transplant LAS.
• The variables in the study’s weighted Cox proportional hazards model were the same in the modified LAS as in the existing LAS, but the coefficient estimates vary due to the weights.

​

​

​

​

​

Fitting the outcome models (cont.)

• Post-transplant survival model
• One-year post-transplant survival was estimated by fitting a weighted Cox proportional hazards model to the post-transplant subset including the covariates in the existing post-transplant LAS. The estimates were weighted by each patient’s post-transplant weight (fixed at the time of transplant).
• The weighted outcome model provides an estimate of survival that reflects the entire waitlist population, whereas the current LAS estimates this quantity only among the subset of individuals who did, in fact, receive transplant. Hence, the estimate of one-year post-transplant survival obtained from this weighted outcome model differs from that included in the current LAS.

​

​

​

​

​

Assessing model performance �

• To assess the discrimination of the pre- and post-transplant outcome models, time-dependent receiver operating characteristic (ROC) curves were constructed. The area under these curves (AUC) is evaluated via nearest-neighbor smoothing at 1-year post-waitlist registration and 1-year post-transplantation, respectively.

​

• This approach accommodates censoring by viewing survival time as a “time-varying binary outcome” at each possible time point, and estimating the sensitivity and specificity of the model among all patients who are still alive and at risk of the outcome at those time points. Separate statistics were computed for the development and testing cohorts.

​

• Calibration was evaluated graphically by defining low-, medium-, and high-risk categories based on tertiles of the linear predictor of the pre- and post-transplant outcome models, averaging the survival functions within each of these risk categories, and then overlaying the observed (Kaplan-Meier) and predicted survival curves for each risk category.

​

​

Results / Discussion

Modified LAS vs. Existing LAS �

• In all cases, the AUC of the modified model is higher than that of the existing LAS, indicating that the modified model has better discrimination (ability to rank patients according to risk). However, the extent of improvement is larger in the pre-transplant population than in the post-transplant population.

​

​

​

​

Observed vs. Predicted Survival �in Development Cohort.

Modified LAS Existing LAS

​

​

​

Pre-transplant

​

​

​

​

​

Post-transplant

​

​

​

Observed vs. Predicted Survival �in Testing Cohort.

Modified LAS Existing LAS

​

​

​

Pre-transplant

​

​

​

​

​

Post-transplant

​

​

​

Difference in post-transplant survival vs. the difference in waitlist survival �

A) Score: blue-red (low to high LAS score) B) Rank: blue-red (low to high priority)

​

​

• The study postulates that the current LAS underestimates predicted transplant benefit because it only predicts this quantity among people who were indeed selected to receive transplant, who tend to be older and sicker.
• Results are consistent with that; patients estimated post-transplant survival under the modified LAS are often greater than their estimated post-transplant survival under the existing LAS.
• Because the estimated pre-transplant survival also tends to be longer under the modified LAS, a sizable number of patients with intermediate scores under the current LAS would receive lower scores under the modified LAS.

​

​

Discussion

• This study was the first to incorporate weighting into the fitting of post-transplant models to account for survivor bias and other forms of selection bias into the post-transplant population.

​

• The study improved upon weighted fitting of pre-transplant models by incorporating additional variables in the weight models to better account for dependent censoring – most notably, geography. Since these variables are only included in the weight models (not the outcome models), they would only influence the performance of the outcome model if they are associated both with survival and with patients’ selection for transplantation.

​

• Performance improvement under the modified pre- and post-transplant outcome models compared to the existing LAS models suggests that regional differences in patient selection may be important to consider when estimating pre- and post-transplant survival.

​

• That said, the extent of improvement is larger in the pre-transplant population (i.e., the full waiting list population) than in the post- transplant subset. An explanation for this could be that selection bias appeared to have a larger impact on the estimate of waiting list urgency than on the estimate of post-transplant survival.

​

​

Limitations

• The first record associated with each identification number was taken to be the initial registration date, but insufficient information is available to distinguish between patients who were newly listed and those who were reactivated after temporary waitlist removal.

​

• Individuals on the waitlist who were highly unlikely to receive transplant, due to certain patient characteristics that prevent them from finding a suitable donor organ match (e.g., high sensitization or small stature) were removed. Study findings can only be generalized to individuals on the waitlist who do not have these clinical contraindications.

​

• Study cannot account for ascertainment bias/informed presence bias. Presence in the UNOS database is not random, but rather indicates that the patient was ill enough to visit the hospital, undergo evaluation for transplant, and be registered on the waitlist).

​

• Transplant organ allocation is a highly selective process, and selection bias can occur at various stages throughout this process (e.g., decision to register a patient on the waitlist, decision to remove a waitlisted patient, decision to transplant). In this paper, the focus was restricted to selection bias with regards to individuals who died or were otherwise censored prior to receiving a transplant and present a quantitative approach to mitigating this bias in the LAS.

​

​

​

Conclusion

• This study presents an intuitive and straightforward approach to address selection bias in organ allocation, and demonstrates how principles from causal inference can be incorporated into existing prediction model frameworks to improve organ allocation.

​

• Finally, this approach can be applied to any organ allocation system that relies on estimates of pre- and post-transplant survival to prioritize patients, including those used for different organs and in other countries.

​

• The findings suggest that future revisions of the LAS and other prediction models in organ transplantation may be needed to ensure fair and equitable organ allocation.

​

​

​

​

Thank you