1 of 11

Differentially Private Survival Models

Advisors/Mentors: Professor Yu-Xiang Wang, Esha Singh, Erchi Wang

Research by: Oscar Benedek, Elaine Ho, Philly Lim, Adyah Rastogi

2 of 11

What if We Could Predict The Future?

Adyah: What if we could predict the future? We could make better informed decisions to improve our lives if we knew what might happen tomorrow. We could prepare for an earthquake, know how to study for a physics exam, or just avoid bad things from happening in general. It would transform our lives, and while we can’t exactly predict the future, we can offer something close.

Imagine you could pick any super power in the world. Some of you may pick flight or perhaps invisibility. But what about predicting the future? Imagine how we could make better informed decisions to improve our lives if we could get some insight on what could potentially happen tomorrow. We could prepare for a heavy rainstorm, or know what to study for your Physics exam, or just avoid bad things from happening overall. It would transform our lives, and while we can’t exactly predict the future, we can offer something close.

3 of 11

Survival Models

Charts progression of Disease vs. Time

Benefits: predicting events such as death, a stroke, or progression of a condition

3

Adyah: Survival Models aim to do exactly that: they predict when exactly an event will happen with the use of Electronic Health Record Data (EHR). While they don’t provide an absolute certain time, itww provides the probability of something happening. Survival models use EHR data are used in healthcare to predict when something will happen, like death, a stroke, or getting a disease. For this reason, they are incredibly useful for doctors to assess risk and diagnose and treat patients. This graph is an example of a survival model that estimates the chance of survival of someone at a certain time given that they have certain conditions.

do this: they want to predict when exactly an event will happen. While it can not provide an absolute certain date, it provides probability of something happening. Survival models are used in healthcare, they predict when something bad will happen, like death, a stroke, or the progression of a condition. For this reason, they are incredibly useful for doctors to assess risk and give patients the intervention they need. Here is one example of a survival model, where it estimates the survival chance of someone at a certain time given that they have certain conditions.

4 of 11

Background

4

>98,000,000: The aggregate number of COVID cases in the US

Heterogeneous data
Non-linear case progression
Missing data
Unaligned data progressions

More accurate survival modeling

Traditional Survival Analysis with “deep elements” - Neural networks, deep exponential families

What if we could help doctors better address COVID?

Oscar:

So how do these survival models work exactly? Well, there’s a lot of survival models to look at. However, we’ll be focusing on a cutting edge survival model that improves on more traditional models that uses ideas from deep learning to tremendously innovate in different areas. Specifically, they’re called deep survival models. The core idea behind them is that they use something akin to a deep neural network–think of this as a black box that can learn to do something–on Electronic Health Records. It essentially trains on data and gets better at predicting what happens. This allows them to work with heterogeneous data types, which is essentially means data that is different from one another, model non linear cases better, missing data, and making its time alignment of a survival model more flexible over traditional models. It seems great, but there’s an ethical issue at play here: privacy.

5 of 11

EHR Data + Current Privacy Issues

Netflix “Anonymized” dataset for public use

EHR Privacy Concerns

5

ELAINE

First what exactly is privacy? By the formal definition from the United Nations,it means that everyone has the protection under law against attacks or interference that might violate this right. EHR data is incredibly useful for deep survival models, but there are fundamental privacy issues. This is especially an issue as EHR data can fetch up to hundreds of dollars on the dark web, and could be misused for identity theft, fraud, and result in financial losses for individuals.

So how can we preserve the privacy of EHR data? One approach that has been tried is just simple anonymization of data. This doesn’t exactly work though, as there are things called linkage attacks, which in essence cross reference other databases to fill in the anonymized gaps.

More specifically, Netflix, made a dataset public for a competition to make the best algorithm to recommend shows and movies. But researchers were able to use a linkage attack by referencing IMDb data to identify the user’s zip code, watched movies, gender, and even political preference. So how can we make a better approach to privacy.

6 of 11

Differential Privacy

6

PHILLY:

To achieve better privacy we want to look at differential privacy. As a big high overview, differential privacy is a guarantee for some randomized magic algorithm or process that works on a dataset, there is plausible deniability for individual data points on its accuracy. It is defined on two very similar databases that differ by one datapoint, that once they have both run this magic randomized algorithm on the dataset, the probability of the output database are essentially the same up to some factor. So what does this mean? It means we can deal with linkage attacks because we can essentially run a randomized algorithm to add some randomness, or noise, to data that makes an attacker uncertain about whether or not they can trust the data. This means in the worst case, where you have two very identical databases that you can cross reference, there is uncertainty for an individual on whether or not they’re in the database. This is because of the probability of an individual being in it and not being in it are about the same. This still allows us to use the data as a whole though because with a large enough database, you can be certain that aggregate statistics are true up to some factor. There are some caveats: if you have a completely randomized algorithm, you can’t really derive a lot of meaningful results, on the flip side if it’s not random at all, then factor privacy isn’t really preserved at that point, so we need to find a good balance with our randomized algorithm.

7 of 11

Problem Statement

7

Our research aims to create a deep survival model that is accurate and private.

Current models are either non-private or lack proper utility

Oscar:

So our problem is that current state of the art deep survival models have this fundamental issue with privacy. Therefore, we want to ensure to apply differential privacy to the process of a deep survival model. This presents with some problems that we have to solve. First, we have to decide on the best randomized algorithm and apply some potential optimizations to it. We also have to find the best balance between the factor of epsilon of the two different databases that allows us to quantify the degree of privacy and how accuracy of our resulting output. We then have to find areas to apply this randomized process in and figure out how to do that. One area notably would be the training of the deep survival model, specifically during stochastic gradient descent, which is basically a method for a deep survival model to reduce the error of intended output or in essence how it “learns”. We then also have to quantify the utility loss of applying a randomized process to a deep survival model.

8 of 11

Solution/Our Contributions

8

Deep Survival Model with a neural network as its loss function (based on Cox Model)
Laplace Mechanism + other noise to privatize model
Find epsilon parameter that makes the DP definition true, make tweaks to model to reduce epsilon

9 of 11

9

OUR PROCESS

9

Collect/Preprocess Large Dataset

1

Build a Deep Survival Model

2

Train, Test, and Validate

3

10 of 11

Our Timeline

10

FALL �QUARTER

• Gain background knowledge

• Learn CS165A and CS291K content

• Build neural network & survival model

• Test differential privacy guarantees

PUBLISH?

• Tune model, make adjustments to reduce privacy parameter

• Write research paper & prepare research presentation

WINTER�QUARTER

SPRING QUARTER

Timeline

11 of 11

Questions?

11