Machine Learning in Biomedicine

Sean Davis, MD, PhD

Center for Cancer Research, National Cancer Institute,

National Institutes of Health

July 9, 2017

@seandavis12

​

​

​

Gartner Hype Cycle

http://www.gartner.com/newsroom/id/3412017

​

ML Tech Triggers

• Lots of Big Data
• Transition to digital in all aspects of life and science
• Systems and ecosystems that drive and are driven by data production and consumption
• Advertising,
• Social media
• Finance
• Health care
• Energy
• Transportation

​

ML Tech Triggers

• Cheap and easy access to lots of compute resources
• Build a small-scale prototype system, immediately scale to petabytes or even larger
• Advances in coprocessors and GPUs
• Bigger, faster memory and storage

​

ML Tech Triggers

• New software
• Commodity ecosystems for big data and machine learning
• Workforce increasingly skilled and engaged
• National and international recognition
• Cyber
• USA Chief Data Scientist

“A computer program is said to learn from experience (E) with some class of tasks (T) and a performance measure (P) if its performance at tasks in T as measured by P improves with E”

​

​

http://machinelearningmastery.com/what-is-machine-learning/

​

Applications

Algorithms

Supervised learning

https://blog.recast.ai/machine-learning-algorithms/

​

Unsupervised Learning

https://blog.recast.ai/machine-learning-algorithms/

​

http://scikit-learn.org/stable/tutorial/machine_learning_map/

​

An early example--gene expression classifier

An early genomics example of supervised ML

Result is a program to

perform predictions!

Naive Bayes

Individual features as predictors

Likelihood ratio

Feature relationships

Exploiting independence

RandomForests

Slides from Anshul Kundaje

Functional Elements in the genome

Ecker et al. 2012

Repressed Gene

Transcription factors

(Regulatory proteins)

Enhancer

Insulator

Promoter

Active Gene

DNA

Motif

Dynamic Regulation of gene expression

Chromatin immunoprecipitation (ChIP-seq)

Protein-DNA binding maps

Maps of histone modifications

Maps of histone variants

Chromatin accessibility (DNase-seq, FAIRE-seq, ATAC-seq) and nucleosome sequencing (MNase-seq)

DNase-seq (ATAC-seq) ~= sum(ChIP-seq for all TFs)

ENCODE functional signal maps

Chromatin modification maps

Transcription factor binding map

H3K4me3

H3K27ac

H3K4me1

H3K36me3

H3K27me3

Relationship between chromatin marks and gene expression

Aggregation analysis and simple univariate correlation analysis suggest strong positive or negative relationships between gene expression and enrichment of chromatin marks at gene promoters

What is the collective predictive power of a set of chromatin marks? Which ones are more predictive?

Multivariate predictive model

Linear Regression model

Minimize square error to find betas

Input variables (features)

Regression coefficients, correlation, independence

A non-linear model (decision/regression tree)

• Top-down, greedy procedure
• Recursively splits data based on a single feature
• Goal is to determine subsets of data with the same classes (for classification), or similar values (for regression)
• Predict the majority class / average value for the subset

H3K4me3 > 5

H3K4me1 > 10

H3K27me3 > 10

Multivariate predictive model (Random forest)

• Ensemble of decision or regression trees learned on bootstrapped samples of training data
• Average prediction from all trees

Random forests

Utilizes random sampling over

• Features - only allow splits on a small subset
• Examples - train trees on subsamples of data

to construct a collection of decision trees (forest) with significantly improved prediction performance

Learning algorithm

• Use random-forests to generate rules
• Sample variable depth decision trees (average depth i.e. size of rule ~ 6)
• 2000 to 5000 potentially redundant rules

​

• L1 regularization to learn sparse set of non-redundant rules
• Optimize squared-error ramp loss

​

​

• L1 penalty for sparsity
• Tries to make many coefficients 0

RuleFit3: Friedman et al. 2005

TF₁

TF₂

TF₃

r_k

Projected co-association scores

Partial Dependence of F(X) on a set (eg. pairs) of TFs ‘g’

G = complement of g

Co-association score between pairs of TFs

RuleFit3: Friedman et al. 2005

Deep learning example

Several slides adapted from Anshul Kundaje

Deep learning

https://www.slideshare.net/LuMa921/deep-learning-a-visual-introduction

What does deep learning learn?

https://www.slideshare.net/LuMa921/deep-learning-a-visual-introduction

https://www.slideshare.net/LuMa921/deep-learning-a-visual-introduction

Chromatin architecture cartoon

ATAC-Seq and fragment lengths

Chromatin architecture and chromatin state

Transform a 1D dataset into 2D

Transformed to an image classification problem!

Convolution

Convolution

Convolution learns multiple low-level features

Pooling, or smoothing, to gain robustness

How does learning proceed?

How does learning proceed?

How does learning proceed?

Learn from sequence as well

Kundaje’s Chromputer

Closing thoughts

MANY, MANY more applications

https://www.cancer.gov/research/key-initiatives/moonshot-cancer-initiative/blue-ribbon-panel/blue-ribbon-panel-report-2016.pdf

Internet of Things

Potential to fundamentally change the way we interact with research subjects, patients, and the general population.