Dynamics and context specificity in biological networks

Oct 7^th 2021

Original slides created by Prof. Sushmita Roy

BMI 826-23 Computational Network Biology�Fall 2021

Anthony Gitter

https://compnetbiocourse.discovery.wisc.edu

Topics in this section

• Types of dynamic network models
• Non-stationary dynamic Bayesian networks
• Input-Output Hidden Markov Models
• Multi-task learning of graphs

Goals for today

• Overview of different models to capture dynamics and context-specificity
• Non-stationary dynamic Bayesian networks

What do we mean by context?

• We will define context broadly
• Context can be time, developmental stage, tissue, cell type, organ, disease, strains/individuals, species, stimuli

Different individuals

Different cell types

Different species

Network dynamics and context specificity

• What does modeling “dynamics” mean?
• The activity of nodes change over time and we want to model how this happens
• The network (structure or parameters) changes with time
• Structure can change due to changes at the node or edge level
• What models can be used for capturing dynamics in networks?
• How do these models capture dynamics?
• Node level
• Edge level

How do dynamics and context-specificity improve network inference?

• Multiple related contexts

​

​

​

​

• Temporal ordering of contexts

​

​

​

​

How do dynamics and context-specificity improve network inference?

• Multiple related contexts
• Assume similar contexts have similar network structures
• Pool information across contexts

• Temporal ordering of contexts
• Resolve ambiguous independence relationships
• Regulator activated/inhibited before target

​

​

​

​

What is special about dynamic models?

• Time points could be treated as contexts

​

• Dynamic models are affected if time points are re-ordered

​

​

​

​

Expression matrix

Reordered expression matrix

Strategies for capturing context and dynamics in networks

• Which network inference methods have we already seen in class that account for dynamics?

Strategies for capturing context and dynamics in networks

• Skeleton network-based approaches
• Input-Output Hidden Markov Models (next week)
• Dynamic networks with temporal transition of edges
• Time-varying networks (today)
• Multi-task learning approaches (next week)
• Time-lagged network models
• Ordinary Differential Equations (ODEs)
• Time-lagged dependency networks
• Vector autoregression
• Dynamic Bayesian Networks and non-stationary extensions

Skeleton network-based approaches

• Assume a background/skeleton network that defines the universe of possible edges
• The network changes because node activity levels change
• Skeleton networks:

A

B

Gene C

Transcription factors

C

A

B

DNA

X

Y

X

Y

Transcriptional regulatory

Protein-protein interaction

Skeleton network-based approaches

• Assume a background/skeleton network that defines the universe of possible edges
• The network changes because node activity levels change

Kim et al. 2013 Briefings in Bioinformatics

Skeleton network-based approaches

• Given
• A fixed network of nodes (genes, proteins, metabolites)
• Activity levels of network nodes in a set of contexts (e.g. tissues, time points)
• Do
• Find subset of edges that are active in each context

​

​

Active subnetworks

• Subnetwork that exhibits an unexpectedly high level of expression change

​

• Not all nodes in the subnetwork must be active

​

• Supports multiple conditions but not time

​

• Yeast galactose pathway knockout example

​

​

Ideker et al. 2002 Bioinformatics

Timing in skeleton networks

• TimeXNet: order nodes in network according to their timing

​

• Based on network flow

​

• Temporal Pathway Synthesizer related but stronger constraints (Köksal et al. 2018 Cell Reports)

​

​

Patil and Nakai 2014 BMC Systems Biology

Time-varying networks

• The network changes between time at the node and edge levels
• Example approaches
• TESLA (Ahmed & Xing 2009 PNAS)
• KELLER (Song et al. 2009 Bioinformatics)
• TESLA
• Temporally smoothed l₁-regularized logistic regression
• Based on temporal Exponential Random Graph Models
• Assumes binary node values
• Imposes a regularization term to make the graphs change smoothly over time
• Estimates the graph structure by solving a set of local regularized logistic regression problems

​

TESLA for Drosophila development

• Summarize 4028 genes by 43 ontology groups
• 66 time points: embryonic, larval, pupal, and adulthood
• Arbeitman et al. 2002 Science
• Observe smooth changes over time

Ahmed & Xing 2009 PNAS

Time-lagged network models

• Regulator activity does not instantaneously influence target
• Look at past activity of regulators to predict current expression of targets

​

• Previously discussed
• Dynamic Bayesian networks
• Inferelator

​

Time-lagged network models

Lu et al. 2021 PLOS Comp Bio

ODE models in networks

• Assume we are modeling a gene regulatory network
• Let x_i denote the expression level of the i^th gene
• ODE for the change in expression of i^th gene is

​

​

​

• This is often approximated by a finite difference approximation

​

Greenfield et al. 2013 Bioinformatics

Regulators of i^th gene

Expression level of regulator p

Degradation rate

Dynamic Bayesian networks (DBNs)

• Suppose we have a time course of node activity levels
• We assume that the activity levels at time t+1 depends upon t
• But this does not change with t
• DBNs can be used to model how the system evolves over time
• We may have a different network at the first time point

​

Non-stationary dynamic Bayesian networks (nsDBNs)

• Standard DBN assumes that the dependencies between the previous time point (t-1) and the current time point (t) are always the same
• Dependencies do not depend on t
• Stationarity assumption temporal model
• Non-stationary DBNs relax this assumption
• Robinson and Hartemink 2010 Journal of Machine Learning Research
• Dependency structure can depend on the time window

Non-stationary dynamic Bayesian networks (nsDBNs)

• When may there be different time windows in a dynamic biological process?

nsDBN graph structure transitions

• Suppose we have three time windows
• nsDBNs require us to define the dependency structure in each time window

Adapted from Robinson & Hartemink 2010

t₁

t₂

Transition times

X₁

X₃

X₄

X₂

X₁

X₃

X₄

X₂

Edge set changes between time windows

G₁

G₂

G₃

nsDBN graph structure transitions

• N discrete time points
• m epochs or time periods between transitions times t_i
• Graph structures G₁,..., G_m for each epoch

Robinson & Hartemink 2010

Transition t₁

Epoch 2

Time point 719

Simulated dataset

Posterior distribution over graph structures

• For m epochs, we would like to find G₁,.., G_m by optimizing their posterior distribution

​

​

​

​

Prior over m graphs; can be used to incorporate our prior knowledge of how the graphs transition

Describe in terms of edge changes at each transition

Plate notation for graphical models

• Shaded variables are observed
• Unshared variables are unobserved (latent)

Dynamic Bayesian network

Graph structure

Pseudocount

One for each of

the n genes

Expression for gene i

Parent set of gene i

Conditional probability distribution parameters for gene i

Robinson & Hartemink 2010

nsDBN: Known Number of Known Times of transitions

nsDBN: KNKT

Graph structure at epoch j

Pseudocount

One for each of

the n genes

Expression for gene i

Parent set of gene i

in epoch j

Conditional probability distribution parameters for gene i for parent set h

Prior on edge transitions

Transition

Number of transitions

• Shaded variables are observed
• Unshared variables are unobserved (latent)

Prior knowledge about the dynamic process

• Is the Known Number of Known Times setting realistic?

​

​

• What would a Bayesian do?

nsDBN: Unknown Number of Unknown Times of transitions

nsDBN: UNUT

Graph at epoch j

Pseudocount

One for each of

the n genes

Expression for gene i

Parent set of gene i

in epoch j

Conditional probability distribution parameters for gene i for parent set h

Prior on edge transitions

Transition

Number of transitions

Prior on number of transitions

• Shaded variables are observed
• Unshared variables are unobserved (latent)

Assumptions on graph structures

• Maximum parents per variable p_max
• Maximum edge changes per transition s_max
• Truncated geometric prior on changes s_i per transition

​

​

• Truncated geometric prior on number of epochs m

​

​

• Convenient form for log likelihood calculations

Metropolis-Hastings for inference

• Similar to inference procedure we saw for stationary DBN

​

• What new states or move sets are needed?

Metropolis-Hastings for inference

• Move sets
• Add edge to G₁
• Delete edge from G₁
• Add edge to Δg_i
• Delete edge from Δg_i
• Move edge from Δg_i to Δg_j
• Shift t_i
• Merge Δg_i and Δg_i+1
• Split Δg_i
• Create Δg_i
• Delete Δg_i

​

KNKT, KNUT, UNUT

KNUT, UNUT

UNUT

Evaluating nsDBN: simulations

Robinson & Hartemink 2010

Evaluating nsDBN: Drosophila

• Same dataset used by TESLA but only use 11 genes
• A: Stationary directed network (Zhao et al. 2006)
• B: Non-stationary undirected network (Guo et al. 2007)
• C: nsDBN in KNKT setting (Robinson & Hartemink 2010)

Take away points

• If we can assume contexts or times are related, don’t learn independent networks for each
• Temporal ordering can reduce ambiguity in network inference

​

• Stationary assumption unrealistic in many cases
• Non-stationary DBN is a generalization
• Suitable for unknown transitions
• Requires sufficient time points
• Limit complexity of graph structure for scalability