ICML (2018)
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, Richard Zemel
Neural Relational Inference for Interacting Systems
CONTENTS
Introduction
01
02
03
04
Model
Experiments
Conclusion
Introduction
01
Introduction
They introduce the neural relational inference (NRI) model: an unsupervised model that learns to infer interactions while simultaneously learning the dynamics purely from observational data. The model takes the form of a variational auto-encoder(VAE), in which the latent code represents
the underlying interaction
graph and the reconstruction is
based on graph neural networks.
Model
02
Model
message passing of graph
Model
Symbols
Model
neural relational inference (NRI)
the NRI consists of two parts trained jointly：the encoder predicts the interactions(edge type) given the trajectories, the decoder learns the dynamic model given the interaction graph.
Model
neural relational inference (NRI)
encoder
x
NN
X
Model
Encoder
Model
Sampling
Model
Decoder
Model
Decoder
Model
Training
Model
Training
ELBO objective：
reconstruction error : want this as large as possible
this term is estimated by：
KL divergence： want this as small as possible
this term just sum of entropy:
NN
x
z
NN
Experiments
03
Result
they run their model on three simulated physical system and compared performance, both in future state prediction and in accuracy of estimating the edge type.
Experiments
Edge prediction
Corr.(path)：estimate the interaction graph by thresholding the matrix of correlaions between trajectory feature vectors.
Corr.(LSTM)： LSTM with shared parameters to model each trajectory individually and calculates correlations between the final hidden states to arrive at an interaction matrix after thresholding.
NRI(sim.) ：NRI model with the ground-truth simulation decoder.
Experiments
Future state prediction
static：copy the previous
state vector.
LSTM(single)：run separate LSTM(shared weights) for each object.
LSTM(joint)：concatenate all state vectors and feeds it into one LSTM that is trained to predict all feature states simultaneously.
NRI(full graph)：message passing decoder on fully-connected graph without edge type(without inferring edges)
NRI(true) ：passing ground true interaction graph to decoder.
Experiments
Examples of predicted walking motion
Experiments
conclusion
04
neural relational inference (NRI) can simultaneously
infer relational structure while learning the
dynamical model of an interacting system.
The NRI model is highly effective
at unsupervised recovery of ground-truth
interaction graphs.
Dynamic Neural Relational Inference for Forecasting Trajectories
(CVPR2020)
The future work that can infer dynamic relations between each objects for
every time steps.
conclusion
Thanks