1 of 21

Module 2

GCNs, AEs, VAEs, AAEs

2 of 21

2

Graph Convolutional Networks

3 of 21

3

1

2

3

4

 

Adjacency matrix

 

 

 

 

 

 

Summing up

1-hop neighbors

Summing up

2-hop neighbors

A primer on Graphs

4 of 21

4

1

2

3

4

 

Adjacency matrix

 

 

Degree matrix

  • Adjacency matrix only sums up feature vectors of all neighboring nodes, but not the node itself (unless there are self-loops)

​

  • Adjacency matrices are typically not normalized

Caveats

Possible solutions

 

 

 

 

 

 

A primer on Graphs

5 of 21

5

 

Types of problems

6 of 21

6

Type 1: Supervised drug discovery

Type 2: Semi-supervised classification of documents in citation networks

Type 3: Link prediction on knowledge graphs

Applications

7 of 21

7

 

GCNs by Kipf & Welling

8 of 21

8

 

GCNs in action

9 of 21

9

 

Original GCN

GCN with edge-features

 

GCNs with edge features

10 of 21

10

Potential Applications

 

 

11 of 21

11

Potential Applications

 

 

12 of 21

Dimensionality Reduction (PCA)

13 of 21

13

Simple AutoEncoders (AEs)

Output

Molecular

Fingerprints

Input

Molecular

Fingerprints

Latent

Layer

Encoder

Decoder

Simple AutoEncoders do not have generative capabilities

14 of 21

14

Variational AutoEncoders (VAEs)

generation_loss = mean(square(generated_image-real_image))

latent_loss = KL-Divergence(latent_variable, unit Gaussian)

loss = generation_loss + latent_loss

15 of 21

15

Variational AutoEncoders (VAEs)

 

 

16 of 21

16

Limitations of VAEs

VAEs often leave regions in the space of the prior distribution that do not map to realistic samples from the data.

​

​

​

​

AAEs aim to improve this by encouraging the output of the encoder to fill the space of the prior distribution entirely.

17 of 21

17

Adversarial AutoEncoders (AAEs)

18 of 21

18

Let’s decode AAEs

19 of 21

19

Let’s decode AAEs

Reconstruction Phase

20 of 21

20

Let’s decode AAEs

Regularization Phase – Training the discriminator

21 of 21

21

Let’s decode AAEs

Regularization Phase – Training the generator (encoder)