Graph Random Neural Networks for

Semi-Supervised Learning on Graphs

Presented by:

Gauransh Sawhney 2018A3PS0325P

Utkarsh Kumar Singh 2018A3PS0368P

​

In partial fulfillment of the requirements of the course:

BITS F464 Machine Learning

Submitted to: Dr. Kamlesh Tiwari

Semi-Supervised Learning

• What is Semi-Supervised Learning
• Semi-Supervised Learning on Graphs

Semi-Supervised Learning on Graphs

Graph Neural Networks

• What are Graph Neural Networks(GNNs)
• Problems with existing implementations

Graph Neural Network

Existing Issues

• Oversmoothing
• Stacking multiple GNN layers makes nodes indistinguishable; coupling the feature propagation and non-linear transformation steps, aggravates this problem
• Not robust to graph attacks
• Each node is highly dependent on neighbors, making it non-robust to noise
• Overfitting in case of semi-supervised
• In the standard setting of semi-supervised training, scarce node label information can be overfit

GRAND: Graph Random Neural Network

• Architecture
• Algorithm
• How does it tackle the issues faced by other GNNs

GRAND

GRAND

Algorithm

Loss Functions

Results

Some of the results presented in the paper:

​

• Comparison with existing architectures on benchmarks
• Generalization analysis
• Robustness analysis
• Over-smoothing analysis
• Results on large datasets

Dataset Description

3 datasets were used to benchmark results

Comparison with existing architectures

Generalization Analysis

1. Without RP

(b) Without CR

(c) GRAND(with RP and CR)

Robustness Analysis

Over-smoothing analysis

Other results presented in the paper

Over-smoothness of GRAND and its variants(on Cora)

Other results presented in the paper

Classification Accuracy of GRAND on large datasets

Experiments

The following experiments were conducted:

​

• MLP v/s GCN as classification network
• Classification accuracy v/s {K, S}
• Sensitivity wrt CR loss coefficient ƛ

1. MLP v/s GCN

Effect of using an MLP vs GCN as the classification network

GRAND very clearly outperforms

GRAND_GCN in terms of classification accuracy

2. Classification Accuracy v/s {K, S}

(i) Effect of K(propagation order) and S(number of data augmentations) on

Classification Accuracy on GRAND(DropNode data augmentation)

2. Classification Accuracy v/s {K, S}

​

2. Classification Accuracy v/s {K, S}

​

(ii) Effect of K(propagation order) and S(number of data augmentations) on

Classification Accuracy on GRAND_dropout and GRAND_dropedge(alternative data augmentation techniques)

3. Sensitivity wrt ƛ

Classification Accuracy v/s ƛ

MLP classification network

GCN classification network

Both using DropNode for data augmentation

Consistency Regularization Loss Coefficient

3. Sensitivity wrt ƛ

​

Classification Accuracy v/s ƛ

Both using MLPs as classification networks

DropEdge data augmentation

Dropout data augmentation

3. Sensitivity wrt ƛ

​

Ablation Study

The effect of the absence of the following parameters was studied:

​

• w/o consistency regularization(CR)
• w/o multiple dropnode(mDN)
• w/o sharpening
• w/o consistency regularization(CR) and dropnode(DN)

Ablation Study

these are classification accuracies

END

Thank You!