RoSA: A Robust Self-Aligned Framework for Node-Node Graph Contrastive Learning

​

Yun Zhu∗ , Jianhao Guo∗ , Fei Wu and Siliang Tang†

Zhejiang University

Introduction

1. node-node contrasting mode(i.e. local-local),
• better for node classification
2. node-graph(subgraph) contrasting level (i.e. local-global), 
3. graph(subgraph)-graph(subgraph) contrasting level (i.e. global-global)
• better for graph classification

An Empirical Study of Graph Contrastive Learning, Zhu et al. [NIPS-21]

Different contrasting modes

augmentation

encoder

readout

Introduction

Cons of Aligned Views

• May hinder the flexibility and variability of sampled views and restrict the expressive power of contrastive learning
• There exist certain circumstances where aligned views are unavailable

Pros of Non-Aligned Views

• Can sample different nodes and their relations more freely

​

• Can assist the model in learning more representative and robust features

How to design sub-sampling methods that can generate unaligned views while maintaining semantic consistency?

1

How to contrast two nonaligned views even the number of nodes and correspondence between nodes are inconsistent?

2

3

How to boost the performance meanwhile enhance the robustness of model for unsupervised graph contrastive learning?

Challenges

Method

We propose a robust self-aligned contrastive learning framework for node-node graph representation learning named RoSA.

1. Non-aligned

Subsampling

2. A novel graph-based optimal transport algorithm

3. Unsupervised graph adversarial training

Method

1. Non-aligned Node-Node Level Sub-Sampling (Augmentation)
2. g-EMD: A Self-Aligned Contrastive Objective (Contrastive Loss)
3. Unsupervised Adversarial Training (Auxiliary)

Other subsampling methods you can apply.

Method—Non-Aligned Sub-Sampling

1

3

2

1

3

2

4

3

1

？

1

3

2

N-N

Prior works

Motivated by optimal transport, we adopt EMD to estimate the similarity of these two embeddings

Method—A Self-Aligned Contrastive Objective

1. Earth Mover’s Distance

Method—A Self-Aligned Contrastive Objective

3. G-Earth Mover’s Distance

address this issue by using a fast iterative solution named Sinkhorn-Knopp algorithm

2. Sinkhorn-Knopp algorithm

Modified for graphs

iterations

https://vincentherrmann.github.io/blog/wasserstein/

Method—A Self-Aligned Contrastive Objective

Method—Unsupervised Adversarial Training

specifically

Experiments

Datasets:

We conduct experiments on ten (+five) public benchmark datasets:

Transductive setting

• Homophilous graphs (4+5)
• Heterophilous graphs (3)

Inductive setting (2)

Dynamic graphs (1)

​

Experimental setup:

The model size and evaluation metrics are the same as previous works(e.g. GRACE, BGRL) for fair comparison.

​

Baselines:

We compare RoSA with two node-graph constrasting methods DGI, SUBG-CON, and four node-node methods GMI, GRACE, GCA and BGRL.

​

​

Experiments—Transductive setting

Note: These five experiments are on the Appendix materials

Node classification(Homophilous graphs) under transductive setting

Experiments

Experiments—Ablation Study

Conclusions

• We propose a robust self-aligned framework for node-node graph contrastive learning
• We introduce unsupervised adversarial training into graph domain to further improve the robustness of the model.
• Extensive experiment results on homophilous, non-homophilous and dynamic graphs datasets demonstrate that our model can effectively be applied to non-aligned situations and outperform other competitors.