1 of 1

The explosion of graph-structured data, encompassing social networks, biological pathways, and knowledge graphs, necessitates the development of robust and generalizable graph representation learning techniques. Traditional Euclidean distance metrics often fail to capture the intricate relationships inherent in graph structures, hindering the effectiveness of learned representations in real-world applications. This research investigates the potential of non-Euclidean distances, graph augmentation techniques, and contrastive learning objectives to overcome these limitations and generate representations with superior impact on real-life tasks. We will explore how various non-Euclidean distance metrics influence the quality of learned embeddings. We will further delve into the efficacy of diverse graph augmentation techniques combined with contrastive learning objectives that encourage the model to distinguish similar from dissimilar graph structures. The effectiveness of the proposed approach will be evaluated on benchmark datasets through downstream tasks relevant to specific application domains. Additionally, we will incorporate visual analysis techniques to investigate the explainability of the learned representations, fostering a deeper understanding of how non-Euclidean distances and augmentations contribute to improved performance. Finally, as a foundation for future work, we aim to automate the contrastive learning framework, making it readily applicable to various graph representation learning problems.

ABSTRACT

OBJECTIVES

A Brief Introduction to GRACE

Deep GRAph Contrastive rEpresentation learning (GRACE) is a contrastive learning framework for graph representation learning that leverages data augmentation to artificially enrich the training data. It incorporates an objective function that encourages the model to distinguish similar from dissimilar augmented graph views. While GRACE has shown promising results, we believe there is potential to further enhance the learned representations by employing a more suitable distance metric for graph data.

Proposed method

Our proposed approach builds upon the core principles of GRACE but incorporates Pearson correlation as the distance metric within the contrastive learning objective function. Pearson correlation, unlike Euclidean distance, measures the linear dependence between two vectors, potentially better capturing the structural relationships within graphs. Additionally, we intend to explore the impact of various graph augmentation techniques in conjunction with Pearson correlation within the contrastive learning framework. Experiments conducted so far are:

Evaluate the Efficacy of Pearson Correlation: We assess how employing Pearson correlation as a distance metric within the contrastive learning objective function of GRACE influences the quality of learned graph embeddings.

Enhance Downstream Task Performance: We compare performance on a downstream task (e.g., node classification) when utilizing Pearson correlation within the contrastive learning framework compared to the baseline GRACE approach with cosine similarity.

​

​

​

​

METHODS

Experiments were performed for Manhattan distance and distance correlation as well, but results were not very promising.

​

​

​

​

​

​

​

Above are training results for obtaining node embeddings using Manhattan distance (left) and Distance correlation (right).

Better training performance was observed when using Pearson’s correlation so further tests were performed for node-level classification on benchmark dataset Cora.

​

​

​

​

​

​

​

​

​

​

​

​

​

These classification reports for cosine similarity on left and Pearson’s correlation on right suggest that using Pearson’s correlation produced better results than using cosine similarity considering precision, recall and f1-score per label. Using Pearson’s correlation improved the accuracy score by 1%.

Further, we perform visual analysis to see how well nodes are clustered initially in a 2-d space based on the obtained embeddings.

​

​

​

​

​

​

​

​

RESULTS

CONCLUSIONS

1. Manhattan distance and distance correlation are not suitable replacement for cosine similarity.
2. Pearson’s correlation has the same range as cosine similarity, and both are similarity-based measures unlike Manhattan distance that measures dissimilarity.
3. Pearson’s correlation is the most suitable replacement for cosine similarity so far.
4. Cosine similarity provides denser embeddings compared to Pearson’s correlation.
5. Pearson’s correlation captures both strength and direction of similarity while cosine similarity only captures strength.

REFERENCES

1. Philip Bachman, R. Devon Hjelm, and William Buchwalter. Learning Representations by Maximizing Mutual Information Across Views. In NeurIPS, 2019.
2. Petar Velickovi ˇ c, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. ´ Deep Graph Infomax. In ICLR, 2019.
3. Michael Tschannen, Josip Djolonga, Paul K. Rubenstein, Sylvain Gelly, and Mario Lucic. On Mutual Information Maximization for Representation Learning. In ICLR, 2020.
4. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS, 2019.

• Investigate the effectiveness of non-Euclidean distance metrics: The focus is on evaluating how different non-Euclidean distance metrics combined with different objective functions influence the quality of learned graph embeddings.
• Explore the impact of graph augmentation techniques: The aim is to understand how diverse graph augmentation techniques enhance the contrastive learning process for graph representation.
• Evaluate the combined approach on downstream tasks: It focuses on assessing the effectiveness of the proposed approach (utilizing non-Euclidean distances and graph augmentations within a contrastive learning framework) on relevant benchmark datasets through downstream tasks.
• Incorporate visual analysis for explainability: To leverage interactive visual analysis techniques to gain a deeper understanding of how non-Euclidean distances and augmentations contribute to improved performance in the learned representations.
• Automate the contrastive learning framework (future work): This objective, aiming for future work, proposes the development of an automated contrastive learning framework that can be readily applied to various graph representation learning problems.

University of Missouri-Kansas city

Udiptaman Das

Towards Robust and Generalizable Graph Representations: Exploring Non-Euclidean Distances, Graph Augmentations, and Contrastive Learning Paradigms

Training GRACE encoder network to generate embeddings using cosine similarity

Training GRACE encoder network to generate embeddings using Pearson’s correlation

ACKNOWLEDGEMENT

We would like to express our gratitude to Dr. Yugyung Lee, for her invaluable guidance and support throughout this research project. Her insightful feedback and encouragement were instrumental in shaping this work.

​

RESEARCH POSTER PRESENTATION DESIGN © 2019

www.PosterPresentations.com