More Like this

1. Control-based Graph Embeddings with Data Augmentation for Contrastive Learning

   Ahmad, Obaid_Ullah; Said, Anwar; Shabbir, Mudassir; Koutsoukos, Xenofon; Abbas, Waseem (July 2024, IEEE)

   In this paper, we study the problem of unsupervised graph representation learning by harnessing the control properties of dynamical networks defined on graphs. Our approach introduces a novel framework for contrastive learning, a widely prevalent technique for unsupervised representation learning. A crucial step in contrastive learning is the creation of ‘augmented’ graphs from the input graphs. Though different from the original graphs, these augmented graphs retain the original graph’s structural characteristics. Here, we propose a unique method for generating these augmented graphs by leveraging the control properties of networks. The core concept revolves around perturbing the original graph to create a new one while preserving the controllability properties specific to networks and graphs. Compared to the existing methods, we demonstrate that this innovative approach enhances the effectiveness of contrastive learning frameworks, leading to superior results regarding the accuracy of the classification tasks. The key innovation lies in our ability to decode the network structure using these control properties, opening new avenues for unsupervised graph representation learning. 

   more » « less
2. Adversarial Graph Contrastive Learning

   https://doi.org/10.1145/3638054  

   Feng, Shengyu; Jing, Baoyu; Zhu, Yada; Tong, Hanghang (May 2024, ACM Transactions on Knowledge Discovery from Data)

   Contrastive learning is an effective unsupervised method in graph representation learning. The key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle. Recently, the data-augmentation-based contrastive learning method has advanced to show great power in the visual domain, and some works have extended this method from images to graphs. However, unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and it is much harder to provide high-quality contrastive samples, which leaves much space for improvement. In this work, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method,Adversarial Graph Contrastive Learning(ArieL), to extract informative contrastive samples within reasonable constraints. We develop a new technique calledinformation regularizationfor stable training and use subgraph sampling for scalability. We generalize our method from node-level contrastive learning to the graph level by treating each graph instance as a super-node.ArieLconsistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks on real-world datasets. We further demonstrate thatArieLis more robust in the face of adversarial attacks. 

   more » « less
3. Adversarial Graph Contrastive Learning with Information Regularization

   https://doi.org/10.1145/3485447.3512183  

   Feng, Shengyu; Jing, Baoyu; Zhu, Yada; Tong, Hanghang (April 2022, Adversarial Graph Contrastive Learning with Information Regularization)

   Contrastive learning is an effective unsupervised method in graph representation learning. Recently, the data augmentation based con- trastive learning method has been extended from images to graphs. However, most prior works are directly adapted from the models designed for images. Unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples, which are the key to the performance of contrastive learning models. This leaves much space for improvement over the existing graph contrastive learning frameworks. In this work, by introducing an adversarial graph view and an information regularizer, we propose a simple but effective method, Adversarial Graph Contrastive Learning (ArieL), to extract informative contrastive samples within a reasonable constraint. It consistently outperforms the current graph contrastive learning methods in the node classification task over various real-world datasets and further improves the robustness of graph contrastive learning. 

   more » « less
4. Scalable Deep Metric Learning on Attributed Graphs

   Li, Xiang; Agrawal, Gagan; Jin, Ruoming; Ramnath, Rajiv (July 2024, LNCS)

   We consider the problem of constructing embeddings of large attributed graphs and supporting multiple downstream learning tasks. We develop a graph embedding method, which is based on extending deep metric and unbiased contrastive learning techniques to 1) work with attributed graphs, 2) enabling a mini-batch based approach, and 3) achieving scalability. Based on a multi-class tuplet loss function, we present two algorithms -- DMT for semi-supervised learning and DMAT-i for the unsupervised case. Analyzing our methods, we provide a generalization bound for the downstream node classification task and for the first time relate tuplet loss to contrastive learning. Through extensive experiments, we show high scalability of representation construction, and in applying the method for three downstream tasks (node clustering, node classification, and link prediction) better consistency over any single existing method. 

   more » « less
5. Learning Backbones: Sparsifying Graphs Through Zero Forcing for Effective Graph-Based Learning

   Ahmad, Obaid Ullah; Said, Anwar; Shabbir, Mudassir; Koutsoukos, Xenofon; Abbas, Waseem (April 2025, Springer Nature Switzerland)

   Cherifi, Hocine; Donduran, Murat; Rocha, Luis; Cherifi, Chantal; Varol, Onur (Ed.)

   This paper introduces a novel framework for graph sparsification that preserves the essential learning attributes of original graphs, improving computational efficiency and reducing complexity in learning algorithms. We refer to these sparse graphs as “learning backbones.” Our approach leverages the zero-forcing (ZF) phenomenon, a dynamic process on graphs with applications in network control. The key idea is to generate a tree from the original graph that retains critical dynamical properties. By correlating these properties with learning attributes, we construct effective learning backbones. We evaluate the performance of our ZF-based backbones in graph classification tasks across eight datasets and six baseline models. The results demonstrate that our method outperforms existing techniques. Additionally, we explore extensions using node distance metrics to further enhance the framework’s utility. 

   more » « less