More Like this

1. Contrastive quant: quantization makes stronger contrastive learning

   https://doi.org/10.1145/3489517.3530419  

   Fu, Yonggan; Yu, Qixuan; Li, Meng; Ouyang, Xu; Chandra, Vikas; Lin, Yingyan (July 2022, DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference)

   Contrastive learning learns visual representations by enforcing feature consistency under different augmented views. In this work, we explore contrastive learning from a new perspective. Interestingly, we find that quantization, when properly engineered, can enhance the effectiveness of contrastive learning. To this end, we propose a novel contrastive learning framework, dubbed Contrastive Quant, to encourage feature consistency under both differently augmented inputs via various data transformations and differently augmented weights/activations via various quantization levels. Extensive experiments, built on top of two state-of-the-art contrastive learning methods SimCLR and BYOL, show that Contrastive Quant consistently improves the learned visual representation. 

   more » « less
2. Debiased Contrastive Learning

   Chuang, Ching-Yao; Robinson, Joshua; Yen-Chen, Lin; Torralba, Antonio; Jegelka, Stefanie (January 2020, 34th Conference on Neural Information Processing Systems (NeurIPS 2020))
3. Contrastive Attraction and Contrastive Repulsion for Representation Learning

   Zheng, Huangjie; Chen, Xu; Yao, Jiangchao; Yang, Hongxia; Li, Chunyuan; Zhang, Ya; Zhang, Hao; Tsang, Ivor; Zhou, Jingren; Zhou, Mingyuan (July 2023, Transactions on Machine Learning Research)
4. Adversarial Graph Contrastive Learning

   https://doi.org/10.1145/3638054  

   Feng, Shengyu; Jing, Baoyu; Zhu, Yada; Tong, Hanghang (May 2024, ACM Transactions on Knowledge Discovery from Data)

   Contrastive learning is an effective unsupervised method in graph representation learning. The key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle. Recently, the data-augmentation-based contrastive learning method has advanced to show great power in the visual domain, and some works have extended this method from images to graphs. However, unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and it is much harder to provide high-quality contrastive samples, which leaves much space for improvement. In this work, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method,Adversarial Graph Contrastive Learning(ArieL), to extract informative contrastive samples within reasonable constraints. We develop a new technique calledinformation regularizationfor stable training and use subgraph sampling for scalability. We generalize our method from node-level contrastive learning to the graph level by treating each graph instance as a super-node.ArieLconsistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks on real-world datasets. We further demonstrate thatArieLis more robust in the face of adversarial attacks. 

   more » « less
5. Graph Communal Contrastive Learning

   https://doi.org/10.1145/3485447.3512208  

   Li, Bolian; Jing, Baoyu; Tong, Hanghang (April 2022, Graph Communal Contrastive Learning)

   Graph representation learning is crucial for many real-world ap- plications (e.g. social relation analysis). A fundamental problem for graph representation learning is how to effectively learn rep- resentations without human labeling, which is usually costly and time-consuming. Graph contrastive learning (GCL) addresses this problem by pulling the positive node pairs (or similar nodes) closer while pushing the negative node pairs (or dissimilar nodes) apart in the representation space. Despite the success of the existing GCL methods, they primarily sample node pairs based on the node- level proximity yet the community structures have rarely been taken into consideration. As a result, two nodes from the same community might be sampled as a negative pair. We argue that the community information should be considered to identify node pairs in the same communities, where the nodes insides are seman- tically similar. To address this issue, we propose a novel Graph Communal Contrastive Learning (𝑔𝐶𝑜𝑜𝐿) framework to jointly learn the community partition and learn node representations in an end-to-end fashion. Specifically, the proposed 𝑔𝐶𝑜𝑜𝐿 consists of two components: a Dense Community Aggregation (𝐷𝑒𝐶𝐴) algo- rithm for community detection and a Reweighted Self-supervised Cross-contrastive (𝑅𝑒𝑆𝐶) training scheme to utilize the community information. Additionally, the real-world graphs are complex and often consist of multiple views. In this paper, we demonstrate that the proposed 𝑔𝐶𝑜𝑜𝐿 can also be naturally adapted to multiplex graphs. Finally, we comprehensively evaluate the proposed 𝑔𝐶𝑜𝑜𝐿 on a variety of real-world graphs. The experimental results show that the 𝑔𝐶𝑜𝑜𝐿 outperforms the state-of-the-art methods. 

   more » « less