More Like this

1. Contrastive quant: quantization makes stronger contrastive learning

   https://doi.org/10.1145/3489517.3530419  

   Fu, Yonggan; Yu, Qixuan; Li, Meng; Ouyang, Xu; Chandra, Vikas; Lin, Yingyan (July 2022, DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference)

   Contrastive learning learns visual representations by enforcing feature consistency under different augmented views. In this work, we explore contrastive learning from a new perspective. Interestingly, we find that quantization, when properly engineered, can enhance the effectiveness of contrastive learning. To this end, we propose a novel contrastive learning framework, dubbed Contrastive Quant, to encourage feature consistency under both differently augmented inputs via various data transformations and differently augmented weights/activations via various quantization levels. Extensive experiments, built on top of two state-of-the-art contrastive learning methods SimCLR and BYOL, show that Contrastive Quant consistently improves the learned visual representation. 

   more » « less
2. Contrastive Unlearning: A Contrastive Approach to Machine Unlearning

   https://doi.org/10.24963/ijcai.2025/830  

   Lee, Hong kyu; Zhang, Qiuchen; Yang, Carl; Lou, Jian; Xiong, Li (September 2025, International Joint Conferences on Artificial Intelligence Organization)

   Machine unlearning aims to eliminate the influence of a subset of training samples (i.e., unlearning samples) from a trained model. Effectively and efficiently removing the unlearning samples without negatively impacting the overall model performance is challenging. Existing works mainly exploit input and output space and classification loss, which can result in ineffective unlearning or performance loss. In addition, they utilize unlearning or remaining samples ineffectively, sacrificing either unlearning efficacy or efficiency. Our main insight is that the direct optimization on the representation space utilizing both unlearning and remaining samples can effectively remove influence of unlearning samples while maintaining representations learned from remaining samples. We propose a contrastive unlearning framework, leveraging the concept of representation learning for more effective unlearning. It removes the influence of unlearning samples by contrasting their embeddings against the remaining samples' embeddings so that their embeddings are closer to the embeddings of unseen samples. Experiments on a variety of datasets and models on both class unlearning and sample unlearning showed that contrastive unlearning achieves the best unlearning effects and efficiency with the lowest performance loss compared with the state-of-the-art algorithms. In addition, it is generalizable to different contrastive frameworks and other models such as vision-language models. Our main code is available on github.com/Emory-AIMS/Contrastive-Unlearning 

   more » « less
3. Contrastive Dual Gating: Learning Sparse Features With Contrastive Learning

   Jian Meng; Li Yang; Jinwoo Shin; Deliang Fan; Jae-sun Seo (June 2022, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR))
4. Debiased Contrastive Learning

   Chuang, Ching-Yao; Robinson, Joshua; Yen-Chen, Lin; Torralba, Antonio; Jegelka, Stefanie (January 2020, 34th Conference on Neural Information Processing Systems (NeurIPS 2020))
5. X -Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

   Sobal, Vlad; Ibrahim, Mark; Balestriero, Randall; Cabannes, Vivien; Bouchacourt, Diane; Astolfi, Pietro; Cho, Kyunghyun; LeCun, Yann (April 2025, The International Conference on Learning Representations (ICLR 2025))

   Learning good representations involves capturing the diverse ways in which data samples relate. Contrastive loss - an objective matching related samples - underlies methods from self-supervised to multimodal learning. Contrastive losses, however, can be viewed more broadly as modifying a similarity graph to indicate how samples should relate in the embedding space. This view reveals a shortcoming in contrastive learning: the similarity graph is binary, as only one sample is the related positive sample. Crucially, similarities \textit{across} samples are ignored. Based on this observation, we revise the standard contrastive loss to explicitly encode how a sample relates to others. We experiment with this new objective, called X -Sample Contrastive, to train vision models based on similarities in class or text caption descriptions. Our study spans three scales: ImageNet-1k with 1 million, CC3M with 3 million, and CC12M with 12 million samples. The representations learned via our objective outperform both contrastive self-supervised and vision-language models trained on the same data across a range of tasks. When training on CC12M, we outperform CLIP by on both ImageNet and ImageNet Real. Our objective appears to work particularly well in lower-data regimes, with gains over CLIP of on ImageNet and on ImageNet Real when training with CC3M. Finally, our objective seems to encourage the model to learn representations that separate objects from their attributes and backgrounds, with gains of - \% over CLIP on ImageNet9. We hope the proposed solution takes a small step towards developing richer learning objectives for understanding sample relations in foundation models. 

   more » « less