More Like this

1. Multi-layered self-attention mechanism for weakly supervised semantic segmentation

   https://doi.org/10.1016/j.cviu.2023.103886  

   Yaganapu, Avinash; Kang, Mingon (February 2024, Computer Vision and Image Understanding)

   Weakly Supervised Semantic Segmentation (WSSS) provides efficient solutions for semantic image segmentation using image-level annotations. WSSS requires no pixel-level labeling that Fully Supervised Semantic Segmentation (FSSS) does, which is time-consuming and label-intensive. Most WSSS approaches have leveraged Class Activation Maps (CAM) or Self-Attention (SA) to generate pseudo pixel-level annotations to perform semantic segmentation tasks coupled with fully supervised approaches (e.g., Fully Convolutional Network). However, those approaches often provides incomplete supervision that mainly includes discriminative regions from the last convolutional layer. They may fail to capture regions of low- or intermediate-level features that may not be present in the last convolutional layer. To address the issue, we proposed a novel Multi-layered Self-Attention (Multi-SA) method that applies a self-attention module to multiple convolutional layers, and then stack feature maps from the self-attention layers to generate pseudo pixel-level annotations. We demonstrated that integrated feature maps from multiple self-attention layers produce higher coverage in semantic segmentation than using only the last convolutional layer through intensive experiments using standard benchmark datasets. 

   more » « less
2. Weakly Supervised Deep Nuclei Segmentation using Points Annotation in Histopathology Images

   Qu, H; Wu, P.; Huang, Q.; Yi, J.; Riedlinger, G. M.; De, S.; Metaxas, D. (July 2019, Proceedings of Machine Learning Research)

   Nuclei segmentation is a fundamental task in histopathological image analysis. Typically, such segmentation tasks require significant effort to manually generate pixel-wise annotations for fully supervised training. To alleviate the manual effort, in this paper we propose a novel approach using points only annotation. Two types of coarse labels with complementary information are derived from the points annotation, and are then utilized to train a deep neural network. The fully- connected conditional random field loss is utilized to further refine the model without introducing extra computational complexity during inference. Experimental results on two nuclei segmentation datasets reveal that the proposed method is able to achieve competitive performance compared to the fully supervised counterpart and the state-of-the-art methods while requiring significantly less annotation effort. Our code is publicly available. 

   more » « less
3. Adaptive Segmentation of Thermal Anomalies in Buildings Using Human-in-the-Loop Active Learning Capsule Networks

   https://doi.org/10.7771/3067-4883.1830  

   Wen, Yining; Chen, Kaiwen (June 2025, CIB Conferences)

   Buildings account for 40% of energy consumption in the U.S., with significant energy losses stemming from poor insulation and leaks. To meet decarbonization goals by 2050, efficient methods for diagnosing and retrofitting thermal anomalies are essential. Infrared thermography (IRT), combined with emerging technologies like computer vision and deep learning, offers the potential for automating thermal anomaly detection and segmentation. However, challenges such as building diversity, scenario variations, and labor-intensive image annotation hinder model reliability and robustness. This study proposes a human-in-the-loop active learning approach to fine-tune the pretrained Capsule-based network (CapsLab) to enhance adaptability to new building scenarios efficiently. Using a Query-by-Committee (QBC) strategy, the method selects the most informative thermal images from a target dataset for expert verification and iterative model refinement. To streamline the expert annotation verification process, a weakly annotation strategy is introduced supporting human-in-the-loop training by Simple Linear Iterative Clustering (SLIC)-based superpixel segmentation and scribble prompts for efficient labeling. The pretrained CapsLab model is iteratively fine-tuned using these verified annotations to enhance segmentation performance. Three fine-tuning methods, including parameter freezing strategies, are evaluated for optimal results. This iterative workflow reduces annotation effort, improves model adaptability, and improves the precision of thermal anomaly segmentation, facilitating energy-efficient building retrofitting and maintenance. 

   more » « less
4. Domain adaptation for supervised integration of scRNA-seq data

   https://doi.org/10.1038/s42003-023-04668-7  

   Sun, Yutong; Qiu, Peng (March 2023, Communications Biology)

   Abstract Large-scale scRNA-seq studies typically generate data in batches, which often induce nontrivial batch effects that need to be corrected. Given the global efforts for building cell atlases and the increasing number of annotated scRNA-seq datasets accumulated, we propose a supervised strategy for scRNA-seq data integration called SIDA (SupervisedIntegration usingDomainAdaptation), which uses the cell type annotations to guide the integration of diverse batches. The supervised strategy is based on domain adaptation that was initially proposed in the computer vision field. We demonstrate that SIDA is able to generate comprehensive reference datasets that lead to improved accuracy in automated cell type mapping analyses. 

   more » « less
5. Joint Semi-supervised and Active Learning for Segmentation of Gigapixel Pathology Images with Cost-Effective Labeling

   https://doi.org/10.1109/ICCVW54120.2021.00072  

   Lai, Zhengfeng; Wang, Chao; Oliveira, Luca Cerny; Dugger, Brittany N.; Cheung, Sen-Ching; Chuah, Chen-Nee (October 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW))

   The need for manual and detailed annotations limits the applicability of supervised deep learning algorithms in medical image analyses, specifically in the field of pathology. Semi-supervised learning (SSL) provides an effective way for leveraging unlabeled data to relieve the heavy reliance on the amount of labeled samples when training a model. Although SSL has shown good performance, the performance of recent state-of-the-art SSL methods on pathology images is still under study. The problem for selecting the most optimal data to label for SSL is not fully explored. To tackle this challenge, we propose a semi-supervised active learning framework with a region-based selection criterion. This framework iteratively selects regions for an-notation query to quickly expand the diversity and volume of the labeled set. We evaluate our framework on a grey-matter/white-matter segmentation problem using gigapixel pathology images from autopsied human brain tissues. With only 0.1% regions labeled, our proposed algorithm can reach a competitive IoU score compared to fully-supervised learning and outperform the current state-of-the-art SSL by more than 10% of IoU score and DICE coefficient. 

   more » « less