More Like this

1. Multiview Cross-supervision for Semantic Segmentation

   Yao, Y. ; Park, H.S. ( January 2020 , IEEE Winter Conference on Applications of Computer Vision)

   This paper presents a semi-supervised learning framework for a customized semantic segmentation task using multiview image streams. A key challenge of the customized task lies in the limited accessibility of the labeled data due to the requirement of prohibitive manual annotation effort. We hypothesize that it is possible to leverage multiview image streams that are linked through the underlying 3D geometry, which can provide an additional supervisionary signal to train a segmentation model. We formulate a new cross-supervision method using a shape belief transfer---the segmentation belief in one image is used to predict that of the other image through epipolarmore »geometry analogous to shape-from-silhouette. The shape belief transfer provides the upper and lower bounds of the segmentation for the unlabeled data where its gap approaches asymptotically to zero as the number of the labeled views increases. We integrate this theory to design a novel network that is agnostic to camera calibration, network model, and semantic category and bypasses the intermediate process of suboptimal 3D reconstruction. We validate this network by recognizing a customized semantic category per pixel from realworld visual data including non-human species and a subject of interest in social videos where attaining large-scale annotation data is infeasible.« less
2. Unsupervised Gaze Prediction in Egocentric Videos by Energy-based Surprise Modeling

   Aakur, Sathyanarayanan N. ; Bagavathi, Arunkumar ( April 2021 , International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications)

   Egocentric perception has grown rapidly with the advent of immersive computing devices. Human gaze prediction is an important problem in analyzing egocentric videos and has primarily been tackled through either saliency-based modeling or highly supervised learning. We quantitatively analyze the generalization capabilities of supervised, deep learning models on the egocentric gaze prediction task on unseen, out-of-domain data. We find that their performance is highly dependent on the training data and is restricted to the domains specified in the training annotations. In this work, we tackle the problem of jointly predicting human gaze points and temporal segmentation of egocentric videos withoutmore »using any training data. We introduce an unsupervised computational model that draws inspiration from cognitive psychology models of event perception. We use Grenander's pattern theory formalism to represent spatial-temporal features and model surprise as a mechanism to predict gaze fixation points. Extensive evaluation on two publicly available datasets - GTEA and GTEA+ datasets-shows that the proposed model can significantly outperform all unsupervised baselines and some supervised gaze prediction baselines. Finally, we show that the model can also temporally segment egocentric videos with a performance comparable to more complex, fully supervised deep learning baselines.« less
3. A Continual Learning Framework for Uncertainty-Aware Interactive Image Segmentation

   Zheng, Ervine ; Yu, Qi ; Li, Rui ; Shi, Pengcheng ; Haake, Anne ( May 2021 , Proceedings of the AAAI Conference on Artificial Intelligence)

   Deep learning models have achieved state-of-the-art performance in semantic image segmentation, but the results provided by fully automatic algorithms are not always guaranteed satisfactory to users. Interactive segmentation offers a solution by accepting user annotations on selective areas of the images to refine the segmentation results. However, most existing models only focus on correcting the current image’s misclassified pixels, with no knowledge carried over to other images. In this work, we formulate interactive image segmentation as a continual learning problem and propose a framework to effectively learn from user annotations, aiming to improve the segmentation on both the current imagemore »and unseen images in future tasks while avoiding deteriorated performance on previously-seen images. It employs a probabilistic mask to control the neural network’s kernel activation and extract the most suitable features for segmenting images in each task. We also design a task-aware architecture to automatically infer the optimal kernel activation for initial segmentation and subsequent refinement. Interactions with users are guided through multi-source uncertainty estimation so that users can focus on the most important areas to minimize the overall manual annotation effort. Extensive experiments are performed on both medical and natural image datasets to illustrate the proposed framework’s effectiveness on basic segmentation performance, forward knowledge transfer, and backward knowledge transfer.« less
4. PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding

   Xie, Saining ; Gu, Jiatao ; Guo, Demi ; Qi, Charles R. ; Guibas, Leonidas ; Litany, Or ( December 2020 , European Conference on Computer Vision)

   Arguably one of the top success stories of deep learning is transfer learning. The finding that pre-training a network on a rich source set (e.g., ImageNet) can help boost performance once fine-tuned on a usually much smaller target set, has been instrumental to many applications in language and vision. Yet, very little is known about its usefulness in 3D point cloud understanding. We see this as an opportunity considering the effort required for annotating data in 3D. In this work, we aim at facilitating research on 3D representation learning. Different from previous works, we focus on high-level scene understanding tasks.more »To this end, we select a suit of diverse datasets and tasks to measure the effect of unsupervised pre-training on a large source set of 3D scenes. Our findings are extremely encouraging: using a unified triplet of architecture, source dataset, and contrastive loss for pre-training, we achieve improvement over recent best results in segmentation and detection across 6 different benchmarks for indoor and outdoor, real and synthetic datasets – demonstrating that the learned representation can generalize across domains. Furthermore, the improvement was similar to supervised pre-training, suggesting that future efforts should favor scaling data collection over more detailed annotation. We hope these findings will encourage more research on unsupervised pretext task design for 3D deep learning.« less
5. JOINT SEGMENTATION AND FINE-GRAINED CLASSIFICATION OF NUCLEI IN HISTOPATHOLOGY IMAGES

   Qu, H. ; Riedlinger, G. ; Wu, P. ; Huang, Q. ; Yi, J. ; De, S. ; Metaxas, D. ( April 2019 , 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019))

   Nuclei segmentation and classification are two important tasks in the histopathology image analysis, because the mor- phological features of nuclei and spatial distributions of dif- ferent types of nuclei are highly related to cancer diagnosis and prognosis. Existing methods handle the two problems independently, which are not able to obtain the features and spatial heterogeneity of different types of nuclei at the same time. In this paper, we propose a novel deep learning based method which solves both tasks in a unified framework. It can segment individual nuclei and classify them into tumor, lymphocyte and stroma nuclei. Perceptual loss ismore »utilized to enhance the segmentation of details. We also take advantages of transfer learning to promote the training of deep neural net- works on a relatively small lung cancer dataset. Experimental results prove the effectiveness of the proposed method. The code is publicly available« less