skip to main content


Title: Learning Instance Occlusion for Panoptic Segmentation
Panoptic segmentation requires segments of both “things” (countable object instances) and “stuff” (uncountable and amorphous regions) within a single output. A common approach involves the fusion of instance segmentation (for “things”) and semantic segmentation (for “stuff”) into a non-overlapping placement of segments, and resolves overlaps. However, instance ordering with detection confidence do not correlate well with natural occlusion relationship. To resolve this issue, we propose a branch that is tasked with modeling how two instance masks should overlap one another as a binary relation. Our method, named OCFusion, is lightweight but particularly effective in the instance fusion process. OCFusion is trained with the ground truth relation derived automatically from the existing dataset annotations. We obtain state-of-the-art results on COCO and show competitive results on the Cityscapes panoptic segmentation benchmark.  more » « less
Award ID(s):
1717431 1618477
NSF-PAR ID:
10166842
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN:
2332-564X
Page Range / eLocation ID:
10720-10729
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In this paper, we explore the possibility to increase the training examples without laborious data collection and annotation for long-tailed instance segmentation. We find that an abundance of instance segments can potentially be obtained freely from object-centric images, according to two insights: (i) an object-centric image usually contains one salient object in a simple background; (ii) objects from the same class often share similar appearances or similar contrasts to the background. Motivated by these insights, we propose a simple and scalable framework FREESEG for extracting and leveraging these “free” object segments to facilitate model training. Concretely, we investigate the similarity among object-centric images of the same class to propose candidate segments of foreground instances, followed by a novel ranking of segment quality. The resulting high quality object segments can then be used to augment the existing long-tailed datasets, e.g., by copying and pasting the segments onto the original training images. Extensive experiments show that FREESEG yields substantial improvements on top of strong baselines and achieves state-of-the-art accuracy for segmenting rare object categories. 
    more » « less
  2. null (Ed.)
    The task of instance segmentation in videos aims to consistently identify objects at pixel level throughout the entire video sequence. Existing state-of-the-art methods either follow the tracking-bydetection paradigm to employ multi-stage pipelines or directly train a complex deep model to process the entire video clips as 3D volumes. However, these methods are typically slow and resourceconsuming such that they are often limited to offline processing. In this paper, we propose SRNet, a simple and efficient framework for joint segmentation and tracking of object instances in videos. The key to achieving both high efficiency and accuracy in our framework is to formulate the instance segmentation and tracking problem into a unified spatial-relation learning task where each pixel in the current frame relates to its object center, and each object center relates to its location in the previous frame. This unified learning framework allows our framework to perform join instance segmentation and tracking through a single stage while maintaining low overheads among different learning tasks. Our proposed framework can handle two different task settings and demonstrates comparable performance with state-of-the-art methods on two different benchmarks while running significantly faster. 
    more » « less
  3. Many seismic tomography investigations have imaged the East Antarctic lithosphere as a thick and continuous cratonic structure that is separated from the thinner lithosphere of the adjacent West Antarctic Rift System by the Transantarctic Mountains. However, recent studies have painted a more complicated picture, suggesting, for instance, a separate cratonic fragment beneath Dronning Maud Land and possible lithospheric delamination beneath the southern Transantarctic Mountains. In addition, patterns of intracratonic seismicity have been identified near the Gamburtsev Subglacial Mountains in East Antarctica, indicating possible rift zones in this region. That said, detailed imaging of the subsurface structure has remained challenging given the sparse distribution of seismic stations and the generally low seismicity rate throughout the interior of East Antarctica. Therefore, new approaches that can leverage existing seismic datasets to elucidate the Antarctic cratonic structure are vital. We are utilizing records of ambient seismic noise recorded by numerous temporary, moderate-term, and long-term seismic networks throughout Antarctica to improve the imaging of the lithospheric structure. Empirical Green’s Functions with periods of 40-340 seconds have been extracted using a frequency-time normalization approach, and these data are being used to constrain our full-waveform inversion. A finite-difference approach with a continental-scale, spherical grid is employed to numerically model synthetic seismograms, and a scattering integral method is used to construct the associated sensitivity kernels. Our initial results suggest that some portions of East Antarctica, particularly those beneath the Wilkes Subglacial Basin and the Aurora Basin, may have reduced shear-wave velocities that potentially indicate regions of thinner lithosphere. Further, possible segmentation may be present in the vicinity of the Gamburtsev Subglacial Mountains. Our new tomographic results will allow for further assessment of the East Antarctic tectonic structure and its relation to local seismicity. 
    more » « less
  4. null (Ed.)
    An important means for disseminating information in social media platforms is by including URLs that point to external sources in user posts. In Twitter, we estimate that about 21% of the daily stream of English-language tweets contain URLs. We notice that NLP tools make little attempt at understanding the relationship between the content of the URL and the text surrounding it in a tweet. In this work, we study the structure of tweets with URLs relative to the content of the Web documents pointed to by the URLs. We identify several segments classes that may appear in a tweet with URLs, such as the title of a Web page and the user's original content. Our goals in this paper are: introduce, define, and analyze the segmentation problem of tweets with URLs, develop an effective algorithm to solve it, and show that our solution can benefit sentiment analysis on Twitter. We also show that the problem is an instance of the block edit distance problem, and thus an NP-hard problem. 
    more » « less
  5. Abstract

    Variations in fault zone maturity have intermittently been invoked to explain variations in some seismological observations for large earthquakes. However, the lack of a unified geological definition of fault maturity makes quantitative assessment of its importance difficult. We evaluate the degree of empirical correlation between geological and geometric measurements commonly invoked as indicative of fault zone maturity and remotely measured seismological source parameters of 34MW ≥ 6.0 shallow strike‐slip events. Metrics based on surface rupture segmentation, such as number of segments and surface rupture azimuth changes, correlate best with seismic source attributes while the correlations with cumulative fault slip are weaker. Average rupture velocity shows the strongest correlation with metrics of maturity, followed by relative aftershock productivity. Mature faults have relatively lower aftershock productivity and higher rupture velocity. A more complex relation is found with moment‐scaled radiated energy. There appears to be distinct behavior of very immature events which radiate modest seismic energy, while intermediate mature faults have events with higher moment‐scaled radiated energy and very mature faults with increasing cumulative slip tend to have events with reduced moment‐scaled radiated energy. These empirical comparisons establish that there are relationships between remote seismological observations and fault system maturity that can help to understand variations in seismic hazard among different fault environments and to assess the relative maturity of inaccessible or blind fault systems for which direct observations of maturity are very limited.

     
    more » « less