skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Joint detection of motion boundaries and occlusions
We propose MONet, a convolutional neural network that jointly detects motion boundaries and occlusion regions in video both forward and backward in time. Detection is difficult because optical flow is discontinuous along motion boundaries and undefined in occlusion regions, while many flow estimators assume smoothness and a flow defined everywhere. To reason in the two time directions simultaneously, we direct-warp the estimated maps between the two frames. Since appearance mismatches between frames often signal vicinity to motion boundaries or occlusion regions, we construct a cost block that for each feature in one frame records the lowest discrepancy with matching features in a search range. This cost block is two-dimensional, and much less expensive than the four-dimensional cost volumes used in flow analysis. Cost-block features are computed by an encoder, and motion boundary and occlusion region estimates are computed by a decoder. We found that arranging decoder layers fine-to- coarse, rather than coarse-to-fine, improves performance. MONet outperforms the prior state of the art for both tasks on the Sintel and FlyingChairsOcc benchmarks without any fine-tuning on them.  more » « less
Award ID(s):
1909821
PAR ID:
10377792
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
British Machine Vision Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, ECCV 2020)
    null (Ed.)
    We propose a novel end-to-end deep scene flow model, called PointPWC-Net, that directly processes 3D point cloud scenes with large motions in a coarse-to-fine fashion. Flow computed at the coarse level is upsampled and warped to a finer level, enabling the algorithm to accommodate for large motion without a prohibitive search space. We introduce novel cost volume, upsampling, and warping layers to efficiently handle 3D point cloud data. Unlike traditional cost volumes that require exhaustively computing all the cost values on a high-dimensional grid, our point-based formulation discretizes the cost volume onto input 3D points, and a PointConv operation efficiently computes convolutions on the cost volume. Experiment results on FlyingThings3D and KITTI outperform the state-of-the-art by a large margin. We further explore novel self-supervised losses to train our model and achieve comparable results to state-of-the-art trained with supervised loss. Without any fine-tuning, our method also shows great generalization ability on the KITTI Scene Flow 2015 dataset, outperforming all previous methods. The code is released at https://github.com/DylanWusee/PointPWC. 
    more » « less
  2. ; ; ; ; (, Mathematics)
    In this paper, we consider unsaturated filtration in heterogeneous porous media with rough surface topography. The surface topography plays an important role in determining the flow process and includes multiscale features. The mathematical model is based on the Richards’ equation with three different types of boundary conditions on the surface: Dirichlet, Neumann, and Robin boundary conditions. For coarse-grid discretization, the Generalized Multiscale Finite Element Method (GMsFEM) is used. Multiscale basis functions that incorporate small scale heterogeneities into the basis functions are constructed. To treat rough boundaries, we construct additional basis functions to take into account the influence of boundary conditions on rough surfaces. We present numerical results for two-dimensional and three-dimensional model problems. To verify the obtained results, we calculate relative errors between the multiscale and reference (fine-grid) solutions for different numbers of multiscale basis functions. We obtain a good agreement between fine-grid and coarse-grid solutions. 
    more » « less
  3. ; ; ; ; ; (, Journal of Computational Physics)
    This paper develops a tree-topological local mesh refinement (TLMR) method on Cartesian grids for the simulation of bio-inspired flow with multiple moving objects. The TLMR nests refinement mesh blocks of structured grids to the target regions and arrange the blocks in a tree topology. The method solves the time-dependent incompressible flow using a fractional-step method and discretizes the Navier-Stokes equation using a finite-difference formulation with an immersed boundary method to resolve the complex boundaries. When iteratively solving the discretized equations across the coarse and fine TLMR blocks, for better accuracy and faster convergence, the momentum equation is solved on all blocks simultaneously, while the Poisson equation is solved recursively from the coarsest block to the finest ones. When the refined blocks of the same block are connected, the parallel Schwarz method is used to iteratively solve both the momentum and Poisson equations. Convergence studies show that the algorithm is second-order accurate in space for both velocity and pressure, and the developed mesh refinement technique is benchmarked and demonstrated by several canonical flow problems. The TLMR enables a fast solution to an incompressible flow problem with complex boundaries or multiple moving objects. Various bio-inspired flows of multiple moving objects show that the solver can save over 80% computational time, proportional to the grid reduction when refinement is applied. 
    more » « less
  4. ; ; ; (, Journal of Geophysical Research: Planets)
    Abstract The lunar surface contains a wide variety of topographic shapes and features, each with different distributions and scales, and any analysis technique to objectively measure roughness must respect these qualities. Coarse‐graining is a naturally scale‐dependent filtering technique that preserves scale‐dependent symmetries and produces coarse elevation maps that gradually erase the smaller features from the original topography. In this study of the lunar surface, we present two surface variability metrics obtained from coarse‐graining lunar topography: fine elevation and coarse curvature. Both metrics are isotropic, deterministic, slope‐independent, and coordinate‐agnostic. Fine (detrended) elevation is acquired by subtracting the coarse elevation from the original topography and contains features that are smaller than the coarse‐graining length‐scale. Coarse curvature is the Laplacian of coarsened topography, and naturally quantifies the curvature at any scale and indicates whether a location is elevated or depressed relative to its neighborhood at that scale. We find that highlands and maria have distinct roughness characteristics at all length‐scales. Our topographic spectra reveal four scale‐breaks that mark characteristic shifts in surface roughness: 100, 300, 1,000, and 4,000 km. Comparing fine elevation distributions between maria and highlands, we show that maria fine elevation is biased toward smaller‐magnitude elevations and that the maria–highland discrepancies are more pronounced at larger length‐scales. We also provide local examples of selected regions to demonstrate that these metrics can successfully distinguish geological features of different length‐scales. 
    more » « less
  5. ; ; ; ; ; ; ; (, The Journal of Neuroscience)
    The functional connectome supports information transmission through the brain at various spatial scales, from exchange between broad cortical regions to finer-scale, vertex-wise connections that underlie specific information processing mechanisms. In adults, while both the coarse- and fine-scale functional connectomes predict cognition, the fine scale can predict up to twice the variance as the coarse-scale functional connectome. Yet, past brain-wide association studies, particularly using large developmental samples, focus on the coarse connectome to understand the neural underpinnings of individual differences in cognition. Using a large cohort of children (age 9–10 years;n = 1,115 individuals; both sexes; 50% female, including 170 monozygotic and 219 dizygotic twin pairs and 337 unrelated individuals), we examine the reliability, heritability, and behavioral relevance of resting-state functional connectivity computed at different spatial scales. We use connectivity hyperalignment to improve access to reliable fine-scale (vertex-wise) connectivity information and compare the fine-scale connectome with the traditional parcel-wise (coarse scale) functional connectomes. Though individual differences in the fine-scale connectome are more reliable than those in the coarse-scale, they are less heritable. Further, the alignment and scale of connectomes influence their ability to predict behavior, whereby some cognitive traits are equally well predicted by both connectome scales, but other, less heritable cognitive traits are better predicted by the fine-scale connectome. Together, our findings suggest there are dissociable individual differences in information processing represented at different scales of the functional connectome which, in turn, have distinct implications for heritability and cognition. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email