skip to main content

Title: MINER: Multiscale Implicit Neural Representation
We introduce a new neural signal model designed for efficient high-resolution representation of large-scale signals. The key innovation in our multiscale implicit neural representation (MINER) is an internal representation via a Laplacian pyramid, which provides a sparse multiscale decomposition of the signal that captures orthogonal parts of the signal across scales. We leverage the advantages of the Laplacian pyramid by representing small disjoint patches of the pyramid at each scale with a small MLP. This enables the capacity of the network to adaptively increase from coarse to fine scales, and only represent parts of the signal with strong signal energy. The parameters of each MLP are optimized from coarse-to-fine scale which results in faster approximations at coarser scales, thereby ultimately an extremely fast training process. We apply MINER to a range of large-scale signal representation tasks, including gigapixel images and very large point clouds, and demonstrate that it requires fewer than 25% of the parameters, 33% of the memory footprint, and 10% of the computation time of competing techniques such as ACORN to reach the same representation accuracy.  more » « less
Award ID(s):
1911094 1730574 1838177
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
European Conference on Computer Vision (ECCV) 2022
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The structure of neural circuitry plays a crucial role in brain function. Previous studies of brain organization generally had to trade off between coarse descriptions at a large scale and fine descriptions on a small scale. Researchers have now reconstructed tens to hundreds of thousands of neurons at synaptic resolution, enabling investigations into the interplay between global, modular organization, and cell type-specific wiring. Analyzing data of this scale, however, presents unique challenges. To address this problem, we applied novel community detection methods to analyze the synapse-level reconstruction of an adult femaleDrosophila melanogasterbrain containing >20,000 neurons and 10 million synapses. Using a machine-learning algorithm, we find the most densely connected communities of neurons by maximizing a generalized modularity density measure. We resolve the community structure at a range of scales, from large (on the order of thousands of neurons) to small (on the order of tens of neurons). We find that the network is organized hierarchically, and larger-scale communities are composed of smaller-scale structures. Our methods identify well-known features of the fly brain, including its sensory pathways. Moreover, focusing on specific brain regions, we are able to identify subnetworks with distinct connectivity types. For example, manual efforts have identified layered structures in the fan-shaped body. Our methods not only automatically recover this layered structure, but also resolve finer connectivity patterns to downstream and upstream areas. We also find a novel modular organization of the superior neuropil, with distinct clusters of upstream and downstream brain regions dividing the neuropil into several pathways. These methods show that the fine-scale, local network reconstruction made possible by modern experimental methods are sufficiently detailed to identify the organization of the brain across scales, and enable novel predictions about the structure and function of its parts.

    Significance StatementThe Hemibrain is a partial connectome of an adult femaleDrosophila melanogasterbrain containing >20,000 neurons and 10 million synapses. Analyzing the structure of a network of this size requires novel and efficient computational tools. We applied a new community detection method to automatically uncover the modular structure in the Hemibrain dataset by maximizing a generalized modularity measure. This allowed us to resolve the community structure of the fly hemibrain at a range of spatial scales revealing a hierarchical organization of the network, where larger-scale modules are composed of smaller-scale structures. The method also allowed us to identify subnetworks with distinct cell and connectivity structures, such as the layered structures in the fan-shaped body, and the modular organization of the superior neuropil. Thus, network analysis methods can be adopted to the connectomes being reconstructed using modern experimental methods to reveal the organization of the brain across scales. This supports the view that such connectomes will allow us to uncover the organizational structure of the brain, which can ultimately lead to a better understanding of its function.

    more » « less
  2. In this paper, we investigate and design multiscale simulations for stochastic multiscale PDEs. As for the space, we consider a coarse grid and a known multiscale method, the generalized multiscale finite element method (GMsFEM). In order to obtain a small dimensional representation of the solution in each coarse block, the uncertainty space needs to be partitioned (coarsened). This coarsenining collects realizations that provide similar multiscale features as outlined in GMsFEM (or other method of choice). This step is known to be computationally demanding as it requires many local solves and clustering based on them. In this work, we take a different approach and learn coarsening the uncertainty space. Our methods use deep learning techniques in identifying clusters (coarsening) in the uncertainty space. We use convolutional neural networks combined with some techniques in adversary neural networks. We define appropriate loss functions in the proposed neural networks, where the loss function is composed of several parts that includes terms related to clusters and reconstruction of basis functions. We present numerical results for channelized permeability fields in the examples of flows in porous media. 
    more » « less
  3. Abstract. Polar sea ice is a critical component of Earth’s climate system. As a material, it is a multiscale composite of pure ice with temperature-dependent millimeter-scale brine inclusions, and centimeter-scale polycrystalline microstructure which is largely determined by how the ice was formed. The surface layer of the polar oceans can be viewed as a granular composite of ice floes in a sea water host, with floe sizes ranging from centimeters to tens of kilometers. A principal challenge in modeling sea ice and its role in climate is how to use information on smaller-scale structures to find the effective or homogenized properties on larger scales relevant to process studies and coarse-grained climate models. That is, how do you predict macroscopic behavior from microscopic laws, like in statistical mechanics and solid state physics? Also of great interest in climate science is the inverse problem of recovering parameters controlling small-scale processes from large-scale observations. Motivated by sea ice remote sensing, the analytic continuation method for obtaining rigorous bounds on the homogenized coefficients of two-phase composites was applied to the complex permittivity of sea ice, which is a Stieltjes function of the ratio of the permittivities of ice and brine. Integral representations for the effective parameters distill the complexities of the composite microgeometry into the spectral properties of a self-adjoint operator like the Hamiltonian in quantum physics. These techniques have been extended to polycrystalline materials, advection diffusion processes, and ocean waves in the sea ice cover. Here we discuss this powerful approach in homogenization, highlighting the spectral representations and resolvent structure of the fields that are shared by the two-component theory and its extensions. Spectral analysis of sea ice structures leads to a random matrix theory picture of percolation processes in composites, establishing parallels to Anderson localization and semiconductor physics and providing new insights into the physics of sea ice. 
    more » « less
  4. null (Ed.)
    A turbulence enrichment model for subfilter-scale motions in large eddy simulations (LES) is comprehensively evaluated in the context of a posteriori analysis. The paper further develops the Gabor mode enrichment model first introduced in Ghate & Lele (J. Fluid Mech., vol. 819, 2017, pp. 494–539) by analysing three key requisites of LES enrichment using solenoidal small-scale velocity fields: (a) consistent spectral extrapolation and improvement of resolved single- and two-point second-order correlations; (b) ability to accurately capture the flow physics responsible for temporal decorrelation at small scales; and (c) accurate representation of spatially localized and intermittent interscale energy transfer between scales resolved by the coarse-grid LES and subfilter scales. We argue that the spatially and spectrally localized Gabor wavepackets offer an optimal basis to represent small-scale turbulence within quasi-homogeneous regions, although the alignment of fine-scale vorticity with large-scale strain appears to be somewhat overemphasized. Consequently, we interpret the resulting subfilter scales as those induced by a set of spatially dispersed Burgers–Townsend vortices with orientations determined by the larger scale velocity gradients resolved by the coarse-grid LES. Enrichment of coarse-grid simulations of two high Reynolds number flow configurations, homogeneous isotropic turbulence and a rough-wall turbulent boundary layer show promising results. 
    more » « less
  5. Abstract

    Satellite precipitation products, as all quantitative estimates, come with some inherent degree of uncertainty. To associate a quantitative value of the uncertainty to each individual estimate, error modeling is necessary. Most of the error models proposed so far compute the uncertainty as a function of precipitation intensity only, and only at one specific spatiotemporal scale. We propose a spectral error model that accounts for the neighboring space–time dynamics of precipitation into the uncertainty quantification. Systematic distortions of the precipitation signal and random errors are characterized distinctively in every frequency–wavenumber band in the Fourier domain, to accurately characterize error across scales. The systematic distortions are represented as a deterministic space–time linear filtering term. The random errors are represented as a nonstationary additive noise. The spectral error model is applied to the IMERG multisatellite precipitation product, and its parameters are estimated empirically through a system identification approach using the GV-MRMS gauge–radar measurements as reference (“truth”) over the eastern United States. The filtering term is found to be essentially low-pass (attenuating the fine-scale variability). While traditional error models attribute most of the error variance to random errors, it is found here that the systematic filtering term explains 48% of the error variance at the native resolution of IMERG. This fact confirms that, at high resolution, filtering effects in satellite precipitation products cannot be ignored, and that the error cannot be represented as a purely random additive or multiplicative term. An important consequence is that precipitation estimates derived from different sources shall not be expected to automatically have statistically independent errors.

    Significance Statement

    Satellite precipitation products are nowadays widely used for climate and environmental research, water management, risk analysis, and decision support at the local, regional, and global scales. For all these applications, knowledge about the accuracy of the products is critical for their usability. However, products are not systematically provided with a quantitative measure of the uncertainty associated with each individual estimate. Various parametric error models have been proposed for uncertainty quantification, mostly assuming that the uncertainty is only a function of the precipitation intensity at the pixel and time of interest. By projecting satellite precipitation fields and their retrieval errors into the Fourier frequency–wavenumber domain, we show that we can explicitly take into account the neighboring space–time multiscale dynamics of precipitation and compute a scale-dependent uncertainty.

    more » « less