An official website of the United States government
We introduce a new neural signal model designed for efficient high-resolution representation of large-scale signals. The key innovation in our multiscale implicit neural representation (MINER) is an internal representation via a Laplacian pyramid, which provides a sparse multiscale decomposition of the signal that captures orthogonal parts of the signal across scales. We leverage the advantages of the Laplacian pyramid by representing small disjoint patches of the pyramid at each scale with a small MLP. This enables the capacity of the network to adaptively increase from coarse to fine scales, and only represent parts of the signal with strong signal energy. The parameters of each MLP are optimized from coarse-to-fine scale which results in faster approximations at coarser scales, thereby ultimately an extremely fast training process. We apply MINER to a range of large-scale signal representation tasks, including gigapixel images and very large point clouds, and demonstrate that it requires fewer than 25% of the parameters, 33% of the memory footprint, and 10% of the computation time of competing techniques such as ACORN to reach the same representation accuracy.
more »
« less
LAPRAN: A Scalable Laplacian Pyramid Reconstructive Adversarial Network for Flexible Compressive Sensing Reconstruction
This paper addresses the single-image compressive sensing (CS) and reconstruction problem. We propose a scalable Laplacian pyramid reconstructive adversarial network (LAPRAN) that enables high-fidelity, flexible and fast CS images reconstruction. LAPRAN progressively reconstructs an image following the concept of the Laplacian pyramid through multiple stages of reconstructive adversarial networks (RANs). At each pyramid level, CS measurements are fused with a contextual latent vector to generate a high-frequency image residual. Consequently, LAPRAN can produce hierarchies of reconstructed images and each with an incremental resolution and improved quality. The scalable pyramid structure of LAPRAN enables high-fidelity CS reconstruction with a flexible resolution that is adaptive to a wide range of compression ratios (CRs), which is infeasible with existing methods. Experimental results on multiple public datasets show that LAPRAN offers an average 7.47dB and 5.98dB PSNR, and an average 57.93% and 33.20% SSIM improvement compared to model-based and data-driven baselines, respectively.
more »
« less
- Award ID(s):
- 1652038
- PAR ID:
- 10084500
- Date Published:
- Journal Name:
- The European Conference on Computer Vision (ECCV)
- Page Range / eLocation ID:
- 485-500
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Atomic Force Microscopy (AFM) can create images of biomolecules under near-native conditions but suffers from limited lateral resolution due to the finite AFM tip size and recording frequency. The recently developed Localization Atomic Force Microscopy or LAFM (Heath et al., Nature 594, 385 (2021)) enhances lateral resolution by reconstructing peak positions in AFM image stacks, but it is less effective for flexible proteins with multiple conformations. Here we introduce an unsupervised deep learning algorithm that simultaneously registers and clusters images by protein conformation, thus making LAFM applicable to more flexible proteins. Using simulated AFM images from molecular dynamics simulations of the SecYEG translocon as a model membrane protein system, we demonstrate improved resolution for individual protein conformations. This work represents a step towards a more general LAFM algorithm that can handle biological macromolecules with multiple distinct conformational states such as SecYEG. Author summaryAtomic Force Microscopy (AFM) enables high-resolution imaging of biomolecules under near-native conditions but faces lateral resolution limits due to the finite AFM tip size and recording frequency. The recently developed Localization Atomic Force Microscopy (LAFM) method addresses this by reconstructing peak positions from AFM image stacks, achieving almost atomic resolution for rigid proteins like bacteriorhodopsin (Heath et al., Nature 594, 385 (2021)). However, flexible membrane proteins with dynamic conformations, such as the SecYEG translocon, which exhibits large and highly mobile cytoplasmic loops, lead to non-physical smearing in standard LAFM reconstructions. Here, we present a computational framework combining unsupervised deep clustering and LAFM to enhance the lateral resolution of AFM images of flexible membrane proteins. Our neural network algorithm (i) groups AFM images into conformationally homogeneous clusters and (ii) registers images within each cluster. Applying LAFM separately to these clusters minimizes smearing artifacts, yielding high-resolution reconstructions for distinct conformations. We validate this approach using synthetic AFM images generated from all-atom molecular dynamics simulations of SecYEG in a solvated POPE lipid bilayer. This advancement extends LAFM’s utility to encompass conformationally diverse membrane proteins.more » « less
-
Chen, Xi (Ed.)In patients with dense breasts or at high risk of breast cancer, dynamic contrast enhanced MRI (DCE-MRI) is a highly sensitive diagnostic tool. However, its specificity is highly variable and sometimes low; quantitative measurements of contrast uptake parameters may improve specificity and mitigate this issue. To improve diagnostic accuracy, data need to be captured at high spatial and temporal resolution. While many methods exist to accelerate MRI temporal resolution, not all are optimized to capture breast DCE-MRI dynamics. We propose a novel, flexible, and powerful framework for the reconstruction of highly-undersampled DCE-MRI data: enhancement-constrained acceleration (ECA). Enhancement-constrained acceleration uses an assumption of smooth enhancement at small time-scale to estimate points of smooth enhancement curves in small time intervals at each voxel. This method is tested in silico with physiologically realistic virtual phantoms, simulating state-of-the-art ultrafast acquisitions at 3.5s temporal resolution reconstructed at 0.25s temporal resolution (demo code available here). Virtual phantoms were developed from real patient data and parametrized in continuous time with arterial input function (AIF) models and lesion enhancement functions. Enhancement-constrained acceleration was compared to standard ultrafast reconstruction in estimating the bolus arrival time and initial slope of enhancement from reconstructed images. We found that the ECA method reconstructed images at 0.25s temporal resolution with no significant loss in image fidelity, a 4x reduction in the error of bolus arrival time estimation in lesions ( p < 0.01) and 11x error reduction in blood vessels ( p < 0.01). Our results suggest that ECA is a powerful and versatile tool for breast DCE-MRI.more » « less
-
Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment in fields such as cardiology and ophthalmology. Such applications can be further facilitated by deep learning-based super-resolution technology, which improves the capability of resolving morphological structures. However, existing deep learning-based method only focuses on spatial distribution and disregards frequency fidelity in image reconstruction, leading to a frequency bias. To overcome this limitation, we propose a frequency-aware super-resolution framework that integrates three critical frequency-based modules (i.e., frequency transformation, frequency skip connection, and frequency alignment) and frequency-based loss function into a conditional generative adversarial network (cGAN). We conducted a large-scale quantitative study from an existing coronary OCT dataset to demonstrate the superiority of our proposed framework over existing deep learning frameworks. In addition, we confirmed the generalizability of our framework by applying it to fish corneal images and rat retinal images, demonstrating its capability to super-resolve morphological details in eye imaging.more » « less
-
The sparse interferometric coverage of the Event Horizon Telescope (EHT) poses a significant challenge for both reconstruction and model fitting of black-hole images. PRIMO is a new principal components analysis-based algorithm for image reconstruction that uses the results of high-fidelity general relativistic, magnetohydrodynamic simulations of low-luminosity accretion flows as a training set. This allows the reconstruction of images that are both consistent with the interferometric data and that live in the space of images that is spanned by the simulations. PRIMO follows Monty Carlo Markov Chains to fit a linear combination of principal components derived from an ensemble of simulated images to interferometric data. We show that PRIMO can efficiently and accurately reconstruct synthetic EHT data sets for several simulated images, even when the simulation parameters are significantly different from those of the image ensemble that was used to generate the principal components. The resulting reconstructions achieve resolution that is consistent with the performance of the array and do not introduce significant biases in image features such as the diameter of the ring of emission.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
