An official website of the United States government
We introduce a new neural signal model designed for efficient high-resolution representation of large-scale signals. The key innovation in our multiscale implicit neural representation (MINER) is an internal representation via a Laplacian pyramid, which provides a sparse multiscale decomposition of the signal that captures orthogonal parts of the signal across scales. We leverage the advantages of the Laplacian pyramid by representing small disjoint patches of the pyramid at each scale with a small MLP. This enables the capacity of the network to adaptively increase from coarse to fine scales, and only represent parts of the signal with strong signal energy. The parameters of each MLP are optimized from coarse-to-fine scale which results in faster approximations at coarser scales, thereby ultimately an extremely fast training process. We apply MINER to a range of large-scale signal representation tasks, including gigapixel images and very large point clouds, and demonstrate that it requires fewer than 25% of the parameters, 33% of the memory footprint, and 10% of the computation time of competing techniques such as ACORN to reach the same representation accuracy.
more »
« less
LAPRAN: A Scalable Laplacian Pyramid Reconstructive Adversarial Network for Flexible Compressive Sensing Reconstruction
This paper addresses the single-image compressive sensing (CS) and reconstruction problem. We propose a scalable Laplacian pyramid reconstructive adversarial network (LAPRAN) that enables high-fidelity, flexible and fast CS images reconstruction. LAPRAN progressively reconstructs an image following the concept of the Laplacian pyramid through multiple stages of reconstructive adversarial networks (RANs). At each pyramid level, CS measurements are fused with a contextual latent vector to generate a high-frequency image residual. Consequently, LAPRAN can produce hierarchies of reconstructed images and each with an incremental resolution and improved quality. The scalable pyramid structure of LAPRAN enables high-fidelity CS reconstruction with a flexible resolution that is adaptive to a wide range of compression ratios (CRs), which is infeasible with existing methods. Experimental results on multiple public datasets show that LAPRAN offers an average 7.47dB and 5.98dB PSNR, and an average 57.93% and 33.20% SSIM improvement compared to model-based and data-driven baselines, respectively.
more »
« less
- Award ID(s):
- 1652038
- PAR ID:
- 10084500
- Date Published:
- Journal Name:
- The European Conference on Computer Vision (ECCV)
- Page Range / eLocation ID:
- 485-500
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Chen, Xi (Ed.)In patients with dense breasts or at high risk of breast cancer, dynamic contrast enhanced MRI (DCE-MRI) is a highly sensitive diagnostic tool. However, its specificity is highly variable and sometimes low; quantitative measurements of contrast uptake parameters may improve specificity and mitigate this issue. To improve diagnostic accuracy, data need to be captured at high spatial and temporal resolution. While many methods exist to accelerate MRI temporal resolution, not all are optimized to capture breast DCE-MRI dynamics. We propose a novel, flexible, and powerful framework for the reconstruction of highly-undersampled DCE-MRI data: enhancement-constrained acceleration (ECA). Enhancement-constrained acceleration uses an assumption of smooth enhancement at small time-scale to estimate points of smooth enhancement curves in small time intervals at each voxel. This method is tested in silico with physiologically realistic virtual phantoms, simulating state-of-the-art ultrafast acquisitions at 3.5s temporal resolution reconstructed at 0.25s temporal resolution (demo code available here). Virtual phantoms were developed from real patient data and parametrized in continuous time with arterial input function (AIF) models and lesion enhancement functions. Enhancement-constrained acceleration was compared to standard ultrafast reconstruction in estimating the bolus arrival time and initial slope of enhancement from reconstructed images. We found that the ECA method reconstructed images at 0.25s temporal resolution with no significant loss in image fidelity, a 4x reduction in the error of bolus arrival time estimation in lesions ( p < 0.01) and 11x error reduction in blood vessels ( p < 0.01). Our results suggest that ECA is a powerful and versatile tool for breast DCE-MRI.more » « less
-
Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment in fields such as cardiology and ophthalmology. Such applications can be further facilitated by deep learning-based super-resolution technology, which improves the capability of resolving morphological structures. However, existing deep learning-based method only focuses on spatial distribution and disregards frequency fidelity in image reconstruction, leading to a frequency bias. To overcome this limitation, we propose a frequency-aware super-resolution framework that integrates three critical frequency-based modules (i.e., frequency transformation, frequency skip connection, and frequency alignment) and frequency-based loss function into a conditional generative adversarial network (cGAN). We conducted a large-scale quantitative study from an existing coronary OCT dataset to demonstrate the superiority of our proposed framework over existing deep learning frameworks. In addition, we confirmed the generalizability of our framework by applying it to fish corneal images and rat retinal images, demonstrating its capability to super-resolve morphological details in eye imaging.more » « less
-
The sparse interferometric coverage of the Event Horizon Telescope (EHT) poses a significant challenge for both reconstruction and model fitting of black-hole images. PRIMO is a new principal components analysis-based algorithm for image reconstruction that uses the results of high-fidelity general relativistic, magnetohydrodynamic simulations of low-luminosity accretion flows as a training set. This allows the reconstruction of images that are both consistent with the interferometric data and that live in the space of images that is spanned by the simulations. PRIMO follows Monty Carlo Markov Chains to fit a linear combination of principal components derived from an ensemble of simulated images to interferometric data. We show that PRIMO can efficiently and accurately reconstruct synthetic EHT data sets for several simulated images, even when the simulation parameters are significantly different from those of the image ensemble that was used to generate the principal components. The resulting reconstructions achieve resolution that is consistent with the performance of the array and do not introduce significant biases in image features such as the diameter of the ring of emission.more » « less
-
We propose a new approach for high resolution semantic image synthesis. It consists of one base image generator and multiple class-specific generators. The base generator generates high quality images based on a segmentation map. To further improve the quality of different objects, we create a bank of Generative Adversarial Networks (GANs) by separately training class-specific models. This has several benefits including – dedicated weights for each class; centrally aligned data for each model; additional training data from other sources, potential of higher resolution and quality; and easy manipulation of a specific object in the scene. Experiments show that our approach can generate high quality images in high resolution while having flexibility of object-level control by using class-specific generators. Project page: https://yuheng-li.github.io/CollageGAN/more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
