An official website of the United States government
Compressed ultrafast photography (CUP) is a computational optical imaging technique that can capture transient dynamics at an unprecedented speed. Currently, the image reconstruction of CUP relies on iterative algorithms, which are time-consuming and often yield nonoptimal image quality. To solve this problem, we develop a deep-learning-based method for CUP reconstruction that substantially improves the image quality and reconstruction speed. A key innovation toward efficient deep learning reconstruction of a large three-dimensional (3D) event datacube ( ) ( , spatial coordinate; , time) is that we decompose the original datacube into massively parallel two-dimensional (2D) imaging subproblems, which are much simpler to solve by a deep neural network. We validated our approach on simulated and experimental data.
more »
« less
Single‐Shot Ultraviolet Compressed Ultrafast Photography
Abstract Compressed ultrafast photography (CUP) is an emerging potent technique that allows imaging a nonrepeatable or difficult‐to‐produce transient event in a single shot. Despite many recent advances, existing CUP techniques operate only at visible and near‐infrared wavelengths. In addition, spatial encoding via a digital micromirror device (DMD) in CUP systems often limits its field of view and imaging speeds. Finally, conventional reconstruction algorithms have limited control of the reconstruction process to further improve the image quality in the recovered datacubes of the scene. To overcome these limitations, this article reports a single‐shot UV‐CUP that exhibits a sequence depth of up to 1500 frames with a size of 1750 × 500 pixels at an imaging speed of 0.5 trillion frames per second. A patterned photocathode is integrated into a streak camera, which overcomes the previous restrictions in DMD‐based spatial encoding and improves the system's compactness. Meanwhile, the plug‐and‐play alternating direction method of multipliers algorithm is implemented to CUP's image reconstruction to enhance reconstructed image quality. UV‐CUP's single‐shot ultrafast imaging ability is demonstrated by recording UV pulses transmitting through various spatial patterns. UV‐CUP is expected to find many applications in both fundamental and applied science.
more »
« less
- Award ID(s):
- 1813848
- PAR ID:
- 10456600
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Laser & Photonics Reviews
- Volume:
- 14
- Issue:
- 10
- ISSN:
- 1863-8880
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
High-speed widefield fluorescence imaging of neural activity in vivo is fundamentally limited by fluctuations in recorded signal due to background contamination and stochastic noise. In this study, we show background and shot noise-reduced imaging of the ultrafast genetically encoded Ca2+indicator GCaMP8f in CA1 pyramidal neurons using periodic structured illumination (SI) with computational image reconstruction. We implement what we believe to be a novel reconstruction method for data acquired using periodic structured illumination, termed pseudo-HiLo (pHiLo), that combines a pseudo-widefield (pWF) reconstruction with individual SI frames to perform a HiLo reconstruction. We compare this new technique to interleaved optical sectioning structured illumination microscopy (OS-SIM) and pWF reconstruction. We quantify the performance of each reconstruction by evaluating contrast, transient peak-to-noise ratio (PNR), pairwise correlation coefficients between ΔF/F time courses extracted from individual in-focus cells, and correlation coefficients between each cell with surrounding cell-free background pixels. We additionally incorporate a self-supervised deep learning method for real-time noise suppression (DeepCAD-RT) into our data preprocessing pipeline. At 500 Hz frame rates, we demonstrate a 75% increase in PNR using the denoised pHiLo reconstruction compared to pWF. Utilizing DeepCAD-RT, we show significant PNR improvements using both structured illumination (SI) reconstruction methods with OS-SIM showing a 59% increase in PNR after denoising. Both pHiLo and OS-SIM reconstructions result in a ≈65% decrease in the mean correlation coefficient of the ΔF/F time courses between ROIs in comparison with pWF, indicating the potential to remove background fluorescent transients from out-of-focus cells.more » « less
-
PurposeTo develop a physics‐guided deep learning (PG‐DL) reconstruction strategy based on a signal intensity informed multi‐coil (SIIM) encoding operator for highly‐accelerated simultaneous multislice (SMS) myocardial perfusion cardiac MRI (CMR). MethodsFirst‐pass perfusion CMR acquires highly‐accelerated images with dynamically varying signal intensity/SNR following the administration of a gadolinium‐based contrast agent. Thus, using PG‐DL reconstruction with a conventional multi‐coil encoding operator leads to analogous signal intensity variations across different time‐frames at the network output, creating difficulties in generalization for varying SNR levels. We propose to use a SIIM encoding operator to capture the signal intensity/SNR variations across time‐frames in a reformulated encoding operator. This leads to a more uniform/flat contrast at the output of the PG‐DL network, facilitating generalizability across time‐frames. PG‐DL reconstruction with the proposed SIIM encoding operator is compared to PG‐DL with conventional encoding operator, split slice‐GRAPPA, locally low‐rank (LLR) regularized reconstruction, low‐rank plus sparse (L + S) reconstruction, and regularized ROCK‐SPIRiT. ResultsResults on highly accelerated free‐breathing first pass myocardial perfusion CMR at three‐fold SMS and four‐fold in‐plane acceleration show that the proposed method improves upon the reconstruction methods use for comparison. Substantial noise reduction is achieved compared to split slice‐GRAPPA, and aliasing artifacts reduction compared to LLR regularized reconstruction, L + S reconstruction and PG‐DL with conventional encoding. Furthermore, a qualitative reader study indicated that proposed method outperformed all methods. ConclusionPG‐DL reconstruction with the proposed SIIM encoding operator improves generalization across different time‐frames /SNRs in highly accelerated perfusion CMR.more » « less
-
Single-shot 3D optical microscopy that can capture high-resolution information over a large volume has broad applications in biology. Existing 3D imaging methods using point-spread-function (PSF) engineering often have limited depth of field (DOF) or require custom and often complex design of phase masks. We propose a new, to the best of our knowledge, PSF approach that is easy to implement and offers a large DOF. The PSF appears to be axially V-shaped, engineered by replacing the conventional tube lens with a pair of axicon lenses behind the objective lens of a wide-field microscope. The 3D information can be reconstructed from a single-shot image using a deep neural network. Simulations in a 10× magnification wide-field microscope show the V-shaped PSF offers excellent 3D resolution (<2.5 µm lateral and ∼15 µm axial) over a ∼350 µm DOF at a 550 nm wavelength. Compared to other popular PSFs designed for 3D imaging, the V-shaped PSF is simple to deploy and provides high 3D reconstruction quality over an extended DOF.more » « less
-
Spectroscopic single-molecule localization microscopy (sSMLM) simultaneously provides spatial localization and spectral information of individual single-molecules emission, offering multicolor super-resolution imaging of multiple molecules in a single sample with the nanoscopic resolution. However, this technique is limited by the requirements of acquiring a large number of frames to reconstruct a super-resolution image. In addition, multicolor sSMLM imaging suffers from spectral cross-talk while using multiple dyes with relatively broad spectral bands that produce cross-color contamination. Here, we present a computational strategy to accelerate multicolor sSMLM imaging. Our method uses deep convolution neural networks to reconstruct high-density multicolor super-resolution images from low-density, contaminated multicolor images rendered using sSMLM datasets with much fewer frames, without compromising spatial resolution. High-quality, super-resolution images are reconstructed using up to 8-fold fewer frames than usually needed. Thus, our technique generates multicolor super-resolution images within a much shorter time, without any changes in the existing sSMLM hardware system. Two-color and three-color sSMLM experimental results demonstrate superior reconstructions of tubulin/mitochondria, peroxisome/mitochondria, and tubulin/mitochondria/peroxisome in fixed COS-7 and U2-OS cells with a significant reduction in acquisition time.more » « less
