skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Friday, May 16 until 2:00 AM ET on Saturday, May 17 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Deep compressed imaging via optimized pattern scanning
The need for high-speed imaging in applications such as biomedicine, surveillance, and consumer electronics has called for new developments of imaging systems. While the industrial effort continuously pushes the advance of silicon focal plane array image sensors, imaging through a single-pixel detector has gained significant interest thanks to the development of computational algorithms. Here, we present a new imaging modality, deep compressed imaging via optimized-pattern scanning, which can significantly increase the acquisition speed for a single-detector-based imaging system. We project and scan an illumination pattern across the object and collect the sampling signal with a single-pixel detector. We develop an innovative end-to-end optimized auto-encoder, using a deep neural network and compressed sensing algorithm, to optimize the illumination pattern, which allows us to reconstruct faithfully the image from a small number of measurements, with a high frame rate. Compared with the conventional switching-mask-based single-pixel camera and point-scanning imaging systems, our method achieves a much higher imaging speed, while retaining a similar imaging quality. We experimentally validated this imaging modality in the settings of both continuous-wave illumination and pulsed light illumination and showed high-quality image reconstructions with a high compressed sampling rate. This new compressed sensing modality could be widely applied in different imaging systems, enabling new applications that require high imaging speeds.  more » « less
Award ID(s):
1847141
PAR ID:
10216097
Author(s) / Creator(s):
; ;
Publisher / Repository:
Optical Society of America
Date Published:
Journal Name:
Photonics Research
Volume:
9
Issue:
3
ISSN:
2327-9125
Page Range / eLocation ID:
Article No. B57
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, High-Speed Biomedical Imaging and Spectroscopy VI)
    Goda, Keisuke; Tsia, Kevin K. (Ed.)
    We present a new deep compressed imaging modality by scanning a learned illumination pattern on the sample and detecting the signal with a single-pixel detector. This new imaging modality allows a compressed sampling of the object, and thus a high imaging speed. The object is reconstructed through a deep neural network inspired by compressed sensing algorithm. We optimize the illumination pattern and the image reconstruction network by training an end-to-end auto-encoder framework. Comparing with the conventional single-pixel camera and point-scanning imaging system, we accomplish a high-speed imaging with a reduced light dosage, while preserving a high imaging quality. 
    more » « less
  2. ; ; (, CLEO: Science and Innovations 2020)
    null (Ed.)
    We propose a new imaging scheme of compressed sensing by scanning an illumination pattern on the object. Comparing with conventional single-pixel cameras, we expect a >50x increase in imaging speed with similar imaging quality. 
    more » « less
  3. ; ; ; ; (, Journal of Analytical Atomic Spectrometry)
    Glow discharge optical emission spectroscopy elemental mapping (GDOES EM), enabled by spectral imaging strategies, is an advantageous technique for direct multi-elemental analysis of solid samples in rapid timeframes. Here, a single-pixel, or point scan, spectral imaging system based on compressed sensing image sampling, is developed and optimized in terms of matrix density, compression factor, sparsifying basis, and reconstruction algorithm for coupling with GDOES EM. It is shown that a 512 matrix density at a compression factor of 30% provides the highest spatial fidelity in terms of the peak signal-to-noise ratio (PSNR) and complex wavelet structural similarity index measure (cw-SSIM) while maintaining fast measurement times. The background equivalent concentration (BEC) of Cu I at 510.5 nm is improved when implementing the discrete wavelet transform (DWT) sparsifying basis and Two-step Iterative Shrinking/Thresholding Algorithm for Linear Inverse Problems (TwIST) reconstruction algorithm. Utilizing these optimum conditions, a GDOES EM of a flexible, etched-copper circuit board was then successfully demonstrated with the compressed sensing single-pixel spectral imaging system (CSSPIS). The newly developed CSSPIS allows taking advantage of the significant cost-efficiency of point-scanning approaches (>10× vs. intensified array detector systems), while overcoming (up to several orders of magnitude) their inherent and substantial throughput limitations. Ultimately, it has the potential to be implemented on readily available commercial GDOES instruments by adapting the collection optics. 
    more » « less
  4. ; ; ; (, IEEE Transactions on Pattern Analysis and Machine Intelligence)
    Abstract—Accurately capturing dynamic scenes with wideranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera’s frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds — both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system’s adaptability and performance in dynamic conditions. Index Terms—High-dynamic-range video, high-speed imaging, CMOS image sensors, programmable sensors, deep learning, convolutional neural networks. 
    more » « less
  5. ; ; ; (, bioRxiv)
    Abstract We present a two-photon fluorescence microscope designed for high-speed imaging of neural activity in cellular resolution. Our microscope uses a new adaptive sampling scheme with line illumination. Instead of building images pixel by pixel via scanning a diffraction-limited spot across the sample, our scheme only illuminates the regions of interest (i.e., neuronal cell bodies), and samples a large area of them in a single measurement. Such a scheme significantly increases the imaging speed and reduces the overall laser power on the brain tissue. Using this approach, we performed high-speed imaging of the neural activity of mouse cortexin vivo. Our method provides a new sampling strategy in laser-scanning two-photon microscopy, and will be powerful for high-throughput imaging of neural activity. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email