skip to main content

Title: Deep compressed imaging via optimized pattern scanning

The need for high-speed imaging in applications such as biomedicine, surveillance, and consumer electronics has called for new developments of imaging systems. While the industrial effort continuously pushes the advance of silicon focal plane array image sensors, imaging through a single-pixel detector has gained significant interest thanks to the development of computational algorithms. Here, we present a new imaging modality, deep compressed imaging via optimized-pattern scanning, which can significantly increase the acquisition speed for a single-detector-based imaging system. We project and scan an illumination pattern across the object and collect the sampling signal with a single-pixel detector. We develop an innovative end-to-end optimized auto-encoder, using a deep neural network and compressed sensing algorithm, to optimize the illumination pattern, which allows us to reconstruct faithfully the image from a small number of measurements, with a high frame rate. Compared with the conventional switching-mask-based single-pixel camera and point-scanning imaging systems, our method achieves a much higher imaging speed, while retaining a similar imaging quality. We experimentally validated this imaging modality in the settings of both continuous-wave illumination and pulsed light illumination and showed high-quality image reconstructions with a high compressed sampling rate. This new compressed sensing modality could be widely applied in different imaging systems, enabling new applications that require high imaging speeds.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ;
Publisher / Repository:
Optical Society of America
Date Published:
Journal Name:
Photonics Research
Page Range / eLocation ID:
Article No. B57
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Goda, Keisuke ; Tsia, Kevin K. (Ed.)
    We present a new deep compressed imaging modality by scanning a learned illumination pattern on the sample and detecting the signal with a single-pixel detector. This new imaging modality allows a compressed sampling of the object, and thus a high imaging speed. The object is reconstructed through a deep neural network inspired by compressed sensing algorithm. We optimize the illumination pattern and the image reconstruction network by training an end-to-end auto-encoder framework. Comparing with the conventional single-pixel camera and point-scanning imaging system, we accomplish a high-speed imaging with a reduced light dosage, while preserving a high imaging quality. 
    more » « less
  2. Glow discharge optical emission spectroscopy elemental mapping (GDOES EM), enabled by spectral imaging strategies, is an advantageous technique for direct multi-elemental analysis of solid samples in rapid timeframes. Here, a single-pixel, or point scan, spectral imaging system based on compressed sensing image sampling, is developed and optimized in terms of matrix density, compression factor, sparsifying basis, and reconstruction algorithm for coupling with GDOES EM. It is shown that a 512 matrix density at a compression factor of 30% provides the highest spatial fidelity in terms of the peak signal-to-noise ratio (PSNR) and complex wavelet structural similarity index measure (cw-SSIM) while maintaining fast measurement times. The background equivalent concentration (BEC) of Cu I at 510.5 nm is improved when implementing the discrete wavelet transform (DWT) sparsifying basis and Two-step Iterative Shrinking/Thresholding Algorithm for Linear Inverse Problems (TwIST) reconstruction algorithm. Utilizing these optimum conditions, a GDOES EM of a flexible, etched-copper circuit board was then successfully demonstrated with the compressed sensing single-pixel spectral imaging system (CSSPIS). The newly developed CSSPIS allows taking advantage of the significant cost-efficiency of point-scanning approaches (>10× vs. intensified array detector systems), while overcoming (up to several orders of magnitude) their inherent and substantial throughput limitations. Ultimately, it has the potential to be implemented on readily available commercial GDOES instruments by adapting the collection optics. 
    more » « less
  3. null (Ed.)
    We propose a new imaging scheme of compressed sensing by scanning an illumination pattern on the object. Comparing with conventional single-pixel cameras, we expect a >50x increase in imaging speed with similar imaging quality. 
    more » « less
  4. Spectroscopic image data has provided molecular discrimination for numerous fields including: remote sensing, food safety and biomedical imaging. Despite the various technologies for acquiring spectral data, there remains a trade-off when acquiring data. Typically, spectral imaging either requires long acquisition times to collect an image stack with high spectral specificity or acquisition times are shortened at the expense of fewer spectral bands or reduced spatial sampling. Hence, new spectral imaging microscope platforms are needed to help mitigate these limitations. Fluorescence excitation-scanning spectral imaging is one such new technology, which allows more of the emitted signal to be detected than comparable emission-scanning spectral imaging systems. Here, we have developed a new optical geometry that provides spectral illumination for use in excitation-scanning spectral imaging microscope systems. This was accomplished using a wavelength-specific LED array to acquire spectral image data. Feasibility of the LED-based spectral illuminator was evaluated through simulation and benchtop testing and assessment of imaging performance when integrated with a widefield fluorescence microscope. Ray tracing simulations (TracePro) were used to determine optimal optical component selection and geometry. Spectral imaging feasibility was evaluated using a series of 6-label fluorescent slides. The LED-based system response was compared to a previously tested thin-film tunable filter (TFTF)-based system. Spectral unmixing successfully discriminated all fluorescent components in spectral image data acquired from both the LED and TFTF systems. Therefore, the LED-based spectral illuminator provided spectral image data sets with comparable information content so as to allow identification of each fluorescent component. These results provide proof-of-principle demonstration of the ability to combine output from many discrete wavelength LED sources using a double-mirror (Cassegrain style) optical configuration that can be further modified to allow for high speed, video-rate spectral image acquisition. Real-time spectral fluorescence microscopy would allow monitoring of rapid cell signaling processes (i.e., Ca2+and other second messenger signaling) and has potential to be translated to clinical imaging platforms.

    more » « less
  5. Artificial intelligence has recently been widely used in computational imaging. The deep neural network (DNN) improves the signal-to-noise ratio of the retrieved images, whose quality is otherwise corrupted due to the low sampling ratio or noisy environments. This work proposes a new computational imaging scheme based on the sequence transduction mechanism with the transformer network. The simulation database assists the network in achieving signal translation ability. The experimental single-pixel detector’s signal will be ‘translated’ into a 2D image in an end-to-end manner. High-quality images with no background noise can be retrieved at a sampling ratio as low as 2%. The illumination patterns can be either well-designed speckle patterns for sub-Nyquist imaging or random speckle patterns. Moreover, our method is robust to noise interference. This translation mechanism opens a new direction for DNN-assisted ghost imaging and can be used in various computational imaging scenarios.

    more » « less