skip to main content

Title: A compressive hyperspectral video imaging system using a single-pixel detector

Capturing fine spatial, spectral, and temporal information of the scene is highly desirable in many applications. However, recording data of such high dimensionality requires significant transmission bandwidth. Current computational imaging methods can partially address this challenge but are still limited in reducing input data throughput. In this paper, we report a video-rate hyperspectral imager based on a single-pixel photodetector which can achieve high-throughput hyperspectral video recording at a low bandwidth. We leverage the insight that 4-dimensional (4D) hyperspectral videos are considerably more compressible than 2D grayscale images. We propose a joint spatial-spectral capturing scheme encoding the scene into highly compressed measurements and obtaining temporal correlation at the same time. Furthermore, we propose a reconstruction method relying on a signal sparsity model in 4D space and a deep learning reconstruction approach greatly accelerating reconstruction. We demonstrate reconstruction of 128 × 128 hyperspectral images with 64 spectral bands at more than 4 frames per second offering a 900× data throughput compared to conventional imaging, which we believe is a first-of-its kind of a single-pixel-based hyperspectral imager.

more » « less
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Nature Communications
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. We demonstrate a kilohertz frame rate snapshot hyperspectral imaging system suitable for high-speed imaging, which we name snapshot hyperspectral imager for emission and reactions (SHEAR). This system splits the sensor of a single high-speed camera to simultaneously capture a conventional image and a spectrally sheared response of the scene under study. Given the small, point-source-like nature of burning metal micro-particles, the spectral response of the species is captured without the need for a slit, as is needed in conventional imaging spectrometers. We pair robust image registration techniques with sparse reconstruction algorithms to computationally disentangle overlapping spectra associated with many burning particles over the course of a combustion experiment. As a proof-of-concept experiment, representative physical vapor deposited Al:Zr composite particles are ignited, and their burn evolution is recorded at a frame rate of 2 kHz using this method. We demonstrate operation over two distinct wavelength ranges spanning hundreds of nanometers in wavelength and with sub-nanometer resolution. We are able to track hundreds of individual Al:Zr particles in a single high-speed video, providing ample statistics of burn time, temperature, and AlO emission timing in a high-throughput method. The demonstrated technology is high-throughput, flexible in wavelength, inexpensive, and relatively easy to implement, and provides a much needed tool forin situcomposite metal fuel diagnostics.

    more » « less
  2. Hyperspectral imaging (HSI) is a spectroscopic technique which captures images at a high contrast over a wide range of wavelengths to show pixel specific composition. Traditional uses of HSI include: satellite imagery, food distribution quality control and digital archaeological reconstruction. Our lab has focused on developing applications of HSI fluorescence imaging systems to study molecule-specific detection for rapid cell signaling events or real-time endoscopic screening. Previously, we have developed a prototype spectral light source, using our modified imaging technique, excitationscanning hyperspectral imaging (HIFEX), coupled to a commercial colonoscope for feasibility testing. The 16 wavelength LED array was combined, using a multi-branched solid light guide, to couple to the scope’s optical input. The prototype acquired a spectral scan at near video-rate speeds (~8 fps). The prototype could operate at very rapid wavelength switch speeds, limited to the on/off rates of the LEDs (~10 μs), but imaging speed was limited due to optical transmission losses (~98%) through the solid light guide. Here we present a continuation of our previous work in performing an in-depth analysis of the solid light guide to optimize the optical intensity throughput. The parameters evaluated include: LED intensity input, geometry (branch curvature and combination) and light propagation using outer claddings. Simulations were conducted using a Monte Carlo ray tracing software (TracePro). Results show that transmission within the branched light guide may be optimized through LED focusing lenses, bend radii and smooth tangential branch merges. Future work will test a new fabricated light guide from the optimized model framework. 
    more » « less
  3. null (Ed.)
    Abstract Effective snow grain radius ( r e ) is mapped at high resolution using near-infrared hyperspectral imaging (NIR-HSI). The NIR-HSI method can be used to quantify r e spatial variability, change in r e due to metamorphism, and visualize water percolation in the snowpack. Results are presented for three different laboratory-prepared snow samples (homogeneous, ice lens, fine grains over coarse grains), the sidewalls of which were imaged before and after melt induced by a solar lamp. The spectral reflectance in each ~3 mm pixel was inverted for r e using the scaled band area of the ice absorption feature centered at 1030 nm, producing r e maps consisting of 54 740 pixels. All snow samples exhibited grain coarsening post-melt as the result of wet snow metamorphism, which is quantified by the change in r e distributions from pre- and post-melt images. The NIR-HSI method was compared to r e retrievals from a field spectrometer and X-ray computed microtomography (micro-CT), resulting in the spectrometer having the same mean r e and micro-CT having 23.9% higher mean r e than the hyperspectral imager. As compact hyperspectral imagers become more widely available, this method may be a valuable tool for assessing r e spatial variability and snow metamorphism in field and laboratory settings. 
    more » « less
  4. Positive outcomes for colorectal cancer treatment have been linked to early detection. The difficulty in detecting early lesions is the limited contrast with surrounding mucosa and minimal definitive markers to distinguish between hyperplastic and carcinoma lesions. Colorectal cancer is the 3rd leading cancer for incidence and mortality rates which is potentially linked to missed early lesions which allow for increased growth and metastatic potential. One potential technology for early-stage lesion detection is hyperspectral imaging. Traditionally, hyperspectral imaging uses reflectance spectroscopic data to provide a component analysis, per pixel, of an image in fields such as remote sensing, agriculture, food processing and archaeology. This work aims to acquire higher signal-to-noise fluorescence spectroscopic data, harnessing the autofluorescence of tissue, adding a hyperspectral contrast to colorectal cancer detection while maintaining spatial resolution at video-rate speeds. We have previously designed a multi-furcated LED-based spectral light source to prove this concept. Our results demonstrated that the technique is feasible, but the initial prototype has a high light transmission loss (~98%) minimizing spatial resolution and slowing video acquisition. Here, we present updated results in developing an optical ray-tracing model of light source geometries to maximize irradiance throughput for excitation-scanning hyperspectral imaging. Results show combining solid light guide branches have a compounding light loss effect, however, there is potential to minimize light loss through the use of optical claddings. This simulation data will provide the necessary metrics to verify and validate future physical optical components within the hyperspectral endoscopic system for detecting colorectal cancer. 
    more » « less
  5. null (Ed.)
    Optical emission spectroscopy (OES) imaging is often used for diagnostics for better understanding of the underlying mechanisms of plasmas. Typical spectral images, however, contain intensity maps that are integrated along the line-of-sight. A widespread method to extract the radial information is Abel's inversion, but most approaches result in accumulation of error toward the plasma axial position, which is often the region of most interest. Here, a Fourier-transform based Abel's inversion algorithm, which spreads the error evenly across the radial profile, is optimized for OES images collected on a push-broom hyperspectral imaging system (PbHSI). Furthermore, a sub-pixel shifting (SPS) sampling protocol is employed on the PbHSI in the direction of the radial reconstruction to allow improved fidelity from the increased number of data points. The accuracy and fidelity of the protocol are characterized and optimized with a software-based 3-dimensional hyperspectral model datacube. A systematic study of the effects of varying levels of representative added noise, different noise filters, number of data points and cosine expansions used in the inversion, as well as the spatial intensity distribution shapes of the radial profile are presented. A 3D median noise filter with 3-pixel radius, a minimum of 50 points and 8 cosine expansions is needed to keep the relative root mean squared error (rRMSE) <8%. The optimized protocol is implemented for the first time on OES images of a micro-capillary dielectric barrier discharge (μDBD) source obtained via SPS PbHSI system and the extracted radial emission of different plasma species (He, N 2 , N 2 + ) are shown. 
    more » « less