An image mapping spectrometer (IMS) is a snapshot hyperspectral imager that simultaneously captures both the spatial (
Multidimensional photography can capture optical fields beyond the capability of conventional image sensors that measure only two-dimensional (2D) spatial distribution of light. By mapping a high-dimensional datacube of incident light onto a 2D image sensor, multidimensional photography resolves the scene along with other information dimensions, such as wavelength and time. However, the application of current multidimensional imagers is fundamentally restricted by their static optical architectures and measurement schemes—the mapping relation between the light datacube voxels and image sensor pixels is fixed. To overcome this limitation, we propose tunable multidimensional photography through active optical mapping. A high-resolution spatial light modulator, referred to as an active optical mapper, permutes and maps the light datacube voxels onto sensor pixels in an arbitrary and programmed manner. The resultant system can readily adapt the acquisition scheme to the scene, thereby maximising the measurement flexibility. Through active optical mapping, we demonstrate our approach in two niche implementations: hyperspectral imaging and ultrafast imaging.
more » « less- Award ID(s):
- 2053080
- PAR ID:
- 10200626
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
, ) and spectral ( ) information of incoming light. The IMS maps a three-dimensional (3D) datacube ( , , ) to a two-dimensional (2D) detector array ( , ) for parallel measurement. To reconstruct the original 3D datacube, one must construct a lookup table that connects voxels in the datacube and pixels in the raw image. Previous calibration methods suffer from either low speed or poor image quality. We herein present a slit-scan calibration method that can significantly reduce the calibration time while maintaining high accuracy. Moreover, we quantitatively analyzed the major artifact in the IMS, the striped image, and developed three numerical methods to correct for it. -
more » « less
Compressed ultrafast photography (CUP) is a computational optical imaging technique that can capture transient dynamics at an unprecedented speed. Currently, the image reconstruction of CUP relies on iterative algorithms, which are time-consuming and often yield nonoptimal image quality. To solve this problem, we develop a deep-learning-based method for CUP reconstruction that substantially improves the image quality and reconstruction speed. A key innovation toward efficient deep learning reconstruction of a large three-dimensional (3D) event datacube (
) ( , spatial coordinate; , time) is that we decompose the original datacube into massively parallel two-dimensional (2D) imaging subproblems, which are much simpler to solve by a deep neural network. We validated our approach on simulated and experimental data. -
more » « less
We introduce perturbative spatial frequency domain imaging (p-SFDI) for fast two-dimensional (2D) mapping of the optical properties and physiological characteristics of skin and cutaneous microcirculation using spatially modulated visible light. Compared to the traditional methods for recovering 2D maps through a pixel-by-pixel inversion, p-SFDI significantly shortens parameter retrieval time, largely avoids the random fitting errors caused by measurement noise, and enhances the image reconstruction quality. The efficacy of p-SFDI is demonstrated by
in vivo imaging forearm of one healthy subject, recovering the 2D spatial distribution of cutaneous hemoglobin concentration, oxygen saturation, scattering properties, the melanin content, and the epidermal thickness over a large field of view. Furthermore, the temporal and spatial variations in physiological parameters under the forearm reactive hyperemia protocol are revealed, showing its applications in monitoring temporal and spatial dynamics. -
Digital camera pixels measure image intensities by converting incident light energy into an analog electrical current, and then digitizing it into a fixed-width binary representation. This direct measurement method, while conceptually simple, suffers from limited dynamic range and poor performance under extreme illumination — electronic noise dominates under low illumination, and pixel full-well capacity results in saturation under bright illumination. We propose a novel intensity cue based on measuring inter-photon timing, defined as the time delay between detection of successive photons. Based on the statistics of inter-photon times measured by a time-resolved single-photon sensor, we develop theory and algorithms for a scene brightness estimator which works over extreme dynamic range; we experimentally demonstrate imaging scenes with a dynamic range of over ten million to one. The proposed techniques, aided by the emergence of single-photon sensors such as single-photon avalanche diodes (SPADs) with picosecond timing resolution, will have implications for a wide range of imaging applications: robotics, consumer photography, astronomy, microscopy and biomedical imaging.more » « less
-
more » « less
Abstract Compressed ultrafast photography (CUP) is an emerging potent technique that allows imaging a nonrepeatable or difficult‐to‐produce transient event in a single shot. Despite many recent advances, existing CUP techniques operate only at visible and near‐infrared wavelengths. In addition, spatial encoding via a digital micromirror device (DMD) in CUP systems often limits its field of view and imaging speeds. Finally, conventional reconstruction algorithms have limited control of the reconstruction process to further improve the image quality in the recovered datacubes of the scene. To overcome these limitations, this article reports a single‐shot UV‐CUP that exhibits a sequence depth of up to 1500 frames with a size of 1750 × 500 pixels at an imaging speed of 0.5 trillion frames per second. A patterned photocathode is integrated into a streak camera, which overcomes the previous restrictions in DMD‐based spatial encoding and improves the system's compactness. Meanwhile, the plug‐and‐play alternating direction method of multipliers algorithm is implemented to CUP's image reconstruction to enhance reconstructed image quality. UV‐CUP's single‐shot ultrafast imaging ability is demonstrated by recording UV pulses transmitting through various spatial patterns. UV‐CUP is expected to find many applications in both fundamental and applied science.
