An official website of the United States government
Low-power 3D perception is useful in a wide range of computer-vision applications. Thanks to the increasing availability of high-resolution single-photon avalanche diode (SPAD) arrays, single-photon LiDARs (SPLs) have emerged as a promising technology for 3D sensing. The conventional image formation model for an SPL involves capturing the time-varying light intensity - which we call the transient distribution - of a reflected laser pulse in the form of an equi-width (EW) histogram. Unfortunately, this approach leads to unmanageable data rates (~gigabytes/second) with high-resolution arrays, severely limiting the applicability of SPLs in power- and bandwidth-constrained scenarios (e.g., mobile devices). The proposal introduces a different approach using race logic processing to construct equi-depth histograms with variable bin widths, which reduces the bandwidth requirement while maintaining similar ranging accuracy, showing bandwidth reduction of over 100× in simulations.
more »
« less
Dithered depth imaging
Single-photon lidar (SPL) is a promising technology for depth measurement at long range or from weak reflectors because of the sensitivity to extremely low light levels. However, constraints on the timing resolution of existing arrays of single-photon avalanche diode (SPAD) detectors limit the precision of resulting depth estimates. In this work, we describe an implementation of subtractively-dithered SPL that can recover high-resolution depth estimates despite the coarse resolution of the detector. Subtractively-dithered measurement is achieved by adding programmable delays into the photon timing circuitry that introduce relative time shifts between the illumination and detection that are shorter than the time bin duration. Careful modeling of the temporal instrument response function leads to an estimator that outperforms the sample mean and results in depth estimates with up to 13 times lower root mean-squared error than if dither were not used. The simple implementation and estimation suggest that globally dithered SPAD arrays could be used for high spatial- and temporal-resolution depth sensing.
more »
« less
- PAR ID:
- 10200653
- Publisher / Repository:
- Optical Society of America
- Date Published:
- Journal Name:
- Optics Express
- Volume:
- 28
- Issue:
- 23
- ISSN:
- 1094-4087; OPEXFF
- Format(s):
- Medium: X Size: Article No. 35143
- Size(s):
- Article No. 35143
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Single-photon avalanche diodes (SPADs) are a rapidly developing image sensing technology with extreme low-light sensitivity and picosecond timing resolution. These unique capabilities have enabled SPADs to be used in applications like LiDAR, non-line-of-sight imaging and fluorescence microscopy that require imaging in photon-starved scenarios. In this work we harness these capabilities for dealing with motion blur in a passive imaging setting in low illumination conditions. Our key insight is that the data captured by a SPAD array camera can be represented as a 3D spatio-temporal tensor of photon detection events which can be integrated along arbitrary spatio-temporal trajectories with dynamically varying integration windows, depending on scene motion. We propose an algorithm that estimates pixel motion from photon timestamp data and dynamically adapts the integration windows to minimize motion blur. Our simulation results show the applicability of this algorithm to a variety of motion profiles including translation, rotation and local object motion. We also demonstrate the real-world feasibility of our method on data captured using a 32x32 SPAD camera.more » « less
-
Single-photon 3D cameras can record the time of arrival of billions of photons per second with picosecond accuracy. One common approach to summarize the photon data stream is to build a per-pixel timestamp histogram, resulting in a 3D histogram tensor that encodes distances along the time axis. As the spatio-temporal resolution of the histogram tensor increases, the in-pixel memory requirements and output data rates can quickly become impractical. To overcome this limitation, we propose a family of linear compressive representations of histogram tensors that can be computed efficiently, in an online fashion, as a matrix operation. We design practical lightweight compressive representations that are amenable to an in-pixel implementation and consider the spatio-temporal information of each timestamp. Furthermore, we implement our proposed framework as the first layer of a neural network, which enables the joint end-to-end optimization of the compressive representations and a downstream SPAD data processing model. We find that a well-designed compressive representation can reduce in-sensor memory and data rates up to 2 orders of magnitude without significantly reducing 3D imaging quality. Finally, we analyze the power consumption implications through an on-chip implementation.more » « less
-
Megapixel single-photon avalanche diode (SPAD) arrays have been developed recently, opening up the possibility of deploying SPADs as generalpurpose passive cameras for photography and computer vision. However, most previous work on SPADs has been limited to monochrome imaging. We propose a computational photography technique that reconstructs high-quality color images from mosaicked binary frames captured by a SPAD array, even for high-dyanamic-range (HDR) scenes with complex and rapid motion. Inspired by conventional burst photography approaches, we design algorithms that jointly denoise and demosaick single-photon image sequences. Based on the observation that motion effectively increases the color sample rate, we design a blue-noise pseudorandom RGBW color filter array for SPADs, which is tailored for imaging dark, dynamic scenes. Results on simulated data, as well as real data captured with a fabricated color SPAD hardware prototype shows that the proposed method can reconstruct high-quality images with minimal color artifacts even for challenging low-light, HDR and fast-moving scenes. We hope that this paper, by adding color to computational single-photon imaging, spurs rapid adoption of SPADs for real-world passive imaging applications.more » « less
-
Techniques to control the spectro-temporal properties of quantum states of light at ultrafast time scales are crucial for numerous applications in quantum information science. In this work, we report an all-optical time lens for quantum signals based on Bragg-scattering four-wave mixing with picosecond resolution. Our system achieves a temporal magnification factor of 158 with single-photon level inputs, which is sufficient to overcome the intrinsic timing jitter of superconducting nanowire single-photon detectors. We demonstrate discrimination of two terahertz-bandwidth, single-photon-level pulses with 2.1 ps resolution (electronic jitter corrected resolution of 1.25 ps). We draw on elegant tools from Fourier optics to further show that the time-lens framework can be extended to perform complex unitary spectro-temporal transformations by imparting optimized temporal and spectral phase profiles to the input waveforms. Using numerical optimization techniques, we show that a four-stage transformation can realize an efficient temporal mode sorter that demultiplexes 10 Hermite–Gaussian (HG) modes. Our time-lens-based framework represents a new toolkit for arbitrary spectro-temporal processing of single photons, with applications in temporal mode quantum processing, high-dimensional quantum key distribution, temporal mode matching for quantum networks, and quantum-enhanced sensing with time-frequency entangled states.more » « less
