Non-line-of-sight (NLOS) detection and ranging aim to identify hidden objects by sensing indirect light reflections. Although numerous computational methods have been proposed for NLOS detection and imaging, the post-signal processing required by peripheral circuits remains complex. One possible solution for simplifying NLOS detection and ranging involves the use of neuromorphic devices, such as memristors, which have intrinsic resistive-switching capabilities and can store spatiotemporal information. In this study, we employed the memristive spike-timing-dependent plasticity learning rule to program the time-of-flight (ToF) depth information directly into a memristor medium. By coupling the transmitted signal from the source with the photocurrent from the target object into a single memristor unit, we were able to induce a tunable programming pulse based on the time interval between the two signals that were superimposed. Here, this neuromorphic ToF principle is employed to detect and range NLOS objects without requiring complex peripheral circuitry to process raw signals. We experimentally demonstrated the effectiveness of the neuromorphic ToF principle by integrating a HfO2 memristor and an avalanche photodiode to detect NLOS objects in multiple directions. This technology has potential applications in various fields, such as automotive navigation, machine learning, and biomedical engineering.
more »
« less
Fast Computational Periscopy in Challenging Ambient Light Conditions through Optimized Preconditioning
Non-line-of-sight (NLOS) imaging is a rapidly advancing technology that provides asymmetric vision: seeing without being seen. Though limited in accuracy, resolution, and depth recovery compared to active methods, the capabilities of passive methods are especially surprising because they typically use only a single, inexpensive digital camera. One of the largest challenges in passive NLOS imaging is ambient background light, which limits the dynamic range of the measurement while carrying no useful information about the hidden part of the scene. In this work we propose a new reconstruction approach that uses an optimized linear transformation to balance the rejection of uninformative light with the retention of informative light, resulting in fast (video-rate) reconstructions of hidden scenes from photographs of a blank wall under high ambient light conditions.
more »
« less
- Award ID(s):
- 1955219
- NSF-PAR ID:
- 10267299
- Date Published:
- Journal Name:
- Proc. IEEE Int. Conf. Computational Photography
- Page Range / eLocation ID:
- 1 to 9
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We propose a novel non-line-of-sight (NLOS) imaging framework with long-wave infrared (IR). At long-wave IR wavelengths, certain physical parameters are more favorable for high-fidelity reconstruction. In contrast to prior work in visible light NLOS, at long-wave IR wavelengths, the hidden heat source acts as a light source. This simplifies the problem to a single bounce problem. In addition, surface reflectance has a much stronger specular reflection in the long-wave IR spectrum than in the visible light spectrum. We reformulate a light transport model that leverages these favorable physical properties of long-wave IR. Specifically, we demonstrate 2D shape recovery and 3D localization of a hidden object. Furthermore, we demonstrate near real-time and robust NLOS pose estimation of a human figure, the first such demonstration, to our knowledge.more » « less
-
more » « less
Abstract Non-line-of-sight (NLOS) imaging is a rapidly growing field seeking to form images of objects outside the field of view, with potential applications in autonomous navigation, reconnaissance, and even medical imaging. The critical challenge of NLOS imaging is that diffuse reflections scatter light in all directions, resulting in weak signals and a loss of directional information. To address this problem, we propose a method for seeing around corners that derives angular resolution from vertical edges and longitudinal resolution from the temporal response to a pulsed light source. We introduce an acquisition strategy, scene response model, and reconstruction algorithm that enable the formation of 2.5-dimensional representations—a plan view plus heights—and a 180∘field of view for large-scale scenes. Our experiments demonstrate accurate reconstructions of hidden rooms up to 3 meters in each dimension despite a small scan aperture (1.5-centimeter radius) and only 45 measurement locations.
-
more » « less
Non-line-of-sight (NLOS) imaging is a light-starving application that suffers from highly noisy measurement data. In order to recover the hidden scene with good contrast, it is crucial for the reconstruction algorithm to be robust against noises and artifacts. We propose here two weighting factors for the filtered backprojection (FBP) reconstruction algorithm in NLOS imaging. The apodization factor modifies the aperture (wall) function to reduce streaking artifacts, and the coherence factor evaluates the spatial coherence of measured signals for noise suppression. Both factors are simple to evaluate, and their synergistic effects lead to state-of-the-art reconstruction quality for FBP with noisy data. We demonstrate the effectiveness of the proposed weighting factors on publicly accessible experimental datasets.
-
more » « less
Abstract The ability to form reconstructions beyond line-of-sight view could be transformative in a variety of fields, including search and rescue, autonomous vehicle navigation, and reconnaissance. Most existing active non-line-of-sight (NLOS) imaging methods use data collection steps in which a pulsed laser is directed at several points on a relay surface, one at a time. The prevailing approaches include raster scanning of a rectangular grid on a vertical wall opposite the volume of interest to generate a collection of confocal measurements. These and a recent method that uses a horizontal relay surface are inherently limited by the need for laser scanning. Methods that avoid laser scanning to operate in a snapshot mode are limited to treating the hidden scene of interest as one or two point targets. In this work, based on more complete optical response modeling yet still without multiple illumination positions, we demonstrate accurate reconstructions of foreground objects while also introducing the capability of mapping the stationary scenery behind moving objects. The ability to count, localize, and characterize the sizes of hidden objects, combined with mapping of the stationary hidden scene, could greatly improve indoor situational awareness in a variety of applications.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICCP51581.2021.9466264
