skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo
Conventional stereo suffers from a fundamental trade-off between imaging volume and signal-to-noise ratio (SNR) – due to the conflicting impact of aperture size on both these variables. Inspired by the extended depth of field cameras, we propose a novel end-to-end learning-based technique to overcome this limitation, by introducing a phase mask at the aperture plane of the cameras in a stereo imaging system. The phase mask creates a depth-dependent yet numerically invertible point spread function, allowing us to recover sharp image texture and stereo correspondence over a significantly extended depth of field (EDOF) than conventional stereo. The phase mask pattern, the EDOF image reconstruction, and the stereo disparity estimation are all trained together using an end-to-end learned deep neural network. We perform theoretical analysis and characterization of the proposed approach and show a 6× increase in volume that can be imaged in simulation. We also build an experimental prototype and validate the approach using real-world results acquired using this prototype system.  more » « less
Award ID(s):
1652633
PAR ID:
10319134
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, IEEE International Conference on Computational Photography)
    There is an increasing need for passive 3D scanning in many applications that have stringent energy constraints. In this paper, we present an approach for single frame, single viewpoint, passive 3D imaging using a phase mask at the aperture plane of a camera. Our approach relies on an end-to-end optimization framework to jointly learn the optimal phase mask and the reconstruction algorithm that allows an accurate estimation of range image from captured data. Using our optimization framework, we design a new phase mask that performs significantly better than existing approaches. We build a prototype by inserting a phase mask fabricated using photolithography into the aperture plane of a conventional camera and show compelling performance in 3D imaging. 
    more » « less
  2. ; ; ; (, Optics Express)
    Conventional continuous-wave amplitude-modulated time-of-flight (CWAM ToF) cameras suffer from a fundamental trade-off between light throughput and depth of field (DoF): a larger lens aperture allows more light collection but suffers from significantly lower DoF. However, both high light throughput, which increases signal-to-noise ratio, and a wide DoF, which enlarges the system’s applicable depth range, are valuable for CWAM ToF applications. In this work, we propose EDoF-ToF, an algorithmic method to extend the DoF of large-aperture CWAM ToF cameras by using a neural network to deblur objects outside of the lens’s narrow focal region and thus produce an all-in-focus measurement. A key component of our work is the proposed large-aperture ToF training data simulator, which models the depth-dependent blurs and partial occlusions caused by such apertures. Contrary to conventional image deblurring where the blur model is typically linear, ToF depth maps are nonlinear functions of scene intensities, resulting in a nonlinear blur model that we also derive for our simulator. Unlike extended DoF for conventional photography where depth information needs to be encoded (or made depth-invariant) using additional hardware (phase masks, focal sweeping, etc.), ToF sensor measurements naturally encode depth information, allowing a completely software solution to extended DoF. We experimentally demonstrate EDoF-ToF increasing the DoF of a conventional ToF system by 3.6 ×, effectively achieving the DoF of a smaller lens aperture that allows 22.1 × less light. Ultimately, EDoF-ToF enables CWAM ToF cameras to enjoy the benefits of both high light throughput and a wide DoF. 
    more » « less
  3. ; (, Optics Express)
    Imaging depth of field is shallow in applications with high magnification and high numerical aperture, such as microscopy, resulting in images with in- and out-of-focus regions. Therefore, methods to extend depth of field are of particular interest. Researchers have previously shown the advantages of using freeform components to extend depth of field, with each optical system requiring a specially designed phase plate. In this paper we present a method to enable extended depth-of-field imaging for a range of numerical apertures using freeform phase plates to create variable cubic wavefronts. The concept is similar to an Alvarez lens which creates variable spherical wavefronts through the relative translation of two transmissive elements with XY polynomial surfaces. We discuss design and optimization methods to enable extended depth of field for lenses with different numerical aperture values by considering through-focus variation of the point spread function and compare on- and off-axis performance through multiple metrics. 
    more » « less
  4. ; ; ; ; (, Applied Physics Letters)
    Light-sheet microscopes must compromise among field of view, optical sectioning, resolution, and detection efficiency. High-numerical-aperture (NA) detection objective lenses provide higher resolution, but their narrow depth of field inefficiently captures the fluorescence signal generated throughout the thickness of the illumination light sheet when imaging large volumes. Here, we present ExD-SPIM (extended depth-of-field selective-plane illumination microscopy), an improved light-sheet microscopy strategy that solves this limitation by extending the depth of field (DOF) of high-NA detection objectives to match the thickness of the illumination light sheet. This extension of the DOF uses a phase mask to axially stretch the point-spread function of the objective lens while largely preserving lateral resolution. This matching of the detection DOF to the illumination-sheet thickness increases the total fluorescence collection, reduces the background, and improves the overall signal-to-noise ratio (SNR), as shown by numerical simulations, imaging of bead phantoms, and imaging living animals. In comparison to conventional light sheet imaging with low-NA detection that yields equivalent DOF, the results show that ExD-SPIM increases the SNR by more than threefold and dramatically reduces the rate of photobleaching. Compared to conventional high-NA detection, ExD-SPIM improves the signal sensitivity and volumetric coverage of whole-brain activity imaging, increasing the number of detected neurons by over a third. 
    more » « less
  5. ; (, SPIE Optical Engineering + Applications, 2020, Online Only)
    Hahlweg, Cornelius F.; Mulley, Joseph R. (Ed.)
    Increasing depth of field in imaging systems can be beneficial, particularly for systems with high numerical apertures and short depth of field, such as microscopy. Extending depth of field has been previously demonstrated, for example, using non-rotationally symmetric (freeform) components such as cubic and logarithmic phase plates. Such fixed phase plates are generally designed for a specific optical system, so a different phase plate is required for each system. Methods that enable variable extended depth of field for multiple optical systems could provide benefits by reducing the number of required components and costs. In this paper, we explore the design of a single pair of transmissive freeform surfaces to enable extended depth of field for multiple lenses with different numerical apertures through relative translation of the freeform components. This work builds on the concept of an Alvarez lens, in which one pair of transmissive XY-polynomial freeform surfaces generates variable optical power through lateral relative shifts between the surfaces. The presented approach is based on the design of multiple fixed phase plates to optimize the through-focus Modulation Transfer Function (MTF) for imaging lenses of given numerical apertures. The freeform surface equation for the desired variable phase plate pair is then derived and the relative shift amounts between the freeform surfaces are calculated to enable extended depth of field for multiple imaging lenses with different numerical aperture values. Design approaches and simulation results will be discussed. © (2020) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only. Citation Download Citation Sara Moein and Thomas J. Suleski "Variable extended depth of field imaging using freeform optics", Proc. SPIE 11483, Novel Optical Systems, Methods, and Applications XXIII, 114830G (21 August 2020); https://doi.org/10.1117/12.2568723 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email