skip to main content


Title: Polarimetric Helmholtz Stereopsis
Helmholtz stereopsis (HS) exploits the reciprocity principle of light propagation (i.e., the Helmholtz reciprocity) for 3D reconstruction of surfaces with arbitrary reflectance. In this paper, we present the polarimetric Helmholtz stereopsis (polar-HS), which extends the classical HS by considering the polarization state of light in the reciprocal paths. With the additional phase information from polar- ization, polar-HS requires only one reciprocal image pair. We formulate new reciprocity and diffuse/specular polari- metric constraints to recover surface depths and normals using an optimization framework. Using a hardware proto- type, we show that our approach produces high-quality 3D reconstruction for different types of surfaces, ranging from diffuse to highly specular.  more » « less
Award ID(s):
1948524 2024795
NSF-PAR ID:
10320125
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
CVF/IEEE International Conference on Computer Vision
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The decomposition of light transport into direct and global components, diffuse and specular interreflections, and subsurface scattering allows for new visualizations of light in everyday scenes. In particular, indirect light contains a myriad of information about the complex appearance of materials useful for computer vision and inverse rendering applications. In this paper, we present a new imaging technique that captures and analyzes components of indirect light via light transport using a synchronized projector-camera system. The rectified system illuminates the scene with epipolar planes corresponding to projector rows, and we vary two key parameters to capture plane-to-ray light transport between projector row and camera pixel: (1)the offset between projector row and camera row in the rolling shutter, and (2)the exposure of the camera row. We describe how this synchronized rolling shutter performs illumination multiplexing, and develop a nonlinear optimization algorithm to demultiplex the resulting 3D light transport operator. Using our system, we are able to capture live short and long-range indirect light transport, disambiguate subsurface scattering, diffuse and specular interreflections, and distinguish materials according to their subsurface scattering properties. In particular, we show the utility of indirect imaging for capturing and analyzing the hidden structure of veins in human skin. 
    more » « less
  2. null (Ed.)
    Dynamic 3D reconstruction of surgical cavities is essential in a wide range of computer-assisted surgical intervention applications, including but not limited to surgical guidance, pre-operative image registration and vision-based force estimation. According to a survey on vision based 3D reconstruction for abdominal minimally invasive surgery (MIS) [1], real-time 3D reconstruction and tissue deformation recovery remain open challenges to researchers. The main challenges include specular reflections from the wet tissue surface and the highly dynamic nature of abdominal surgical scenes. This work aims to overcome these obstacles by using multiple viewpoint and independently moving RGB cameras to generate an accurate measurement of tissue deformation at the volume of interest (VOI), and proposes a novel efficient camera pairing algorithm. Experimental results validate the proposed camera grouping and pair sequencing, and were evaluated with the Raven-II [2] surgical robot system for tool navigation, the Medtronic Stealth Station s7 surgical navigation system for real- time camera pose monitoring, and the Space Spider white light scanner to derive the ground truth 3D model. 
    more » « less
  3. null (Ed.)
    It is demonstrated that acoustic transmission through a phononic crystal with anisotropic solid scatterers becomes non-reciprocal if the background fluid is viscous. In an ideal (inviscid) fluid, the transmission along the direction of broken P symmetry is asymmetric. This asymmetry is compatible with reciprocity since time-reversal symmetry ( T symmetry) holds. Viscous losses break T symmetry, adding a non-reciprocal contribution to the transmission coefficient. The non-reciprocal transmission spectra for a phononic crystal of metallic circular cylinders in water are experimentally obtained and analysed. The surfaces of the cylinders were specially processed in order to weakly break P symmetry and increase viscous losses through manipulation of surface features. Subsequently, the non-reciprocal part of transmission is separated from its asymmetric reciprocal part in numerically simulated transmission spectra. The level of non-reciprocity is in agreement with the measure of broken P symmetry. The reported study contradicts commonly accepted opinion that linear dissipation cannot be a reason leading to non-reciprocity. It also opens a way for engineering passive acoustic diodes exploring the natural viscosity of any fluid as a factor leading to non-reciprocity. 
    more » « less
  4. We propose a novel non-line-of-sight (NLOS) imaging framework with long-wave infrared (IR). At long-wave IR wavelengths, certain physical parameters are more favorable for high-fidelity reconstruction. In contrast to prior work in visible light NLOS, at long-wave IR wavelengths, the hidden heat source acts as a light source. This simplifies the problem to a single bounce problem. In addition, surface reflectance has a much stronger specular reflection in the long-wave IR spectrum than in the visible light spectrum. We reformulate a light transport model that leverages these favorable physical properties of long-wave IR. Specifically, we demonstrate 2D shape recovery and 3D localization of a hidden object. Furthermore, we demonstrate near real-time and robust NLOS pose estimation of a human figure, the first such demonstration, to our knowledge. 
    more » « less
  5. Reconstructing 3D objects in natural environments requires solving the ill-posed problem of geometry, spatially-varying material, and lighting estimation. As such, many approaches impractically constrain to a dark environment, use controlled lighting rigs, or use few handheld captures but suffer reduced quality. We develop a method that uses just two smartphone exposures captured in ambient lighting to reconstruct appearance more accurately and practically than baseline methods. Our insight is that we can use a flash/no-flash RGB-D pair to pose an inverse rendering problem using point lighting. This allows efficient differentiable rendering to optimize depth and normals from a good initialization and so also the simultaneous optimization of diffuse environment illumination and SVBRDF material. We find that this reduces diffuse albedo error by 25%, specular error by 46%, and normal error by 30% against single and paired-image baselines that use learning-based techniques. Given that our approach is practical for everyday solid objects, we enable photorealistic relighting for mobile photography and easier content creation for augmented reality. 
    more » « less