An official website of the United States government
Reconstructing 3D objects in natural environments requires solving the ill-posed problem of geometry, spatially-varying material, and lighting estimation. As such, many approaches impractically constrain to a dark environment, use controlled lighting rigs, or use few handheld captures but suffer reduced quality. We develop a method that uses just two smartphone exposures captured in ambient lighting to reconstruct appearance more accurately and practically than baseline methods. Our insight is that we can use a flash/no-flash RGB-D pair to pose an inverse rendering problem using point lighting. This allows efficient differentiable rendering to optimize depth and normals from a good initialization and so also the simultaneous optimization of diffuse environment illumination and SVBRDF material. We find that this reduces diffuse albedo error by 25%, specular error by 46%, and normal error by 30% against single and paired-image baselines that use learning-based techniques. Given that our approach is practical for everyday solid objects, we enable photorealistic relighting for mobile photography and easier content creation for augmented reality.
more »
« less
Sensors in a Flash! Oxygen Nanosensors for Microbial Metabolic Monitoring Synthesized by Flash Nanoprecipitation
- Award ID(s):
- 1944204
- PAR ID:
- 10372034
- Publisher / Repository:
- American Chemical Society
- Date Published:
- Journal Name:
- ACS Sensors
- Volume:
- 7
- Issue:
- 9
- ISSN:
- 2379-3694
- Page Range / eLocation ID:
- p. 2606-2614
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
