An official website of the United States government
Digital camera pixels measure image intensities by converting incident light energy into an analog electrical current, and then digitizing it into a fixed-width binary representation. This direct measurement method, while conceptually simple, suffers from limited dynamic range and poor performance under extreme illumination — electronic noise dominates under low illumination, and pixel full-well capacity results in saturation under bright illumination. We propose a novel intensity cue based on measuring inter-photon timing, defined as the time delay between detection of successive photons. Based on the statistics of inter-photon times measured by a time-resolved single-photon sensor, we develop theory and algorithms for a scene brightness estimator which works over extreme dynamic range; we experimentally demonstrate imaging scenes with a dynamic range of over ten million to one. The proposed techniques, aided by the emergence of single-photon sensors such as single-photon avalanche diodes (SPADs) with picosecond timing resolution, will have implications for a wide range of imaging applications: robotics, consumer photography, astronomy, microscopy and biomedical imaging.
more »
« less
Snapshot multidimensional photography through active optical mapping
Abstract Multidimensional photography can capture optical fields beyond the capability of conventional image sensors that measure only two-dimensional (2D) spatial distribution of light. By mapping a high-dimensional datacube of incident light onto a 2D image sensor, multidimensional photography resolves the scene along with other information dimensions, such as wavelength and time. However, the application of current multidimensional imagers is fundamentally restricted by their static optical architectures and measurement schemes—the mapping relation between the light datacube voxels and image sensor pixels is fixed. To overcome this limitation, we propose tunable multidimensional photography through active optical mapping. A high-resolution spatial light modulator, referred to as an active optical mapper, permutes and maps the light datacube voxels onto sensor pixels in an arbitrary and programmed manner. The resultant system can readily adapt the acquisition scheme to the scene, thereby maximising the measurement flexibility. Through active optical mapping, we demonstrate our approach in two niche implementations: hyperspectral imaging and ultrafast imaging.
more »
« less
- Award ID(s):
- 2053080
- PAR ID:
- 10200626
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Compressed ultrafast photography (CUP) is an emerging potent technique that allows imaging a nonrepeatable or difficult‐to‐produce transient event in a single shot. Despite many recent advances, existing CUP techniques operate only at visible and near‐infrared wavelengths. In addition, spatial encoding via a digital micromirror device (DMD) in CUP systems often limits its field of view and imaging speeds. Finally, conventional reconstruction algorithms have limited control of the reconstruction process to further improve the image quality in the recovered datacubes of the scene. To overcome these limitations, this article reports a single‐shot UV‐CUP that exhibits a sequence depth of up to 1500 frames with a size of 1750 × 500 pixels at an imaging speed of 0.5 trillion frames per second. A patterned photocathode is integrated into a streak camera, which overcomes the previous restrictions in DMD‐based spatial encoding and improves the system's compactness. Meanwhile, the plug‐and‐play alternating direction method of multipliers algorithm is implemented to CUP's image reconstruction to enhance reconstructed image quality. UV‐CUP's single‐shot ultrafast imaging ability is demonstrated by recording UV pulses transmitting through various spatial patterns. UV‐CUP is expected to find many applications in both fundamental and applied science.more » « less
-
Photonics provides a promising approach for image processing by spatial filtering, with the advantage of faster speeds and lower power consumption compared to electronic digital solutions. However, traditional optical spatial filters suffer from bulky form factors that limit their portability. Here we present a new approach based on pixel arrays of plasmonic directional image sensors, designed to selectively detect light incident along a small, geometrically tunable set of directions. The resulting imaging systems can function as optical spatial filters without any external filtering elements, leading to extreme size miniaturization. Furthermore, they offer the distinct capability to perform multiple filtering operations at the same time, through the use of sensor arrays partitioned into blocks of adjacent pixels with different angular responses. To establish the image processing capabilities of these devices, we present a rigorous theoretical model of their filter transfer function under both coherent and incoherent illumination. Next, we use the measured angle-resolved responsivity of prototype devices to demonstrate two examples of relevant functionalities: (1) the visualization of otherwise invisible phase objects and (2) spatial differentiation with incoherent light. These results are significant for a multitude of imaging applications ranging from microscopy in biomedicine to object recognition for computer vision.more » « less
-
Abstract Wavefront sensing is the simultaneous measurement of the amplitude and phase of an incoming optical field. Traditional wavefront sensors such as Shack-Hartmann wavefront sensor (SHWFS) suffer from a fundamental tradeoff between spatial resolution and phase estimation and consequently can only achieve a resolution of a few thousand pixels. To break this tradeoff, we present a novel computational-imaging-based technique, namely, the Wavefront Imaging Sensor with High resolution (WISH). We replace the microlens array in SHWFS with a spatial light modulator (SLM) and use a computational phase-retrieval algorithm to recover the incident wavefront. This wavefront sensor can measure highly varying optical fields at more than 10-megapixel resolution with the fine phase estimation. To the best of our knowledge, this resolution is an order of magnitude higher than the current noninterferometric wavefront sensors. To demonstrate the capability of WISH, we present three applications, which cover a wide range of spatial scales. First, we produce the diffraction-limited reconstruction for long-distance imaging by combining WISH with a large-aperture, low-quality Fresnel lens. Second, we show the recovery of high-resolution images of objects that are obscured by scattering. Third, we show that WISH can be used as a microscope without an objective lens. Our study suggests that the designing principle of WISH, which combines optical modulators and computational algorithms to sense high-resolution optical fields, enables improved capabilities in many existing applications while revealing entirely new, hitherto unexplored application areas.more » « less
-
Abstract A new device termed “Optomagnetic Micromirror Arrays” (OMA) is demonstrated capable of mapping the stiffness distribution of biomimetic materials across a 5.1 mm × 7.2 mm field of view with cellular resolution. The OMA device comprises an array of 50 000 magnetic micromirrors with optical grating structures embedded beneath an elastic PDMS film, with biomimetic materials affixed on top. Illumination of a broadband white light beam onto these micromirrors results in the reflection of microscale rainbow light rays on each micromirror. When a magnetic field is applied, it causes each micromirror to tilt differently depending on the local stiffness of the biomimetic materials. Through imaging these micromirrors with low N.A. optics, a specific narrow band of reflection light rays from each micromirror is captured. Changing a micromirror's tilt angle also alters the color spectrum it reflects back to the imaging system and the color of the micromirror image it represents. As a result, OMA can infer the local stiffness of the biomimetic materials through the color change detected on each micromirror. OMA offers the potential for high‐throughput stiffness mapping at the tissue‐level while maintaining spatial resolution at the cellular level.more » « less
