skip to main content

Title: Foveated thermal computational imaging prototype using all-silicon meta-optics

Foveated imaging provides a better tradeoff between situational awareness (field of view) and resolution, and is critical in long wavelength infrared regimes because of the size, weight, power, and cost of thermal sensors. We demonstrate computational foveated imaging by exploiting the ability of a meta-optical frontend to discriminate between different polarization states and a computational backend to reconstruct the captured image/video. The frontend is a three-element optic: the first element, which we call the “foveal” element, is a metalens that focuses s-polarized light at a distance off1without affecting the p-polarized light; the second element, which we call the “perifovea” element, is another metalens that focuses p-polarized light at a distance off2without affecting thes-polarized light. The third element is a freely rotating polarizer that dynamically changes the mixing ratios between the two polarization states. Both the foveal element (focal length=150mm; diameter=75mm) and the perifoveal element (focal length=25mm; diameter=25mm) were fabricated as polarization-sensitive, all-silicon, meta surfaces resulting in a large-aperture, 1:6 foveal expansion, thermal imaging capability. A computational backend then utilizes a deep image prior to separate the resultant multiplexed image or video into a foveated image consisting of a high resolution center and a lower-resolution large field of view context. We build a prototype system and demonstrate 12 frames per second real-time, thermal, foveated image and video capture..

more » « less
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ;
Publisher / Repository:
Optical Society of America
Date Published:
Journal Name:
Medium: X Size: Article No. 18
["Article No. 18"]
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Metasurfaces offer a unique platform to precisely control optical wavefronts and enable the realization of flat lenses, or metalenses, which have the potential to substantially reduce the size and complexity of imaging systems and to realize new imaging modalities. However, it is a major challenge to create achromatic metalenses that produce a single focal length over a broad wavelength range because of the difficulty in simultaneously engineering phase profiles at distinct wavelengths on a single metasurface. For practical applications, there is a further challenge to create broadband achromatic metalenses that work in the transmission mode for incident light waves with any arbitrary polarization state. We developed a design methodology and created libraries of meta-units—building blocks of metasurfaces—with complex cross-sectional geometries to provide diverse phase dispersions (phase as a function of wavelength), which is crucial for creating broadband achromatic metalenses. We elucidated the fundamental limitations of achromatic metalens performance by deriving mathematical equations that govern the tradeoffs between phase dispersion and achievable lens parameters, including the lens diameter, numerical aperture (NA), and bandwidth of achromatic operation. We experimentally demonstrated several dielectric achromatic metalenses reaching the fundamental limitations. These metalenses work in the transmission mode with polarization-independent focusing efficiencies up to 50% and continuously provide a near-constant focal length overλ = 1200–1650 nm. These unprecedented properties represent a major advance compared to the state of the art and a major step toward practical implementations of metalenses.

    more » « less
  2. Super-resolution (SR) is a well-studied technique for reconstructing high-resolution (HR) images from low-resolution (LR) ones. SR holds great promise for video streaming since an LR video segment can be transmitted from the video server to the client that then reconstructs the HR version using SR, resulting in a significant reduction in network bandwidth. However, SR is seldom used in practice for real-time video streaming, because the computational overhead of frame reconstruction results in large latency and low frame rate. To reduce the computational overhead and make SR practical, we propose a deep-learning-based SR method called Fo veated Cas caded Video Super Resolution (focas). focas relies on the fact that human eyes only have high acuity in a tiny central foveal region of the retina. focas uses more neural network blocks in the foveal region to provide higher video quality, while using fewer blocks in the periphery as lower quality is sufficient. To optimize the computational resources and reduce reconstruction latency, focas formulates and solves a convex optimization problem to decide the number of neural network blocks to use in each region of the frame. Using extensive experiments, we show that focas reduces the latency by 50%-70% while maintaining comparable visual quality as traditional (non-foveated) SR. Further, focas provides a 12-16x reduction in the client-to-server network bandwidth in comparison with sending the full HR video segments. 
    more » « less
  3. We present a foveated rendering method to accelerate the amplitude-only computer-generated hologram (AO-CGH) calculation in a holographic near-eye 3D display. For a given target image, we compute a high-resolution foveal region and a low-resolution peripheral region with dramatically reduced pixel numbers. Our technique significantly improves the computation speed of the AO-CGH while maintaining the perceived image quality in the fovea. Moreover, to accommodate the eye gaze angle change, we develop an algorithm to laterally shift the foveal image with negligible extra computational cost. Our technique holds great promise in advancing the holographic 3D display in real-time use.

    more » « less
  4. TolTEC is an upcoming millimeter-wave imaging polarimeter designed to fill the focal plane of the 50-m-diameter Large Millimeter Telescope (LMT). Combined with the LMT, TolTEC will offer high-angular-resolution (5–10 ) simultaneous, polarization-sensitive observations in three wavelength bands: 1.1, 1.4, and 2.0 mm. Additionally, TolTEC will feature mapping speeds greater than 2 deg2∕mJy2∕h , thus enabling wider surveys of large-scale structure, galaxy evolution, and star formation. These improvements are only possible through the integration of approximately 7000 low-noise, high-responsivity superconducting Lumped Element Kinetic Inductance Detectors. Utilizing three focal planes of detector arrays requires the design, fabrication, and characterization of a unique, large-scale cryogenic system. Based on thermal models and expected photon loading, the focal planes must have a base operational temperature below 150 mK. To achieve this base temperature, TolTEC utilizes two cryocoolers, a Cryomech pulse tube cooler and an Oxford Instruments dilution refrigerator, to establish four thermal stages: 45 K, 4 K, 1 K, and 100 mK. During the design phase, we developed an object-oriented Python code to model the heat loading on each stage as well as the thermal gradients throughout the system. This model has allowed us to improve thermal gradients in the system as well as locate areas of poor thermal conductivity prior to ending a cooldown. The results of our model versus measurements from our cooldowns will be presented along with a detailed overview of TolTEC’s cryogenic system. We anticipate TolTEC to be commissioned at the LMT by Spring 2020. 
    more » « less
  5. Abstract We report an inverse-designed, high numerical aperture (∼0.44), extended depth of focus (EDOF) meta-optic, which exhibits a lens-like point spread function (PSF). The EDOF meta-optic maintains a focusing efficiency comparable to that of a hyperboloid metalens throughout its depth of focus. Exploiting the extended depth of focus and computational post processing, we demonstrate broadband imaging across the full visible spectrum using a 1 mm, f/1 meta-optic. Unlike other canonical EDOF meta-optics, characterized by phase masks such as a log-asphere or cubic function, our design exhibits a highly invariant PSF across ∼290 nm optical bandwidth, which leads to significantly improved image quality, as quantified by structural similarity metrics. 
    more » « less