Image processing plays a vital role in artificial visual systems, which have diverse applications in areas such as biomedical imaging and machine vision. In particular, optical analog image processing is of great interest because of its parallel processing capability and low power consumption. Here, we present ultra-compact metasurfaces performing all-optical geometric image transformations, which are essential for image processing to correct image distortions, create special image effects, and morph one image into another. We show that our metasurfaces can realize binary image transformations by modifying the spatial relationship between pixels and converting binary images from Cartesian to log-polar coordinates with unparalleled advantages for scale- and rotation-invariant image preprocessing. Furthermore, we extend our approach to grayscale image transformations and convert an image with Gaussian intensity profile into another image with flat-top intensity profile. Our technique will potentially unlock new opportunities for various applications such as target tracking and laser manufacturing.
This content will become publicly available on December 1, 2025
Signal processing is of critical importance for various science and technology fields. Analog optical processing can provide an effective solution to perform large-scale and real-time data processing, superior to its digital counterparts, which have the disadvantages of low operation speed and large energy consumption. As an important branch of modern optics, Fourier optics exhibits great potential for analog optical image processing, for instance for edge detection. While these operations have been commonly explored to manipulate the spatial content of an image, mathematical operations that act directly over the angular spectrum of an image have not been pursued. Here, we demonstrate manipulation of the angular spectrum of an image, and in particular its differentiation, using dielectric metasurfaces operating across the whole visible spectrum. We experimentally show that this technique can be used to enhance desired portions of the angular spectrum of an image. Our approach can be extended to develop more general angular spectrum analog meta-processors, and may open opportunities for optical analog data processing and biological imaging.
more » « less- Award ID(s):
- 2106752
- NSF-PAR ID:
- 10518301
- Publisher / Repository:
- Nature Communications
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 15
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract Light’s ability to perform massive linear operations in parallel has recently inspired numerous demonstrations of optics-assisted artificial neural networks (ANN). However, a clear system-level advantage of optics over purely digital ANN has not yet been established. While linear operations can indeed be optically performed very efficiently, the lack of nonlinearity and signal regeneration require high-power, low-latency signal transduction between optics and electronics. Additionally, a large power is needed for lasers and photodetectors, which are often neglected in the calculation of the total energy consumption. Here, instead of mapping traditional digital operations to optics, we co-designed a hybrid optical-digital ANN, that operates on incoherent light, and is thus amenable to operations under ambient light. Keeping the latency and power constant between a purely digital ANN and a hybrid optical-digital ANN, we identified a low-power/latency regime, where an optical encoder provides higher classification accuracy than a purely digital ANN. We estimate our optical encoder enables ∼10 kHz rate operation of a hybrid ANN with a power of only 23 mW. However, in that regime, the overall classification accuracy is lower than what is achievable with higher power and latency. Our results indicate that optics can be advantageous over digital ANN in applications, where the overall performance of the ANN can be relaxed to prioritize lower power and latency.
-
more » « less
Abstract Label-free optical microscopy has matured as a noninvasive tool for biological imaging; yet, it is criticized for its lack of specificity, slow acquisition and processing times, and weak and noisy optical signals that lead to inaccuracies in quantification. We introduce FOCALS (Fast Optical Coherence, Autofluorescence Lifetime imaging, and Second harmonic generation) microscopy capable of generating NAD(P)H fluorescence lifetime, second harmonic generation (SHG), and polarization-sensitive optical coherence microscopy (OCM) images simultaneously. Multimodal imaging generates quantitative metabolic and morphological profiles of biological samples in vitro, ex vivo, and in vivo. Fast analog detection of fluorescence lifetime and real-time processing on a graphical processing unit enables longitudinal imaging of biological dynamics. We detail the effect of optical aberrations on the accuracy of FLIM beyond the context of undistorting image features. To compensate for the sample-induced aberrations, we implemented a closed-loop single-shot sensorless adaptive optics solution, which uses computational adaptive optics of OCM for wavefront estimation within 2 s and improves the quality of quantitative fluorescence imaging in thick tissues. Multimodal imaging with complementary contrasts improves the specificity and enables multidimensional quantification of the optical signatures in vitro, ex vivo, and in vivo, fast acquisition and real-time processing improve imaging speed by 4–40 × while maintaining enough signal for quantitative nonlinear microscopy, and adaptive optics improves the overall versatility, which enable FOCALS microscopy to overcome the limits of traditional label-free imaging techniques.
-
more » « less
Abstract Conventional imaging and recognition systems require an extensive amount of data storage, pre-processing, and chip-to-chip communications as well as aberration-proof light focusing with multiple lenses for recognizing an object from massive optical inputs. This is because separate chips (
i .e ., flat image sensor array, memory device, and CPU) in conjunction with complicated optics should capture, store, and process massive image information independently. In contrast, human vision employs a highly efficient imaging and recognition process. Here, inspired by the human visual recognition system, we present a novel imaging device for efficient image acquisition and data pre-processing by conferring the neuromorphic data processing function on a curved image sensor array. The curved neuromorphic image sensor array is based on a heterostructure of MoS2and poly(1,3,5-trimethyl-1,3,5-trivinyl cyclotrisiloxane). The curved neuromorphic image sensor array features photon-triggered synaptic plasticity owing to its quasi-linear time-dependent photocurrent generation and prolonged photocurrent decay, originated from charge trapping in the MoS2-organic vertical stack. The curved neuromorphic image sensor array integrated with a plano-convex lens derives a pre-processed image from a set of noisy optical inputs without redundant data storage, processing, and communications as well as without complex optics. The proposed imaging device can substantially improve efficiency of the image acquisition and recognition process, a step forward to the next generation machine vision. -
more » « less
Abstract Optical metasurfaces performing analog image processing – such as spatial differentiation and edge detection – hold the potential to reduce processing times and power consumption, while avoiding bulky 4 F lens systems. However, current designs have been suffering from trade-offs between spatial resolution, throughput, polarization asymmetry, operational bandwidth, and isotropy. Here, we show that dispersion engineering provides an elegant way to design metasurfaces where all these critical metrics are simultaneously optimized. We experimentally demonstrate silicon metasurfaces performing isotropic and dual-polarization edge detection, with numerical apertures above 0.35 and spectral bandwidths of 35 nm around 1500 nm. Moreover, we introduce quantitative metrics to assess the efficiency of these devices. Thanks to the low loss nature and dual-polarization response, our metasurfaces feature large throughput efficiencies, approaching the theoretical maximum for a given NA. Our results pave the way for low-loss, high-efficiency and broadband optical computing and image processing with free-space metasurfaces.
