An official website of the United States government
Diffractive optical neural networks (DONNs) are emerging as high‐throughput and energy‐efficient hardware platforms to perform all‐optical machine learning (ML) in machine vision systems. However, the current demonstrated applications of DONNs are largely image classification tasks, which undermine the prospect of developing and utilizing such hardware for other ML applications. Herein, the deployment of an all‐optical reconfigurable DONNs system for scientific computing is demonstrated numerically and experimentally, including guiding two‐dimensional quantum material synthesis, predicting the properties of two‐dimensional quantum materials and small molecular cancer drugs, predicting the device response of nanopatterned integrated photonic power splitters, and the dynamic stabilization of an inverted pendulum with reinforcement learning. Despite a large variety of input data structures, a universal feature engineering approach is developed to convert categorical input features to images that can be processed in the DONNs system. The results open up new opportunities for employing DONNs systems for a broad range of ML applications.
more »
« less
Physics‐Aware Machine Learning and Adversarial Attack in Complex‐Valued Reconfigurable Diffractive All‐Optical Neural Network
Abstract Diffractive optical neural networks have shown promising advantages over electronic circuits for accelerating modern machine learning (ML) algorithms. However, it is challenging to achieve fully programmable all‐optical implementation and rapid hardware deployment. Here, a large‐scale, cost‐effective, complex‐valued, and reconfigurable diffractive all‐optical neural networks system in the visible range is demonstrated based on cascaded transmissive twisted nematic liquid crystal spatial light modulators. The employment of categorical reparameterization technique creates a physics‐aware training framework for the fast and accurate deployment of computer‐trained models onto optical hardware. Such a full stack of hardware and software enables not only the experimental demonstration of classifying handwritten digits in standard datasets, but also theoretical analysis and experimental verification of physics‐aware adversarial attacks onto the system, which are generated from a complex‐valued gradient‐based algorithm. The detailed adversarial robustness comparison with conventional multiple layer perceptrons and convolutional neural networks features a distinct statistical adversarial property in diffractive optical neural networks. The developed full stack of software and hardware provides new opportunities of employing diffractive optics in a variety of ML tasks and in the research on optical adversarial ML.
more »
« less
- PAR ID:
- 10369882
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Laser & Photonics Reviews
- Volume:
- 16
- Issue:
- 12
- ISSN:
- 1863-8880
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Deep neural networks (DNNs) have substantial computational requirements, which greatly limit their performance in resource-constrained environments. Recently, there are increasing efforts on optical neural networks and optical computing based DNNs hardware, which bring significant advantages for deep learning systems in terms of their power efficiency, parallelism and computational speed. Among them, free-space diffractive deep neural networks (D 2 NNs) based on the light diffraction, feature millions of neurons in each layer interconnected with neurons in neighboring layers. However, due to the challenge of implementing reconfigurability, deploying different DNNs algorithms requires re-building and duplicating the physical diffractive systems, which significantly degrades the hardware efficiency in practical application scenarios. Thus, this work proposes a novel hardware-software co-design method that enables first-of-its-like real-time multi-task learning in D 2 2NNs that automatically recognizes which task is being deployed in real-time. Our experimental results demonstrate significant improvements in versatility, hardware efficiency, and also demonstrate and quantify the robustness of proposed multi-task D 2 NN architecture under wide noise ranges of all system components. In addition, we propose a domain-specific regularization algorithm for training the proposed multi-task architecture, which can be used to flexibly adjust the desired performance for each task.more » « less
-
Multilayer diffractive optical neural networks (DONNs) can perform machine learning (ML) tasks at the speed of light with low energy consumption. Decreasing the number of diffractive layers can reduce inevitable material and diffraction losses to improve system performance, and incorporating compact devices can reduce the system footprint. However, current analytical DONN models cannot accurately describe such physical systems. Here we show the ever-ignored effects of interlayer reflection and interpixel interaction on the deployment performance of DONNs through full-wave electromagnetic simulations and terahertz (THz) experiments. We demonstrate that the drop of handwritten digit classification accuracy due to reflection is negligible with conventional low-index THz polymer materials, while it can be substantial with high-index materials. We further show that one- and few-layer DONN systems can achieve high classification accuracy, but there is a trade-off between accuracy and model-system matching rate because of the fast-varying spatial distribution of optical responses in diffractive masks. Deep DONNs can break down such a trade-off because of reduced mask spatial complexity. Our results suggest that new accurate and trainable DONN models are needed to advance the development and deployment of compact DONN systems for sophisticated ML tasks.more » « less
-
Abstract Solving partial differential equations (PDEs) is the cornerstone of scientific research and development. Data-driven machine learning (ML) approaches are emerging to accelerate time-consuming and computation-intensive numerical simulations of PDEs. Although optical systems offer high-throughput and energy-efficient ML hardware, their demonstration for solving PDEs is limited. Here, we present an optical neural engine (ONE) architecture combining diffractive optical neural networks for Fourier space processing and optical crossbar structures for real space processing to solve time-dependent and time-independent PDEs in diverse disciplines, including Darcy flow equation, the magnetostatic Poisson’s equation in demagnetization, the Navier-Stokes equation in incompressible fluid, Maxwell’s equations in nanophotonic metasurfaces, and coupled PDEs in a multiphysics system. We numerically and experimentally demonstrate the capability of the ONE architecture, which not only leverages the advantages of high-performance dual-space processing for outperforming traditional PDE solvers and being comparable with state-of-the-art ML models but also can be implemented using optical computing hardware with unique features of low-energy and highly parallel constant-time processing irrespective of model scales and real-time reconfigurability for tackling multiple tasks with the same architecture. The demonstrated architecture offers a versatile and powerful platform for large-scale scientific and engineering computations.more » « less
-
Abstract Free‐space optical systems are emerging as a hardware platform for high‐throughput and energy‐efficient computing. In this review, the pioneering works are first introduced to lay the foundation for the principles and architectures of systems. The modern hardware implementations of two types of optical computing systems, matrix, and vector multiplication systems and diffractive optical neural network systems, are covered from material, device, and system perspectives. Further, the system deployment to various applications is also discussed. This review serves as an introduction and guideline to the current progress of developing and utilizing free‐space optical computing systems in various domains.more » « less
