An official website of the United States government
AI-powered applications often involve multiple deep neural network (DNN)-based prediction tasks to support application level functionalities. However, executing multi-DNNs can be challenging due to the high resource demands and computation costs that increase linearly with the number of DNNs. Multi-task learning (MTL) addresses this problem by designing a multi-task model that shares parameters across tasks based on a single backbone DNN. This paper explores an alternative approach called model fusion: rather than training a single multi-task model from scratch as MTL does, model fusion fuses multiple task-specific DNNs that are pre-trained separately and can have heterogeneous architectures into a single multi-task model. We materialize model fusion in a software framework called GMorph to accelerate multi- DNN inference while maintaining task accuracy. GMorph features three main technical contributions: graph mutations to fuse multi-DNNs into resource-efficient multi-task models, search-space sampling algorithms, and predictive filtering to reduce the high search costs. Our experiments show that GMorph can outperform MTL baselines and reduce the inference latency of multi-DNNs by 1.1-3X while meeting the target task accuracy.
more »
« less
Leveraging multiplexed metasurfaces for multi-task learning with all-optical diffractive processors
Abstract Diffractive Neural Networks (DNNs) leverage the power of light to enhance computational performance in machine learning, offering a pathway to high-speed, low-energy, and large-scale neural information processing. However, most existing DNN architectures are optimized for single tasks and thus lack the flexibility required for the simultaneous execution of multiple tasks within a unified artificial intelligence platform. In this work, we utilize the polarization and wavelength degrees of freedom of light to achieve optical multi-task identification using the MNIST, FMNIST, and KMNIST datasets. Employing bilayer cascaded metasurfaces, we construct dual-channel DNNs capable of simultaneously classifying two tasks, using polarization and wavelength multiplexing schemes through a meta-atom library. Numerical evaluations demonstrate performance accuracies comparable to those of individually trained single-channel, single-task DNNs. Extending this approach to three-task parallel recognition reveals an expected performance decline yet maintains satisfactory classification accuracies of greater than 80 % for all tasks. We further introduce a novel end-to-end joint optimization framework to redesign the three-task classifier, demonstrating substantial improvements over the meta-atom library design and offering the potential for future multi-channel DNN designs. Our study could pave the way for the development of ultrathin, high-speed, and high-throughput optical neural computing systems.
more »
« less
- Award ID(s):
- 2240448
- PAR ID:
- 10612644
- Publisher / Repository:
- Nanophotonics
- Date Published:
- Journal Name:
- Nanophotonics
- Volume:
- 13
- Issue:
- 24
- ISSN:
- 2192-8614
- Page Range / eLocation ID:
- 4505 to 4517
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
AI-powered applications often involve multiple deep neural network (DNN)-based prediction tasks to support application level functionalities. However, executing multi-DNNs can be challenging due to the high resource demands and computation costs that increase linearly with the number of DNNs. Multi-task learning (MTL) addresses this problem by designing a multi-task model that shares parameters across tasks based on a single backbone DNN. This paper explores an alternative approach called model fusion: rather than training a single multi-task model from scratch as MTL does, model fusion fuses multiple task-specific DNNs that are pre-trained separately and can have heterogeneous architectures into a single multi-task model. We materialize model fusion in a software framework called GMorph to accelerate multi- DNN inference while maintaining task accuracy. GMorph features three main technical contributions: graph mutations to fuse multi-DNNs into resource-efficient multi-task models, search-space sampling algorithms, and predictive filtering to reduce the high search costs. Our experiments show that GMorph can outperform MTL baselines and reduce the inference latency of multi-DNNs by 1.1-3X while meeting the target task accuracy.more » « less
-
Abstract Terahertz (THz) communication is an up‐and‐coming technology for the sixth‐generation wireless network. The realization of ultra‐high‐speed THz communication requires the combination of multi‐dimensional multiplexing schemes, including polarization division multiplexing (PDM), mode division multiplexing (MDM), and wavelength division multiplexing, to increase channel capacity. However, most existing devices for MDM in the THz regime are single‐purpose and incapable of multi‐dimensional modulation. Here, all‐dielectric metasurfaces are designed for 2D multiplexing/demultiplexing, which takes the lead in combining orbital angular momentum (OAM) MDM and PDM in the THz regime. The multi‐functional wavefront phase modulations and interleaved meta‐atom arrangements are used to realize polarization‐selective multichannel OAM mode (de)multiplexing, in which the linear‐polarized 4‐channel and circular‐polarized 6‐channel demultiplexing are experimentally demonstrated. Between different linear‐polarized channels, the measured maximum crosstalk is −16.88 dB, and the isolation of each channel can be greater than 10 dB in a range wider than 0.1 THz. This study paves the way for multi‐dimensional multiplexing in the THz regime, which may benefit extremely high‐capacity and integrated THz communication systems. The proposed design strategy is readily applied to multi‐functional metasurfaces for microwaves and far infrared light, facilitating the development of multiplexing technology and OAM‐related applications.more » « less
-
null (Ed.)FinFET SRAM cells suffer from front-end wearout mechanisms, such as bias temperature instability and hot carrier injection. In this paper, we built a library based on deep neural networks (DNNs) to speed up the process of simulating FinFET SRAM cells' degradation. This library consists of two parts. The first part calculates circuit configuration parameters, wearout parameters, and the other input variables for the DNN. The second part calls for the DNN to determine the shifted circuit performance metrics. A DNN with more than 99% accuracy is achieved with training data from standard Hspice simulations. The correctness of the DNN is also validated in the presence of input variations. With this library, the simulation speed is one hundred times faster than Hspice simulations. We can display the cell's degradation under various configurations easily and quickly. Also, the DNN-based library can help protect intellectual property without showing users the circuit's details.more » « less
-
null (Ed.)Abstract Deep neural networks (DNNs) have substantial computational requirements, which greatly limit their performance in resource-constrained environments. Recently, there are increasing efforts on optical neural networks and optical computing based DNNs hardware, which bring significant advantages for deep learning systems in terms of their power efficiency, parallelism and computational speed. Among them, free-space diffractive deep neural networks (D 2 NNs) based on the light diffraction, feature millions of neurons in each layer interconnected with neurons in neighboring layers. However, due to the challenge of implementing reconfigurability, deploying different DNNs algorithms requires re-building and duplicating the physical diffractive systems, which significantly degrades the hardware efficiency in practical application scenarios. Thus, this work proposes a novel hardware-software co-design method that enables first-of-its-like real-time multi-task learning in D 2 2NNs that automatically recognizes which task is being deployed in real-time. Our experimental results demonstrate significant improvements in versatility, hardware efficiency, and also demonstrate and quantify the robustness of proposed multi-task D 2 NN architecture under wide noise ranges of all system components. In addition, we propose a domain-specific regularization algorithm for training the proposed multi-task architecture, which can be used to flexibly adjust the desired performance for each task.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1515/nanoph-2024-0483
