An official website of the United States government
This paper investigates the combined potential of neuromorphic and edge computing to develop a flexible machine learning (ML) system designed for processing data from dynamic vision sensors. We build and train hybrid models that integrate spiking neural networks (SNNs) and artificial neural networks (ANNs) using the PyTorch and Lava frameworks. We explore the effects of quantization on ANN models to assess its impact on both accuracy and energy efficiency. Additionally, we address the challenges of deploying hybrid models on hardware by implementing individual components on specific edge platforms. We also propose an accumulator circuit to bridge the spiking and non-spiking domains. Comprehensive performance analyses are conducted on a heterogeneous system of neuromorphic and edge AI hardware, assessing accuracy, latency, and energy consumption. Our results show that hybrid spiking networks improve accuracy and energy efficiency. Moreover, we find that quantization improves hybrid networks, further reducing energy consumption while boosting accuracy.
more »
« less
Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements
An emerging use-case of machine learning (ML) is to train a model on a high-performance system and deploy the trained model on energy-constrained embedded systems. Neuromorphic hardware platforms, which operate on principles of the biological brain, can significantly lower the energy overhead of a machine learning inference task, making these platforms an attractive solution for embedded ML systems. We present a design-technology tradeoff analysis to implement such inference tasks on the processing elements (PEs) of a Non-Volatile Memory (NVM)-based neuromorphic hardware. Through detailed circuit-level simulations at scaled process technology nodes, we show the negative impact of technology scaling on the information-processing latency, which impacts the quality-of-service (QoS) of an embedded ML system. At a finer granularity, the latency inside a PE depends on 1) the delay introduced by parasitic components on its current paths, and 2) the varying delay to sense different resistance states of its NVM cells. Based on these two observations, we make the following three contributions. First, on the technology front, we propose an optimization scheme where the NVM resistance state that takes the longest time to sense is set on current paths having the least delay, and vice versa, reducing the average PE latency, which improves the QoS. Second, on the architecture front, we introduce isolation transistors within each PE to partition it into regions that can be individually power-gated, reducing both latency and energy. Finally, on the system-software front, we propose a mechanism to leverage the proposed technological and architectural enhancements when implementing a machine-learning inference task on neuromorphic PEs of the hardware. Evaluations with a recent neuromorphic hardware architecture show that our proposed design-technology co-optimization approach improves both performance and energy efficiency of machine-learning inference tasks without incurring high cost-per-bit.
more »
« less
- Award ID(s):
- 1937419
- PAR ID:
- 10357641
- Date Published:
- Journal Name:
- ACM Transactions on Embedded Computing Systems
- ISSN:
- 1539-9087
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The architecture of a coarse-grained reconfigurable array (CGRA) processing element (PE) has a significant effect on the performance and energy-efficiency of an application running on the CGRA. This paper presents APEX, an automated approach for generating specialized PE architectures for an application or an application domain. APEX first analyzes application domain benchmarks using frequent subgraph mining to extract commonly occurring computational subgraphs. APEX then generates specialized PEs by merging subgraphs using a datapath graph merging algorithm. The merged datapath graphs are translated into a PE specification from which we automatically generate the PE hardware description in Verilog along with a compiler that maps applications to the PE. The PE hardware and compiler are inserted into a flexible CGRA generation and compilation toolchain that allows for agile evaluation of CGRAs. We evaluate APEX for two domains, machine learning and image processing. For image processing applications, our automatically generated CGRAs with specialized PEs achieve from 5% to 30% less area and from 22% to 46% less energy compared to a general-purpose CGRA. For machine learning applications, our automatically generated CGRAs consume 16% to 59% less energy and 22% to 39% less area than a general-purpose CGRA. This work paves the way for creation of application domain-driven design-space exploration frameworks that automatically generate efficient programmable accelerators, with a much lower design effort for both hardware and compiler generation.more » « less
-
The Software-Defined Networking (SDN) paradigm has significantly improved network efficiency by integrating machine learning (ML) capabilities into the control plane (CP). This integration allows adaptive management of network resources in response to dynamic traffic conditions. However, the significant geographical distance between the CP and data plane (DP) causes considerable round-trip latency, measured in milliseconds, which poses a major challenge to time-sensitive traffic. To address this challenge, our research proposes a novel in-network Reinforcement Learning (RL) inference framework. This framework extends programmability from the CP to the DP, enabling timely and precise control of network resources to meet the stringent Quality of Service (QoS) requirements of mission-critical applications. The in-network RL inference is implemented using match-action tables within the Protocol Independent Switch Architecture (PISA) framework in the DP and validated through hardware deployment of protocol-independent packet processors (P4). To allocate bandwidth precisely based on QoS requirements, a P4 meter extern is used to differentiate the unique demands of individual traffic flows. Our enhanced deepdeterministic policy gradient (eDDPG)-based RL inference achieves superior performance with minimal processing overhead, reducing latency by 88.7% and jitter by 89.1% compared to systems without RL inference.more » « less
-
This paper explores the synergistic potential of neuromorphic and edge computing to create a versatile machine learning (ML) system tailored for processing data captured by dynamic vision sensors. We construct and train hybrid models, blending spiking neural networks (SNNs) and artificial neural networks (ANNs) using PyTorch and Lava frameworks. Our hybrid architecture integrates an SNN for temporal feature extraction and an ANN for classification. We delve into the challenges of deploying such hybrid structures on hardware. Specifically, we deploy individual components on Intel's Neuromorphic Processor Loihi (for SNN) and Jetson Nano (for ANN). We also propose an accumulator circuit to transfer data from the spiking to the non-spiking domain. Furthermore, we conduct comprehensive performance analyses of hybrid SNN-ANN models on a heterogeneous system of neuromorphic and edge AI hardware, evaluating accuracy, latency, power, and energy consumption. Our findings demonstrate that the hybrid spiking networks surpass the baseline ANN model across all metrics and outperform the baseline SNN model in accuracy and latency.more » « less
-
The success of machine learning has prospered Machine-Learning-as-a-Service (MLaaS) - deploying trained machine learning (ML) models in cloud to provide low latency inference services at scale. To meet latency Service-Level-Objective (SLO), judicious parallelization at both request and operation levels is utterly important. However, existing ML systems (e.g., Tensorflow) and cloud ML serving platforms (e.g., SageMaker) are SLO-agnostic and rely on users to manually configure the parallelism. To provide low latency ML serving, this paper proposes a swift machine learning serving scheduling framework with a novel Region-based Reinforcement Learning (RRL) approach. RRL can efficiently identify the optimal parallelism configuration under different workloads by estimating performance of similar configurations with that of the known ones. We both theoretically and experimentally show that the RRL approach can outperform state-of-the-art approaches by finding near optimal solutions over 8 times faster while reducing inference latency up to 79.0% and reducing SLO violation up to 49.9%.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1145/3524068
