More Like this

1. AI on the Edge: Characterizing AI-based IoT Applications Using Specialized Edge Architectures

   https://doi.org/10.1109/IISWC50251.2020.00023  

   Liang, Qianlin ; Shenoy, Prashant ; Irwin, David ( October 2020 , IEEE International Symposium on Workload Characterization (IISWC))

   null (Ed.)

   Edge computing has emerged as a popular paradigm for supporting mobile and IoT applications with low latency or high bandwidth needs. The attractiveness of edge computing has been further enhanced due to the recent availability of special-purpose hardware to accelerate specific compute tasks, such as deep learning inference, on edge nodes. In this paper, we experimentally compare the benefits and limitations of using specialized edge systems, built using edge accelerators, to more traditional forms of edge and cloud computing. Our experimental study using edge-based AI workloads shows that today's edge accelerators can provide comparable, and in many cases better, performance, when normalized for power or cost, than traditional edge and cloud servers. They also provide latency and bandwidth benefits for split processing, across and within tiers, when using model compression or model splitting, but require dynamic methods to determine the optimal split across tiers. We find that edge accelerators can support varying degrees of concurrency for multi-tenant inference applications, but lack isolation mechanisms necessary for edge cloud multi-tenant hosting. 

   more » « less
2. MaGNAS: A Mapping-Aware Graph Neural Architecture Search Framework for Heterogeneous MPSoC Deployment

   https://doi.org/10.1145/3609386  

   Odema, Mohanad ; Bouzidi, Halima ; Ouarnoughi, Hamza ; Niar, Smail ; Al Faruque, Mohammad Abdullah ( October 2023 , ACM Transactions on Embedded Computing Systems)

   Graph Neural Networks (GNNs) are becoming increasingly popular for vision-based applications due to their intrinsic capacity in modeling structural and contextual relations between various parts of an image frame. On another front, the rising popularity of deep vision-based applications at the edge has been facilitated by the recent advancements in heterogeneous multi-processor Systems on Chips (MPSoCs) that enable inference under real-time, stringent execution requirements. By extension, GNNs employed for vision-based applications must adhere to the same execution requirements. Yet contrary to typical deep neural networks, the irregular flow of graph learning operations poses a challenge to running GNNs on such heterogeneous MPSoC platforms. In this paper, we propose a novel unifieddesign-mappingapproach for efficient processing of vision GNN workloads on heterogeneous MPSoC platforms. Particularly, we develop MaGNAS, a mapping-aware Graph Neural Architecture Search framework. MaGNAS proposes a GNN architectural design space coupled with prospective mapping options on a heterogeneous SoC to identify model architectures that maximize on-device resource efficiency. To achieve this, MaGNAS employs a two-tier evolutionary search to identify optimalGNNsandmappingpairings that yield the best performance trade-offs. Through designing a supernet derived from the recent Vision GNN (ViG) architecture, we conducted experiments on four (04) state-of-the-art vision datasets using both (i) a real hardware SoC platform (NVIDIA Xavier AGX) and (ii) a performance/cost model simulator for DNN accelerators. Our experimental results demonstrate that MaGNAS is able to provide1.57× latency speedup and is3.38× more energy-efficient for several vision datasets executed on the Xavier MPSoC vs. the GPU-only deployment while sustaining an average0.11%accuracy reduction from the baseline.

    

   more » « less
3. Dynamic Heterogeneous Voltage Regulation for Systolic Array-Based DNN Accelerators

   https://doi.org/10.1109/ICCD50377.2020.00088  

   Chen, Jianhao ; Riad, Joseph ; Sanchez-Sinencio, Edgar ; Li, Peng ( October 2020 , 2020 IEEE 38th International Conference on Computer Design (ICCD))

   null (Ed.)

   With the growing performance and wide application of deep neural networks (DNNs), recent years have seen enormous efforts on DNN accelerator hardware design for platforms from mobile devices to data centers. The systolic array has been a popular architectural choice for many proposed DNN accelerators with hundreds to thousands of processing elements (PEs) for parallel computing. Systolic array-based DNN accelerators for datacenter applications have high power consumption and nonuniform workload distribution, which makes power delivery network (PDN) design challenging. Server-class multicore processors have benefited from distributed on-chip voltage regulation and heterogeneous voltage regulation (HVR) for improving energy efficiency while guaranteeing power delivery integrity. This paper presents the first work on HVR-based PDN architecture and control for systolic array-based DNN accelerators. We propose to employ a PDN architecture comprising heterogeneous on-chip and off-chip voltage regulators and multiple power domains. By analyzing patterns of typical DNN workloads via a modeling framework, we propose a DNN workload-aware dynamic PDN control policy to maximize system energy efficiency while ensuring power integrity. We demonstrate significant energy efficiency improvements brought by the proposed PDN architecture, dynamic control, and power gating, which lead to a more than five-fold reduction of leakage energy and PDN energy overhead for systolic array DNN accelerators. 

   more » « less
4. Resource Management in Mobile Edge Computing: A Comprehensive Survey

   https://doi.org/10.1145/3589639  

   Zhang, Xiaojie ; Debroy, Saptarshi ( December 2023 , ACM Computing Surveys)

   With the evolution of 5G and Internet of Things technologies, Mobile Edge Computing (MEC) has emerged as a major computing paradigm. Compared to cloud computing, MEC integrates network control, computing, and storage to customizable, fast, reliable, and secure distributed services that are closer to the user and data site. Although a popular research topic, MEC resource management comes in many forms due to its emerging nature and there exists little consensus in the community. In this survey, we present a comprehensive review of existing research problems and relevant solutions within MEC resource management. We first describe the major problems in MEC resource allocation when the user applications have diverse performance requirements. We discuss the unique challenges caused by the dynamic nature of the environments and use cases where MEC is adopted. We also explore and categorize existing solutions that address such challenges. We particularly explore traditional optimization-based methods and deep learning-based approaches. In addition, we take a deeper dive into the most popular applications and use cases that adopt MEC paradigm and how MEC provides customized solutions for each use cases, in particular, video analytics applications. Finally, we outline the open research challenges and future directions. 1 

   more » « less
5. Flow-level Dynamic Bandwidth Allocation in SDN-enabled Edge Cloud using Heuristic Reinforcement Learning

   Arslan Qadeer, Myung Lee ( January 2021 , IEEE Conference, on Future Internet of Things and Cloud)

   Edge Cloud (EC) is poised to brace massive machine type communication (mMTC) for 5G and IoT by providing compute and network resources at the edge. Yet, the EC being regionally domestic with a smaller scale, faces the challenges of bandwidth and computational throughput. Resource management techniques are considered necessary to achieve efficient resource allocation objectives. Software Defined Network (SDN) enabled EC architecture is emerging as a potential solution that enables dynamic bandwidth allocation and task scheduling for latency sensitive and diverse mobile applications in the EC environment. This study proposes a novel Heuristic Reinforcement Learning (HRL) based flowlevel dynamic bandwidth allocation framework and validates it through end-to-end implementation using OpenFlow meter feature. OpenFlow meter provides granular control and allows demand-based flow management to meet the diverse QoS requirements germane to IoT traffics. The proposed framework is then evaluated by emulating an EC scenario based on real NSF COSMOS testbed topology at The City College of New York. A specific heuristic reinforcement learning with linear-annealing technique and a pruning principle are proposed and compared with the baseline approach. Our proposed strategy performs consistently in both Mininet and hardware OpenFlow switches based environments. The performance evaluation considers key metrics associated with real-time applications: throughput, end-to-end delay, packet loss rate, and overall system cost for bandwidth allocation. Furthermore, our proposed linear annealing method achieves faster convergence rate and better reward in terms of system cost, and the proposed pruning principle remarkably reduces control traffic in the network. 

   more » « less