More Like this

1. AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators

   https://doi.org/10.1109/HPCA47549.2020.00036  

   Song, Linghao; Chen, Fan; Zhuo, Youwei; Qian, Xuehai; Li, Hai; Chen, Yiran (February 2020, 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA))

   Deep neural network (DNN) accelerators as an example of domain-specific architecture have demonstrated great success in DNN inference. However, the architecture acceleration for equally important DNN training has not yet been fully studied. With data forward, error backward and gradient calculation, DNN training is a more complicated process with higher computation and communication intensity. Because the recent research demonstrates a diminishing specialization return, namely, “accelerator wall”, we believe that a promising approach is to explore coarse-grained parallelism among multiple performance-bounded accelerators to support DNN training. Distributing computations on multiple heterogeneous accelerators to achieve high throughput and balanced execution, however, remaining challenging. We present ACCPAR, a principled and systematic method of determining the tensor partition among heterogeneous accelerator arrays. Compared to prior empirical or unsystematic methods, ACCPAR considers the complete tensor partition space and can reveal previously unknown new parallelism configurations. ACCPAR optimizes the performance based on a cost model that takes into account both computation and communication costs of a heterogeneous execution environment. Hence, our method can avoid the drawbacks of existing approaches that use communication as a proxy of the performance. The enhanced flexibility of tensor partitioning in ACCPAR allows the flexible ratio of computations to be distributed among accelerators with different performances. The proposed search algorithm is also applicable to the emerging multi-path patterns in modern DNNs such as ResNet. We simulate ACCPAR on a heterogeneous accelerator array composed of both TPU-v2 and TPU-v3 accelerators for the training of large-scale DNN models such as Alexnet, Vgg series and Resnet series. The average performance improvements of the state-of-the-art “one weird trick” (OWT) and HYPAR, and ACCPAR, normalized to the baseline data parallelism scheme where each accelerator replicates the model and processes different input data in parallel, are 2.98×, 3.78×, and 6.30×, respectively. 

   more » « less
2. SecureLoop: Design Space Exploration of Secure DNN Accelerators

   Kyungmi Lee, Mengjia Yan (October 2023, MICRO’23)

   Deep neural networks (DNNs) are gaining popularity in a wide range of domains, ranging from speech and video recognition to healthcare. With this increased adoption comes the pressing need for securing DNN execution environments on CPUs, GPUs, and ASICs. While there are active research efforts in supporting a trusted execution environment (TEE) on CPUs, the exploration in supporting TEEs on accelerators is limited, with only a few solutions available. A key limitation along this line of work is that these secure DNN accelerators narrowly consider a few specific architectures. The design choices and the associated cost for securing these architectures do not transfer to other diverse architectures. This paper strives to address this limitation by developing a design space exploration tool for supporting TEEs on diverse DNN accelerators. We target secure DNN accelerators equipped with cryptographic engines where the cryptographic operations are closely coupled with the data movement in the accelerators. These operations significantly complicate the scheduling for DNN accelerators, as the scheduling needs to account for the extra on-chip computation and off-chip memory accesses introduced by these cryptographic operations, and even needs to account for potential interactions across DNN layers. We tackle these challenges in our tool, called SecureLoop, by introducing a scheduling search engine with the following attributes: 1) considers the cryptographic overhead associated with every offchip data access, 2) uses an efficient modular arithmetic technique to compute the optimal authentication block assignment for each individual layer, and 3) uses a simulated annealing algorithm to perform cross-layer optimizations. Compared to the conventional schedulers, our tool finds the schedule for secure DNN designs with up to 33.2% speedup and 50.2% improvement of energy-delay product. 

   more » « less
3. A Formalism of DNN Accelerator Flexibility

   https://doi.org/10.1145/3530907  

   Kao, Sheng-Chun; Kwon, Hyoukjun; Pellauer, Michael; Parashar, Angshuman; Krishna, Tushar (May 2022, Proceedings of the ACM on Measurement and Analysis of Computing Systems)

   The high efficiency of domain-specific hardware accelerators for machine learning (ML) has come from specialization, with the trade-off of less configurability/ flexibility. There is growing interest in developing flexible ML accelerators to make them future-proof to the rapid evolution of Deep Neural Networks (DNNs). However, the notion of accelerator flexibility has always been used in an informal manner, restricting computer architects from conducting systematic apples-to-apples design-space exploration (DSE) across trillions of choices. In this work, we formally define accelerator flexibility and show how it can be integrated for DSE. Specifically, we capture DNN accelerator flexibility across four axes: tiling, ordering, parallelization, and array shape. We categorize existing accelerators into 16 classes based on their axes of flexibility support, and define a precise quantification of the degree of flexibility of an accelerator across each axis. We leverage these to develop a novel flexibility-aware DSE framework. We demonstrate how this can be used to perform first-of-their-kind evaluations, including an isolation study to identify the individual impact of the flexibility axes. We demonstrate that adding flexibility features to a hypothetical DNN accelerator designed in 2014 improves runtime on future (i.e., present-day) DNNs by 11.8x geomean. 

   more » « less
4. An Evidence Gap Map Shiny Application for Effect Size or Summary Level Data

   Polanin, J.R.; Zhang, Q.; Taylor, J.; Williams, R.T.; Joshi, M.; Burr, L. (February 2022, Evidence Synthesis & Meta-Analysis in R Conference)

   Systematic reviews and meta-analyses are important techniques because they synthesize results from multiple primary studies on a similar topic. To influence policy, practice, and research, however, synthesis researchers must translate the results for various audiences. Ideally, the translation drives future research agendas, informs policymaking, or assists in practical decision-making. An Evidence Gap Map (EGM), a graphical or tabular visualization of systematic review and meta-analysis results, is one ideal translation technique because it provides a structured framework to assess contexts for which primary evidence is available or to determine whether the effectiveness of an intervention or a program differs across populations, conditions, and settings. To bolster the field and promote the use of EGMs, we provide an overview of what constitutes an informative EGM, detail multiple examples of EGMs using extant meta-analytic results, and present a free R Shiny application we created to easily generate EGMs from typical meta-analytic datasets. We conclude by reviewing education-based systematic reviews that included an EGM to describe the current state of the field. 

   more » « less
5. MAESTRO: A Data-Centric Approach to Understand Reuse, Performance, and Hardware Cost of DNN Mappings

   Kwon, Hyoukjun; Chatarasi, Prasanth; Pellauer, Michael; Parashar, Angshuman; Sarkar, Vivek; Krishna, Tushar (July 2020, IEEE micro)

   The efficiency of an accelerator depends on three factors—mapping, deep neural network (DNN) layers, and hardware—constructing extremely complicated design space of DNN accelerators. To demystify such complicated design space and guide the DNN accelerator design for better efficiency, we propose an analytical cost model, MAESTRO. MAESTRO receives DNN model description and hardware resources information as a list, and mapping described in a data-centric representation we propose as inputs. The data centric representation consists of three directives that enable concise description of mappings in a compiler-friendly form. MAESTRO analyzes various forms of data reuse in an accelerator based on inputs quickly and generates more than 20 statistics including total latency, energy, throughput, etc., as outputs. MAESTRO’s fast analysis enables various optimization tools for DNN accelerators such as hardware design exploration tool we present as an example. 

   more » « less