More Like this

1. Getting the Best Bang For Your Buck: Choosing What to Evaluate for Faster Bayesian Optimization

   Iqbal, Shahriar; Su, Jianhai; Kotthoff, Lars; Jamshidi, Pooyan (June 2022, 1st International Conference on Automated Machine Learning)

   Machine learning system design frequently necessitates balancing multiple objectives, such as prediction error and energy consumption, for deep neural networks (DNNs). Typically, no single design performs well across all objectives; thus, finding Pareto-optimal designs is of interest. Measuring different objectives frequently incurs different costs; for example, measuring the prediction error of DNNs is significantly more expensive than measuring the energy consumption of a pre-trained DNN because it requires re-training the DNN. Current state-of-the-art methods do not account for this difference in objective evaluation cost, potentially wasting costly evaluations of objective functions for little information gain. To address this issue, we propose a novel cost-aware decoupled approach that weights the improvement of the hypervolume of the Pareto region by the measurement cost of each objective. To evaluate our approach, we perform experiments on several machine learning systems deployed on energy constraints environments. 

   more » « less
2. FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization Approach for Deep Neural Networks

   https://doi.org/10.1613/jair.1.14139  

   Iqbal, Md Shahriar; Su, Jianhai; Kotthoff, Lars; Jamshidi, Pooyan (May 2023, Journal of Artificial Intelligence Research)

   The design of machine learning systems often requires trading off different objectives, for example, prediction error and energy consumption for deep neural networks (DNNs). Typically, no single design performs well in all objectives; therefore, finding Pareto-optimal designs is of interest. The search for Pareto-optimal designs involves evaluating designs in an iterative process, and the measurements are used to evaluate an acquisition function that guides the search process. However, measuring different objectives incurs different costs. For example, the cost of measuring the prediction error of DNNs is orders of magnitude higher than that of measuring the energy consumption of a pre-trained DNN as it requires re-training the DNN. Current state-of-the-art methods do not consider this difference in objective evaluation cost, potentially incurring expensive evaluations of objective functions in the optimization process. In this paper, we develop a novel decoupled and cost-aware multi-objective optimization algorithm, which we call Flexible Multi-Objective Bayesian Optimization (FlexiBO) to address this issue. For evaluating each design, FlexiBO selects the objective with higher relative gain by weighting the improvement of the hypervolume of the Pareto region with the measurement cost of each objective. This strategy, therefore, balances the expense of collecting new information with the knowledge gained through objective evaluations, preventing FlexiBO from performing expensive measurements for little to no gain. We evaluate FlexiBO on seven state-of-the-art DNNs for image recognition, natural language processing (NLP), and speech-to-text translation. Our results indicate that, given the same total experimental budget, FlexiBO discovers designs with 4.8% to 12.4% lower hypervolume error than the best method in state-of-the-art multi-objective optimization. 

   more » « less
3. Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization

   https://doi.org/10.1109/TETC.2023.3346944  

   Balaskas, Konstantinos; Karatzas, Andreas; Sad, Christos; Siozios, Kostas; Anagnostopoulos, Iraklis; Zervakis, Georgios; Henkel, J¨org (January 2024, IEEE Transactions on Emerging Topics in Computing)

   Deep Neural Networks (DNNs) have shown significant advantages in a wide variety of domains. However, DNNs are becoming computationally intensive and energy hungry at an exponential pace, while at the same time, there is a vast demand for running sophisticated DNN-based services on resource constrained embedded devices. In this paper, we target energy-efficient inference on embedded DNN accelerators. To that end, we propose an automated framework to compress DNNs in a hardware-aware manner by jointly employing pruning and quantization. We explore, for the first time, per-layer fine- and coarse-grained pruning, in the same DNN architecture, in addition to low bit-width mixed-precision quantization for weights and activations. Reinforcement Learning (RL) is used to explore the associated design space and identify the pruning-quantization configuration so that the energy consumption is minimized whilst the prediction accuracy loss is retained at acceptable levels. Using our novel composite RL agent we are able to extract energy-efficient solutions without requiring retraining and/or fine-tuning. Our extensive experimental evaluation over widely used DNNs and the CIFAR-10/100 and ImageNet datasets demonstrates that our framework achieves 39% average energy reduction for 1.7% average accuracy loss and outperforms significantly the state-of-the-art approaches. 

   more » « less
4. LiteNoC: Developing Low-Cost Network-on-Chips for Deep Neural Networks

   Ho, Khoa; Biglari, Siamak; Garrigus, Justin; Zhao, Hui; Mohanty, Saraju (June 2025, ACM)

   With the rapid advance in Deep Neural Networks (DNNs), GPU’s role as a hardware accelerator becomes increasingly important. Due to the GPU’s significant power consumption, developing high- performance and power-efficient GPU systems is a critical challenge. DNN applications need to move a large amount of data between memory and the processing cores, which consumes a great amount of NoC power. However, prior proposed lossless data compressions cannot achieve optimal performance and energy efficiency because they did not take advantage of the error resilience of DNNs. In this work, we propose an NoC architecture that can reduce power consumption without compromising performance and accu- racy. Our technique takes advantage of the error resilience of DNNs as well as the data locality in the floating-point data representation of DNNs. Each data packet is reorganized by grouping data with similar bits such as in the exponents, and redundant bits are sent only once. We further compress the mantissa fields by appropri- ately selecting "proxy" values for data sharing the same exponent. Our evaluation results show that the proposed technique can ef- fectively reduce the amount of data transmitted and lead to better performance and power trade-offs while preserving accuracy. 

   more » « less
5. ALERT: Accurate Learning for Energy and Timeliness

   Chengcheng Wan, Muhammad Santriaji (July 2020, Proceedings of the 2020 USENIX Annual Technical Conference)

   An increasing number of software applications incorporate runtime Deep Neural Networks (DNNs) to process sensor data and return inference results to humans. Effective deployment of DNNs in these interactive scenarios requires meeting latency and accuracy constraints while minimizing energy, a problem exacerbated by common system dynamics. %nature of computation resources and the accuracy, latency, or energy constraints. Prior approaches handle dynamics through either (1) system-oblivious DNN adaptation, which adjusts DNN latency/accuracy tradeoffs, or (2) application-oblivious system adaptation, which adjusts resources to change latency/energy tradeoffs. In contrast, this paper improves on the state-of-the-art by coordinating application- and system-level adaptation. ALERT, our runtime scheduler, uses a probabilistic model to detect environmental volatility and then simultaneously select both a DNN and a system resource configuration to meet latency, accuracy, and energy constraints. We evaluate ALERT on CPU and GPU platforms for image and speech tasks in dynamic environments. ALERT's holistic approach achieves more than 13% energy reduction, and 27% error reduction over prior approaches that adapt solely at the application or system level. Furthermore, ALERT incurs only 3% more energy consumption and 2% higher DNN-inference error than an oracle scheme with perfect application and system knowledge. 

   more » « less