More Like this

1. Radio Frequency Fingerprinting on the Edge

   https://doi.org/10.1109/TMC.2021.3064466  

   Jian, Tong; Gong, Yifan; Zhan, Zheng; Shi, Runbin; Soltani, Nasim; Wang, Zifeng; Dy, Jennifer G.; Chowdhury, Kaushik Roy; Wang, Yanzhi; Ioannidis, Stratis (March 2021, IEEE Transactions on Mobile Computing)

   null (Ed.)

   Deep learning methods have been very successful at radio frequency fingerprinting tasks, predicting the identity of transmitting devices with high accuracy. We study radio frequency fingerprinting deployments at resource-constrained edge devices. We use structured pruning to jointly train and sparsify neural networks tailored to edge hardware implementations. We compress convolutional layers by a 27.2× factor while incurring a negligible prediction accuracy decrease (less than 1%). We demonstrate the efficacy of our approach over multiple edge hardware platforms, including a Samsung Galaxy S10 phone and a Xilinx-ZCU104 FPGA. Our method yields significant inference speedups, 11.5× on the FPGA and 3× on the smartphone, as well as high efficiency: the FPGA processing time is 17× smaller than in a V100 GPU. To the best of our knowledge, we are the first to explore the possibility of compressing networks for radio frequency fingerprinting; as such, our experiments can be seen as a means of characterizing the informational capacity associated with this specific learning task. 

   more » « less
2. Model-driven Cluster Resource Management for AI Workloads in Edge Clouds

   https://doi.org/10.1145/3582080  

   Liang, Qianlin; Hanafy, Walid A.; Ali-Eldin, Ahmed; Shenoy, Prashant (March 2023, ACM Transactions on Autonomous and Adaptive Systems)

   Since emerging edge applications such as Internet of Things (IoT) analytics and augmented reality have tight latency constraints, hardware AI accelerators have been recently proposed to speed up deep neural network (DNN) inference run by these applications. Resource-constrained edge servers and accelerators tend to be multiplexed across multiple IoT applications, introducing the potential for performance interference between latency-sensitive workloads. In this article, we design analytic models to capture the performance of DNN inference workloads on shared edge accelerators, such as GPU and edgeTPU, under different multiplexing and concurrency behaviors. After validating our models using extensive experiments, we use them to design various cluster resource management algorithms to intelligently manage multiple applications on edge accelerators while respecting their latency constraints. We implement a prototype of our system in Kubernetes and show that our system can host 2.3× more DNN applications in heterogeneous multi-tenant edge clusters with no latency violations when compared to traditional knapsack hosting algorithms. 

   more » « less
3. Demo Abstract: Lightweight Training and Inference for Self-Supervised Depth Estimation on Edge Devices

   https://doi.org/10.1145/3715014.3724373  

   Farcas, Allen-Jasmin; Marculescu, Radu (May 2025, ACM)

   Not AvailablDeploying monocular depth estimation on resource-constrained edge devices is a significant challenge, particularly when attempting to perform both training and inference concurrently. Current lightweight, self-supervised approaches typically rely on complex frameworks that are hard to implement and deploy in real-world settings. To address this gap, we introduce the first framework for Lightweight Training and Inference (LITI) that combines ready-to-deploy models with streamlined code and fully functional, parallel training and inference pipelines. Our experiments show various models being deployed for inference, training, or both inference and training, leveraging inputs from a real-time RGB camera sensor. Thus, our framework enables training and inference on resource-constrained edge devices for complex applications such as depth estimation. 

   more » « less
4. Automating Deep Neural Network Model Selection for Edge Inference

   https://doi.org/10.1109/CogMI48466.2019.00035  

   Lu, Bingqian; Yang, Jianyi; Chen, Lydia Y.; Ren, Shaolei (December 2019, IEEE First International Conference on Cognitive Machine Intelligence (CogMI))

   The ever increasing size of deep neural network (DNN) models once implied that they were only limited to cloud data centers for runtime inference. Nonetheless, the recent plethora of DNN model compression techniques have successfully overcome this limit, turning into a reality that DNN-based inference can be run on numerous resource-constrained edge devices including mobile phones, drones, robots, medical devices, wearables, Internet of Things devices, among many others. Naturally, edge devices are highly heterogeneous in terms of hardware specification and usage scenarios. On the other hand, compressed DNN models are so diverse that they exhibit different tradeoffs in a multi-dimension space, and not a single model can achieve optimality in terms of all important metrics such as accuracy, latency and energy consumption. Consequently, how to automatically select a compressed DNN model for an edge device to run inference with optimal quality of experience (QoE) arises as a new challenge. The state-of-the-art approaches either choose a common model for all/most devices, which is optimal for a small fraction of edge devices at best, or apply device-specific DNN model compression, which is not scalable. In this paper, by leveraging the predictive power of machine learning and keeping end users in the loop, we envision an automated device-level DNN model selection engine for QoE-optimal edge inference. To concretize our vision, we formulate the DNN model selection problem into a contextual multi-armed bandit framework, where features of edge devices and DNN models are contexts and pre-trained DNN models are arms selected online based on the history of actions and users' QoE feedback. We develop an efficient online learning algorithm to balance exploration and exploitation. Our preliminary simulation results validate our algorithm and highlight the potential of machine learning for automating DNN model selection to achieve QoE-optimal edge inference. 

   more » « less
5. Dělen: Enabling Flexible and Adaptive Model-serving for Multi-tenant Edge AI

   https://doi.org/10.1145/3576842.3582375  

   Liang, Qianlin; Hanafy, Walid A.; Bashir, Noman; Ali-Eldin, Ahmed; Irwin, David; Shenoy, Prashant (May 2023, ACM/IEEE Conference on Internet of Things Design and Implementation)

   Model-serving systems expose machine learning (ML) models to applications programmatically via a high-level API. Cloud plat- forms use these systems to mask the complexities of optimally managing resources and servicing inference requests across multi- ple applications. Model serving at the edge is now also becoming increasingly important to support inference workloads with tight latency requirements. However, edge model serving differs substan- tially from cloud model serving in its latency, energy, and accuracy constraints: these systems must support multiple applications with widely different latency and accuracy requirements on embedded edge accelerators with limited computational and energy resources. To address the problem, this paper presents Dělen,1 a flexible and adaptive model-serving system for multi-tenant edge AI. Dělen exposes a high-level API that enables individual edge applications to specify a bound at runtime on the latency, accuracy, or energy of their inference requests. We efficiently implement Dělen using conditional execution in multi-exit deep neural networks (DNNs), which enables granular control over inference requests, and evalu- ate it on a resource-constrained Jetson Nano edge accelerator. We evaluate Dělen flexibility by implementing state-of-the-art adapta- tion policies using Dělen’s API, and evaluate its adaptability under different workload dynamics and goals when running single and multiple applications. 

   more » « less