More Like this

1. CondenseNet: An Efficient DenseNet using Learned Group Convolutions

   Huang, Gao; Shichen, Liu; Van der Maaten, Laurens; Weinberger, Kilian (April 2018, CVPR 2018)

   Deep neural networks are increasingly used on mobile devices, where computational resources are limited. In this paper we develop CondenseNet, a novel network architec- ture with unprecedented efficiency. It combines dense con- nectivity between layers with a mechanism to remove un- used connections. The dense connectivity facilitates feature re-use in the network, whereas learned group convolution- s remove connections between layers for which this feature re-use is superfluous. At test time, our model can be imple- mented using standard grouped convolutions—allowing for efficient computation in practice. Our experiments demon- strate that CondenseNets are much more efficient than state- of-the-art compact convolutional networks such as Mo- bileNets and ShuffleNets. 

   more » « less
2. CondenseNet: An Efficient DenseNet using Learned Group Convolutions

   Gao Huang, Shichen Liu (June 2018, CVPR)

   Deep neural networks are increasingly used on mobile devices, where computational resources are limited. In this paper we develop CondenseNet, a novel network architec- ture with unprecedented efficiency. It combines dense con- nectivity between layers with a mechanism to remove un- used connections. The dense connectivity facilitates feature re-use in the network, whereas learned group convolution- s remove connections between layers for which this feature re-use is superfluous. At test time, our model can be imple- mented using standard grouped convolutions—allowing for efficient computation in practice. Our experiments demon- strate that CondenseNets are much more efficient than state- of-the-art compact convolutional networks such as Mo- bileNets and ShuffleNets. 

   more » « less
3. EQ-ViT: Algorithm-Hardware Co-Design for End-to-End Acceleration of Real-Time Vision Transformer Inference on Versal ACAP Architecture

   Dong, Peiyan; Zhuang, Jinming; Yang, Zhuoping; Ji, Shixin; Li, Yanyu; Xu, Dongkuan; Huang, Heng; Hu, Jingtong; Jones, Alex K; Shi, Yiyu; et al (October 2024, IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS)

   While Vision Transformers (ViTs) have shown consistent progress in computer vision, deploying them for real-time decision-making scenarios (< 1 ms) is challenging. Current computing platforms like CPUs, GPUs, or FPGA-based solutions struggle to meet this deterministic low-latency real-time requirement, even with quantized ViT models. Some approaches use pruning or sparsity to reduce model size and latency, but this often results in accuracy loss. To address the aforementioned constraints, in this work, we propose EQ-ViT, an end-to-end acceleration framework with novel algorithm and architecture co-design features to enable real-time ViT acceleration on AMD Versal Adaptive Compute Acceleration Platform (ACAP). The contributions are four-fold. First, we perform in-depth kernel- level performance profiling & analysis and explain the bottlenecks for existing acceleration solutions on GPU, FPGA, and ACAP. Second, on the hardware level, we introduce a new spatial and heterogeneous accelerator architecture, EQ-ViT architec- ture. This architecture leverages the heterogeneous features of ACAP, where both FPGA and artificial intelligence engines (AIEs) coexist on the same system-on-chip (SoC). Third, On the algorithm level, we create a comprehensive quantization-aware training strategy, EQ-ViT algorithm. This strategy concurrently quantizes both weights and activations into 8-bit integers, aiming to improve accuracy rather than compromise it during quanti- zation. Notably, the method also quantizes nonlinear functions for efficient hardware implementation. Fourth, we design EQ- ViT automation framework to implement the EQ-ViT architec- ture for four different ViT applications on the AMD Versal ACAP VCK190 board, achieving accuracy improvement with 2.4%, and average speedups of 315.0x, 3.39x, 3.38x, 14.92x, 59.5x, 13.1x over computing solutions of Intel Xeon 8375C vCPU, Nvidia A10G, A100, Jetson AGX Orin GPUs, and AMD ZCU102, U250 FPGAs. The energy efficiency gains are 62.2x, 15.33x, 12.82x, 13.31x, 13.5x, 21.9x. 

   more » « less
4. Inertial Navigation on Extremely Resource-Constrained Platforms: Methods, Opportunities and Challenges

   https://doi.org/10.1109/PLANS53410.2023.10139997  

   Saha, Swapnil Sayan; Du, Yayun; Sandha, Sandeep Singh; Garcia, Luis Antonio; Jawed, Mohammad Khalid; Srivastava, Mani (April 2023, 2023 IEEE/ION Position, Location and Navigation Symposium (PLANS))

   Inertial navigation provides a small footprint, low-power, and low-cost pathway for localization in GPS-denied environments on extremely resource-constrained Internet-of-Things (IoT) platforms. Traditionally, application-specific heuristics and physics-based kinematic models are used to mitigate the curse of drift in inertial odometry. These techniques, albeit lightweight, fail to handle domain shifts and environmental non-linearities. Recently, deep neural-inertial sequence learning has shown superior odometric resolution in capturing non-linear motion dynamics without human knowledge over heuristic-based methods. These AI-based techniques are data-hungry, suffer from excessive resource usage, and cannot guarantee following the underlying system physics. This paper highlights the unique methods, opportunities, and challenges in porting real-time AI-enhanced inertial navigation algorithms onto IoT platforms. First, we discuss how platform-aware neural architecture search coupled with ultra-lightweight model backbones can yield neural-inertial odometry models that are 31–134 x smaller yet achieve or exceed the localization resolution of state-of-the-art AI-enhanced techniques. The framework can generate models suitable for locating humans, animals, underwater sensors, aerial vehicles, and precision robots. Next, we showcase how techniques from neurosymbolic AI can yield physics-informed and interpretable neural-inertial navigation models. Afterward, we present opportunities for fine-tuning pre-trained odometry models in a new domain with as little as 1 minute of labeled data, while discussing inexpensive data collection and labeling techniques. Finally, we identify several open research challenges that demand careful consideration moving forward. 

   more » « less
5. Time-Stamped Language Model: Teaching Language Models to Understand The Flow of Events

   Rajaby Faghihi, Hossein; Kordjamshidi, Parisa (April 2021, The 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-2021))

   null (Ed.)

   Tracking entities throughout a procedure de- scribed in a text is challenging due to the dy- namic nature of the world described in the pro- cess. Firstly, we propose to formulate this task as a question answering problem. This en- ables us to use pre-trained transformer-based language models on other QA benchmarks by adapting those to the procedural text un- derstanding. Secondly, since the transformer- based language models cannot encode the flow of events by themselves, we propose a Time- Stamped Language Model (TSLM model) to encode event information in LMs architec- ture by introducing the timestamp encoding. Our model evaluated on the Propara dataset shows improvements on the published state- of-the-art results with a 3.1% increase in F1 score. Moreover, our model yields better re- sults on the location prediction task on the NPN-Cooking dataset. This result indicates that our approach is effective for procedural text understanding in general. 

   more » « less