More Like this

1. A Neural Particle Level Set Method for Dynamic Interface Tracking

   https://doi.org/10.1145/3730399  

   Chen, Duowen; Zhou, Junwei; Zhu, Bo (May 2025, ACM Transactions on Graphics)

   We propose a neural particle level set (Neural PLS) method to accommodate tracking and evolving dynamic neural representations. At the heart of our approach is a set of oriented particles serving dual roles of interface trackers and sampling seeders. These dynamic particles are used to evolve the interface and construct neural representations on a multi-resolution grid-hash structure to hybridize coarse sparse distance fields and multi-scale feature encoding. Based on these parallel implementations and neural-network-friendly architectures, our neural particle level set method combines the computational merits on both ends of the traditional particle level sets and the modern implicit neural representations, in terms of feature representation and dynamic tracking. We demonstrate the efficacy of our approach by showcasing its performance surpassing traditional level-set methods in both benchmark tests and physical simulations. 

   more » « less
2. Binarized Depthwise Separable Neural Network for Object Tracking in FPGA

   https://doi.org/10.1145/3299874.3318034  

   Yang, Li; He, Zhezhi; Fan, Deliang (May 2019, 2019 on Great Lakes Symposium on VLSI)

   Object tracking has achieved great advances in the past few years and has been widely applied in vision-based application. Nowadays, deep convolutional neural network has taken an important role in object tracking tasks. However, its enormous model size and massive computation cost have became the main obstacle for deployment of such powerful algorithm in low power and resource limited embedded system, such as FPGA. Due to the popularization of the power-sensitive mobile platform, low power real-time tracking solution is strongly required. In order to address these challenges, we propose a low power and energy-efficient object tracking FPGA implementation based on a newly proposed binarized depthwise separable deep convolutional neural network. It can significantly reduce the model size and computation complexity simultaneously utilizing binarized (i.e., +1 and -1) depthwise separable convolution kernel and our proposed trainable threshold group binarization activation function. It can completely converts the dot product and accumulation based convolution operations into bit-wise XNOR and bit-count operations, while achieving state-of-the-art accuracy. Our proposed binarized depthwise separable model achieves ~57% Intersection over Union (IOU) on DJI object tracking dataset with only ~143.9Kb model parameter size. We then deploy our proposed model into the Xilinx PYNQ Z1 board with only 4.9Mb on-chip RAM. The experiment results show that our FPGA implementation achieves 11.1 frames per second for object tracking with only 2.61W. 

   more » « less
3. HSIM-DNN: Hardware Simulator for Computation-, Storage- and Power-Efficient Deep Neural Networks

   https://doi.org/10.1145/3299874.3317996  

   Sun, Mengshu; Zhao, Pu; Wang, Yanzhi; Chang, Naehyuck; Lin, Xue (May 2019, the 2019 on Great Lakes Symposium on VLSI)

   Deep learning that utilizes large-scale deep neural networks (DNNs) is effective in automatic high-level feature extraction but also computation and memory intensive. Constructing DNNs using block-circulant matrices can simultaneously achieve hardware acceleration and model compression while maintaining high accuracy. This paper proposes HSIM-DNN, an accurate hardware simulator on the C++ platform, to simulate the exact behavior of DNN hardware implementations and thereby facilitate the block-circulant matrix-based design of DNN training and inference procedures in hardware. Real FPGA implementations validate the simulator with various circulant block sizes and data bit lengths taking into account accuracy, compression ratio and power consumption, which provides excellent insights for hardware design. 

   more » « less
4. Design and FPGA Implementation of an Adaptive Video Subsampling Algorithm for Energy-Efficient Single Object Tracking

   Iqbal, Odrika; Siddiqui, Saquib; Martin, Joshua; Katoch, Sameeksha; Spanias, Andreas; Bliss, Daniel; Jayasuriya, Suren (January 2020, IEEE International Conference on Image Processing)

   Image sensors with programmable region-of-interest (ROI) readout are a new sensing technology important for energyefficient embedded computer vision. In particular, ROIs can subsample the number of pixels being readout while performing single object tracking in a video. In this paper, we develop adaptive sampling algorithms which perform joint object tracking and predictive video subsampling. We utilize an object detection consisting of either mean shift tracking or a neural network, coupled with a Kalman filter for prediction. We show that our algorithms achieve mean average precision of 0.70 or higher on a dataset of 20 videos in software. Further, we implement hardware acceleration of mean shift tracking with Kalman filter adaptive subsampling on an FPGA. Hardware results show a 23× improvement in clock cycles and latency as compared to baseline methods and achieves 38FPS real-time performance. This research points to a new domain of hardware-software co-design for adaptive video subsampling in embedded computer vision. 

   more » « less
5. HF-LDPC: HLS-friendly QC-LDPC FPGA Decoder with High Throughput and Flexibility

   https://doi.org/10.1109/ICCD58817.2023.00091  

   Zhang, Yifan; Cao, Qiang; Wang, Shaohua; Yao, Jie; Jiang, Hong (November 2023, IEEE)

   LDPC (Low-Density Parity-Check) codes have become a cornerstone of transforming a noise-filled physical channel into a reliable and high-performance data channel in communication and storage systems. FPGA (Field-Programmable Gate Array) based LDPC hardware, especially for decoding with high complexity, is essential to realizing the high-bandwidth channel prototypes. HLS (High-Level Synthesis) is introduced to speed up the FPGA development of LDPC hardware by automatically compiling high-level abstract behavioral descriptions into RTL-level implementations, but often sub-optimally due to lacking effective low-level descriptions. To overcome this problem, this paper proposes an HLS-friendly QC-LDPC FPGA decoder architecture, HF-LDPC, that employs HLS not only to precisely characterize high-level behaviors but also to effectively optimize low-level RTL implementation, thus achieving both high throughput and flexibility. First, HF-LDPC designs a multi-unit framework with a balanced I/O-computing dataflow to adaptively match code parameters with FPGA configurations. Second, HFLDPC presents a novel fine-grained task-level pipeline with interleaved updating to eliminate stalls due to data interdependence within each updating task. HF-LDPC also presents several HLSenhanced approaches. We implement and evaluate HF-LDPC on Xilinx U50, which demonstrates that HF-LDPC outperforms existing implementations by 4× to 84× with the same parameter and linearly scales to up to 116 Gbps actual decoding throughput with high hardware efficiency. 

   more » « less