More Like this

1. CloVR: Fast-Startup Low-Latency Cloud VR

   https://doi.org/10.1109/TVCG.2024.3372059  

   Zhou, Yuqi; Popescu, Voicu (May 2024, IEEE Transactions on Visualization and Computer Graphics)
2. Low latency RNN inference with cellular batching

   https://doi.org/10.1145/3190508.3190541  

   Gao, Pin; Yu, Lingfan; Wu, Yongwei; Li, Jinyang (April 2018, EuroSys '18: Proceedings of the Thirteenth EuroSys Conference)

   Performing inference on pre-trained neural network models must meet the requirement of low-latency, which is often at odds with achieving high throughput. Existing deep learning systems use batching to improve throughput, which do not perform well when serving Recurrent Neural Networks with dynamic dataflow graphs. We propose the technique of cellular batching, which improves both the latency and throughput of RNN inference. Unlike existing systems that batch a fixed set of dataflow graphs, cellular batching makes batching decisions at the granularity of an RNN "cell" (a subgraph with shared weights) and dynamically assembles a batched cell for execution as requests join and leave the system. We implemented our approach in a system called BatchMaker. Experiments show that BatchMaker achieves much lower latency and also higher throughput than existing systems. 

   more » « less
3. Stochastic Dividers for Low Latency Neural Networks

   https://doi.org/10.1109/TCSI.2021.3103926  

   Liu, Shanshan; Tang, Xiaochen; Niknia, Farzad; Reviriego, Pedro; Liu, Weiqiang; Louri, Ahmed; Lombardi, Fabrizio (October 2021, IEEE Transactions on Circuits and Systems I: Regular Papers)
4. PULS: Processor-Supported Ultra-Low Latency Scheduling

   https://doi.org/10.1145/3209582.3209609  

   Yau, Simon; Hsieh, Ping-Chun; Bhattacharyya, Rajarshi; Bhargav, K. R.; Shakkottai, Srinivas; Hou, I-Hong; Kumar, P. R. (January 2018, Nineteenth International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc 2018))
5. A Low-Latency Fft-Ifft Cascade Architecture

   https://doi.org/10.1109/ICASSP48485.2024.10447370  

   Parhi, Keshab K. (April 2024, ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))

   This paper addresses the design of a partly-parallel cascaded FFT-IFFT architecture that does not require any intermediate buffer. Folding can be used to design partly-parallel architectures for FFT and IFFT. While many cascaded FFT-IFFT architectures can be designed using various folding sets for the FFT and the IFFT, for a specified folded FFT architecture, there exists a unique folding set to design the IFFT architecture that does not require an intermediate buffer. Such a folding set is designed by processing the output of the FFT as soon as possible (ASAP) in the folded IFFT. Elimination of the intermediate buffer reduces latency and saves area. The proposed approach is also extended to interleaved processing of multi-channel time-series. The proposed FFT-IFFT cascade architecture saves about N/2 memory elements and N/4 clock cycles of latency compared to a design with identical folding sets. For the 2-interleaved FFT-IFFT cascade, the memory and latency savings are, respectively, N/2 units and N/2 clock cycles, compared to a design with identical folding sets. 

   more » « less