Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to non-federal websites. Their policies may differ from this site.
-
We propose a jointly optimized edge-based frame-work to multi-user 360→ Virtual Reality (VR) stored videostreaming. At its core is a predictor that estimates a statisticalmodel for users’ future sequences of viewing directions (view-ports) based on a Deep Recurrent Neural Network (DRNN).Such predictions, along with the tracking of users’ past qualityof experience, are used in a novel scheduler that fairly prioritizesthe delivery of the high definition content in viewing directionswhich are most likely to be relevant. We achieve robustnessto view prediction errors by ensuring low definition content for all viewports is always delivered. We achieve robustness to wireless capacity variations by proactively delivering (possibly through multicasting) and caching high definition content whenthe available capacity to users is high.more » « lessFree, publicly-accessible full text available July 27, 2027
-
Collaborative VR and social-metaverse applications present a unique challenge for edge caching due to highly correlated client request patterns. Unlike typical web browsing where requests are independent, VR users might navigate spaces in groups. This generates sequential “leader-follower” requests that defy the traditional Independent Reference Model (IRM) used to design most caches. In this paper, we introduce the Grouped Client Request Model to formally capture these dependencies between client requests. Our analysis shows that standard policies like Least Recently Used (LRU) fail to adapt to these dynamics, often evicting content right before a follower needs it. To solve this, we propose Least Following and Recently Used (LFRU), a policy that dynamically learns the causal relationships driving these request patterns. By prioritizing objects based on inferreddependencies rather than just recency, LFRU significantly improves the cache hit ratio. Evaluations using emulated VR traces show that, in structured following scenarios, LFRU achieves up to 2.9! higher hit ratios than LRU and 1.9! higher than LFU.more » « lessFree, publicly-accessible full text available July 27, 2027
-
Collaborative VR and social-metaverse applicationspresent a unique challenge for edge caching due to highly correlated client request patterns. Unlike typical web browsing whererequests are independent, VR users might navigate spaces in groups. This generates sequential “leader-follower” requests that defy the traditional Independent Reference Model (IRM) used to design most caches. In this paper, we introduce the Grouped Client Request Model to formally capture these dependencies between client requests. Our analysis shows that standard policies like Least Recently Used (LRU) fail to adapt to these dynamics, often evicting content right before a follower needs it. To solve this, we propose Least Following and Recently Used (LFRU), a policy that dynamically learns the causal relationships driving these request patterns. By prioritizing objects based on inferred dependencies rather than just recency, LFRU significantly improves the cache hit ratio. Evaluations using emulated VR traces show that, in structured following scenarios, LFRU achieves up to 2.9x higher hit ratios than LRU and 1.9x higher than LFU.more » « lessFree, publicly-accessible full text available July 27, 2027
-
We propose a jointly optimized edge-based framework to multi-user 360→ Virtual Reality (VR) stored video streaming. At its core is a predictor that estimates a statistical model for users’ future sequences of viewing directions (view-ports) based on a Deep Recurrent Neural Network (DRNN). Such predictions, along with the tracking of users’ past quality of experience, are used in a novel scheduler that fairly prioritizes the delivery of the high definition content in viewing directions which are most likely to be relevant. We achieve robustness to view prediction errors by ensuring low definition content for all viewports is always delivered. We achieve robustness to wireless capacity variations by proactively delivering (possibly through multicasting) and caching high definition content whenthe available capacity to users is high.more » « lessFree, publicly-accessible full text available July 27, 2027
-
With the growing use of Large Language Model (LLM)-based tools like ChatGPT, Perplexity, and Gemini across industries, there is a rising need for efficient LLM inference systems. These systems handle requests with a unique two-phase computation structure: a prefill-phase that processes the full input prompt and a decode-phase that autoregressively generates tokens one at a time. This structure calls for new strategies for routing and scheduling requests. In this paper, we take a comprehensive approach to this challenge by developing a theoretical framework that models routing and scheduling in LLM inference systems. We identify two key design principles-optimal tiling and dynamic resource allocation-that are essential for achieving high throughput. Guided by these principles, we propose the Resource-Aware Dynamic (RAD) scheduler and prove that it achieves throughput optimality under mild conditions. To address practical Service Level Objectives (SLOs) such as serving requests with different Time Between Token (TBT) constraints, we design the SLO-Aware LLM Inference (SLAI) scheduler. SLAI uses real-time measurements to prioritize decode requests that are close to missing their TBT deadlines and reorders prefill requests based on known prompt lengths to further reduce the Time To First Token (TTFT) delays. We evaluate SLAI on the openchat_shareGPT4 dataset using the Mistral-7B model on an NVIDIA RTX ADA 6000 GPU. Compared to Sarathi-Serve, SLAI reduces the median TTFT by 53% and increases the maximum serving capacity by 26% such that median TTFT is below 0.5 seconds, while meeting tail TBT latency constraints. The complete source code is available at: https://github.com/agrimUT/SLAI.more » « lessFree, publicly-accessible full text available December 1, 2026
-
EXtended Reality (XR) is rapidly gaining popularity. XR systems are often networked, in the sense that one or more client devices such as headsets or glasses fetch data from one or more servers over a (typically wireless last-link) network. A crucial step to advancing the design of XR systems is realizing reproducible performance evaluation based on a set of representative benchmark test scenarios. In this article, we define what constitutes an XR benchmark, including XR environment representations, user views and actions, system capabilities and conditions, and evaluation metrics for user Quality of Experience (QoE) and for system resource consumption. We believe that community involvement in creating such benchmarks can significantly accelerate the development of usable and cost-effective XR technology.more » « less
-
The effective management of the vast amounts of data processed or required by modern cloud and edge computing systems remains a fundamental challenge. This paper focuses on cache management for applications where data objects can be stored in layered representations. In such representations, each additional data layer enhances the “quality” of the object’s version, albeit at the cost of increased memory usage. This layered approach is advantageous in various scenarios, including the delivery of zoomable maps, video coding, future virtual reality gaming, and layered neural network models, where additional data layers improve quality/inference accuracy. In systems where users or devices request different versions of a data object, layered representations provide the flexibility needed for caching policies to achieve improved hit rates, i.e., delivering the specific representations required by users. This paper investigates the performance of the Least Recently Used (LRU) caching policy in the context of layered representation for data, referred to as Layered LRU (LLRU). To this end, we develop an asymptotically accurate analytical model for LLRU. We analyze how LLRU’s performance is influenced by factors such as the number of layers, as well as the popularity and size of an object’s layers. For example, our results demonstrate that, in the case of LLRU, adding more layers does not always enhance performance. Instead, the effectiveness of LLRU depends intricately on the popularity distribution and size characteristics of the layers.more » « less
-
The effective management of the vast amounts of data processed or required by modern cloud and edge computing systems remains a fundamental challenge. This paper focuses on cache management for applications where data objects can be stored in layered representations. In such representations, each additional data layer enhances the “quality” of the object’s version, albeit at the cost of increased memory usage. This layered approach is advantageous in various scenarios, including the delivery of zoomable maps, video coding, future virtual reality gaming, and layered neural network models, where additional data layers improve quality/inference accuracy. In systems where users or devices request different versions of a data object, layered representations provide the flexibility needed for caching policies to achieve improved hit rates, i.e., delivering the specific representations required by users. This paper investigates the performance of the Least Recently Used (LRU) caching policy in the context of layered representation for data, referred to as Layered LRU (LLRU). To this end, we develop an asymptotically accurate analytical model for LLRU. We analyze how LLRU’s performance is influenced by factors such as the number of layers, as well as the popularity and size of an object’s layers. For example, our results demonstrate that, in the case of LLRU, adding more layers does not always enhance performance. Instead, the effectiveness of LLRU depends intricately on the popularity distribution and size characteristics of the layers.more » « less
-
We consider a hierarchical inference system with multiple clients connected to a server via a shared communication resource. When necessary, clients with low-accuracy machine learning models can offload classification tasks to a server for processing on a high-accuracy model. We propose a distributed online offloading algorithm which maximizes the accuracy subject to a shared resource utilization constraint thus indirectly realizing accuracy-delay tradeoffs possible given an underlying network scheduler. The proposed algorithm, named Lyapunov-EXP4, introduces a loss structure based on Lyapunov-drift minimization techniques to the bandits with expert advice framework. We prove that the algorithm converges to a near-optimal threshold policy on the confidence of the clients’ local inference without prior knowledge of the system’s statistics and efficiently solves a constrained bandit problem with sublinear regret. We further consider settings where clients may employ multiple thresholds, allowing more aggressive optimization of overall accuracy at a possible loss in fairness. Extensive simulation results on real and synthetic data demonstrate convergence of Lyapunov-EXP4, and show themore » « less
An official website of the United States government

Full Text Available