skip to main content

This content will become publicly available on June 16, 2023

Title: Millimeter Wave and Free-Space-Optics for Future Dual-Connectivity 6DOF Mobile Multi-User VR Streaming
Dual-connectivity streaming is a key enabler of next generation six Degrees Of Freedom (6DOF) Virtual Reality (VR) scene immersion. Indeed, using conventional sub-6 GHz WiFi only allows to reliably stream a low-quality baseline representation of the VR content, while emerging high-frequency communication technologies allow to stream in parallel a high-quality user viewport-specific enhancement representation that synergistically integrates with the baseline representation, to deliver high-quality VR immersion. We investigate holistically as part of an entire future VR streaming system two such candidate emerging technologies, Free Space Optics (FSO) and millimeter-Wave (mmWave) that benefit from a large available spectrum to deliver unprecedented data rates. We analytically characterize the key components of the envisioned dual-connectivity 6DOF VR streaming system that integrates in addition edge computing and scalable 360° video tiling, and we formulate an optimization problem to maximize the immersion fidelity delivered by the system, given the WiFi and mmWave/FSO link rates, and the computing capabilities of the edge server and the users’ VR headsets. This optimization problem is mixed integer programming of high complexity and we formulate a geometric programming framework to compute the optimal solution at low complexity. We carry out simulation experiments to assess the performance of the proposed system more » using actual 6DOF navigation traces from multiple mobile VR users that we collected. Our results demonstrate that our system considerably advances the traditional state-of-the-art and enables streaming of 8K-120 frames-per-second (fps) 6DOF content at high fidelity. « less
; ; ;
Award ID(s):
Publication Date:
Journal Name:
ACM Transactions on Multimedia Computing, Communications, and Applications
Sponsoring Org:
National Science Foundation
More Like this
  1. We investigate a novel communications system that integrates scalable multi-layer 360-degree video tiling, viewport-adaptive rate-distortion optimal resource allocation, and VR-centric edge computing and caching, to enable future high-quality untethered VR streaming. Our system comprises a collection of 5G small cells that can pool their communication, computing, and storage resources to collectively deliver scalable 360-degree video content to mobile VR clients at much higher quality. Our major contributions are rigorous design of multi-layer 360-degree tiling and related models of statistical user navigation, and analysis and optimization of edge-based multi-user VR streaming that integrates viewport adaptation and server cooperation. We also explore the possibility of network coded data operation and its implications for the analysis, optimization, and system performance we pursue here. We demonstrate considerable gains in delivered immersion fidelity, featuring much higher 360-degree viewport peak signal to noise ratio (PSNR) and VR video frame rates and spatial resolutions.
  2. The goal of this study was to evaluate driver risk behavior in response to changes in their risk perception inputs, specifically focusing on the effect of augmented visual representation technologies. This experiment was conducted for the purely real-driving scenario, establishing a baseline by which future, augmented visual representation scenarios can be compared. Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) simulation technologies have rapidly improved over the last three decades to where, today, they are widely used and more heavily relied upon than before, particularly in the areas of training, research, and design. The resulting utilization of these capabilities has proven simulation technologies to be a versatile and powerful tool. Virtual immersion, however, introduces a layer of abstraction and safety between the participant and the designed artifact, which includes an associated risk compensation. Quantifying and modeling the relationship between this risk compensation and levels of virtual immersion is the greater goal of this project. This study focuses on the first step, which is to determine the level of risk perception for a purely real environment for a specific man-machine system - a ground vehicle – operated in a common risk scenario – traversing a curve at high speeds.more »Specifically, passengers are asked to assess whether the vehicle speed within a constant-radius curve is perceived as comfortable. Due to the potential for learning effects to influence risk perception, the experiment was split into two separate protocols: the latent response protocol and the learned response protocol. The latent response protocol applied to the first exposure of an experimental condition to the subject. It consisted of having the subjects in the passenger seat assess comfort or discomfort within a vehicle that was driven around a curve at a randomlychosen value among a selection of test speeds; subjects were asked to indicate when they felt uncomfortable by pressing a brake pedal that was instrumented to alert the driver. Next, the learned response protocol assessed the subjects for repeated exposures but allowing subjects to use brake and throttle pedals to indicate if they wanted to go faster or slower; the goal was to allow subjects to iterate toward their maximum comfortable speed. These pedals were instrumented to alert the driver who responded accordingly. Both protocols were repeated for a second curve with a different radius. Questionnaires were also administered after each trial that addressed the subjective perception of risk and provided a means to substantiate the measured risk compensation behavior. The results showed that, as expected, the latent perception of risk for a passenger traversing a curve was higher than the learned perception for successive exposures to the same curve; in other words, as drivers ‘learned’ a curve, they were more comfortable with higher speeds. Both the latent and learned speeds provide a suitable metric by which to compare future replications of this experiment at different levels of virtual immersion. Correlations were found between uncomfortable subject responses and the yaw acceleration of the vehicle. Additional correlation of driver discomfort was found to occur at specific locations on the curves. The yaw acceleration is a reflection of the driver’s ability to maintain a steady steering input, whereas the location on the curve was found to correlate with variations in the lane-markings and environmental cues.« less
  3. The extreme bandwidth and performance of 5G mobile networks changes the way we develop and utilize digital services. Within a few years, 5G will not only touch technology and applications, but dramatically change the economy, our society and individual life. One of the emerging technologies that enables the evolution to 5G by bringing cloud capabilities near to the end users is Edge Computing or also known as Multi-Access Edge Computing (MEC) that will become pertinent towards the evolution of 5G. This evolution also entails growth in the threat landscape and increase privacy in concerns at different application areas, hence security and privacy plays a central role in the evolution towards 5G. Since MEC application instantiated in the virtualized infrastructure, in this paper we present a distributed application that aims to constantly introspect multiple virtual machines (VMs) in order to detect malicious activities based on their anomalous behavior. Once suspicious processes detected, our IDS in real-time notifies system administrator about the potential threat. Developed software is able to detect keyloggers, rootkits, trojans, process hiding and other intrusion artifacts via agent-less operation, by operating remotely or directly from the host machine. Remote memory introspection means no software to install, no notice tomore »malware to evacuate or destroy data. Experimental results of remote VMI on more than 50 different malicious code demonstrate average anomaly detection rate close to 97%. We have established wide testbed environment connecting networks of two universities Kyushu Institute of Technology and The City College of New York through secure GRE tunnel. Conducted experiments on this testbed deliver high response time of the proposed system.« less
  4. Delivering videos under less-than-ideal network conditions without compromising end-users' quality of experiences is a hard problem. Virtually all prior work follow a piecemeal approach---either "tweaking" the fully reliable transport layer or making the client "smarter." We propose VOXEL, a cross-layer optimization system for video streaming. We use VOXEL to demonstrate how to combine application-provided "insights" with a partially reliable protocol for optimizing video streaming. To this end, we present a novel ABR algorithm that explicitly trades off losses for improving end-users' video-watching experiences. VOXEL is fully compatible with DASH, and backward-compatible with VOXEL-unaware servers and clients. In our experiments emulating a wide range of network conditions, VOXEL outperforms the state-of-the-art: We stream videos in the 90th-percentile with up to 97% less rebuffering than the state-of-the-art without sacrificing visual fidelity. We also demonstrate the benefits of VOXEL for small-buffer regimes like the emerging use case of low-latency and live streaming. In a survey of 54 real users, 84% of the participants indicated that they prefer videos streamed using VOXEL compared to the state-of-the-art.
  5. Finding the connected components of a graph is a fundamental prob- lem with uses throughout computer science and engineering. The task of computing connected components becomes more difficult when graphs are very large, or when they are dynamic, meaning the edge set changes over time subject to a stream of edge inser- tions and deletions. A natural approach to computing the connected components on a large, dynamic graph stream is to buy enough RAM to store the entire graph. However, the requirement that the graph fit in RAM is prohibitive for very large graphs. Thus, there is an unmet need for systems that can process dense dynamic graphs, especially when those graphs are larger than available RAM. We present a new high-performance streaming graph-processing system for computing the connected components of a graph. This system, which we call GraphZeppelin, uses new linear sketching data structures (CubeSketch) to solve the streaming connected components problem and as a result requires space asymptotically smaller than the space required for a lossless representation of the graph. GraphZeppelin is optimized for massive dense graphs: GraphZeppelin can process millions of edge updates (both inser- tions and deletions) per second, even when the underlying graph is farmore »too large to fit in available RAM. As a result GraphZeppelin vastly increases the scale of graphs that can be processed.« less