Search for: All records

Recent advances in virtual reality cameras have contributed to a phenomenal growth of 360∘∘ videos. Estimating regions likely to attract user attention is critical for efficiently streaming and rendering 360∘∘ videos. In this article, we present a simple, novel, GPU-driven pipeline for saliency computation and virtual cinematography in 360∘∘ videos using spherical harmonics (SH). We efficiently compute the 360∘∘ video saliency through the spectral residual of the SH coefficients between multiple bands at over 60FPS for 4K resolution videos. Further, our interactive computation of spherical saliency can be used for saliency-guided virtual cinematography in 360∘∘ videos. 

Current techniques for characterizing cybersickness (visually induced motion sickness) in virtual environments rely on qualitative questionnaires. For interactive graphics to create visual experiences that enhance the illusion of presence while mitigating cybersickness, interactive measures are needed to characterize cybersickness. In this paper, we acquire EEG signals from participants as they experience vection-induced cybersickness and compare those signals to a baseline. Our study shows that there is a correlation between the participant-reported cybersickness (as measured by movements of a joystick) and brain EEG signals. Through independent component analysis, we separate those signals which are a result of cybersickness from other sources (such as eye blinks). Our user study finds that there is a highly correlative and statistically significant Delta- (1.0–4.0 Hz), Theta- (4.0–7.0 Hz), and Alpha-wave (7.0–13.0 Hz) increase associated with cybersickness in immersive virtual environments across participants. Establishing a strong correlation between cybersickness and EEG-measured brain activity provides us with the first step toward interactively characterizing and mitigating cybersickness in virtual environments. 

Holographic displays and computer-generated holography offer a unique opportunity in improving optical resolutions and depth characteristics of near-eye displays. The thermally-modulated Nanopho-tonic Phased Array (NPA), a new type of holographic display, affords several advantages, including integrated light source and higher refresh rates, over other holographic display technologies. However, the thermal phase modulation of the NPA makes it susceptible to the thermal proximity effect where heating one pixel affects the temperature of nearby pixels. Proximity effect correction (PEC) methods have been proposed for 2D Fourier holograms in the far field but not for Fresnel holograms at user-specified depths. Here we extend an existing PEC method for the NPA to Fresnel holograms with phase-only hologram optimization and validate it through computational simulations. Our method is not only effective in correcting the proximity effect for the Fresnel holograms of 2D images at desired depths but can also leverage the fast refresh rate of the NPA to display 3D scenes with time-division multiplexing. 

While 360° images are becoming ubiquitous due to popularity of panoramic content, they cannot directly work with most of the existing depth estimation techniques developed for perspective images. In this paper, we present a deep-learning-based framework of estimating depth from 360° images. We present an adaptive depth refinement procedure that refines depth estimates using normal estimates and pixel-wise uncertainty scores. We introduce double quaternion approximation to combine the loss of the joint estimation of depth and surface normal. Furthermore, we use the double quaternion formulation to also measure stereo consistency between the horizontally displaced depth maps, leading to a new loss function for training a depth estimation CNN. Results show that the new double-quaternion-based loss and the adaptive depth refinement procedure lead to better network performance. Our proposed method can be used with monocular as well as stereo images. When evaluated on several datasets, our method surpasses state-of-the-art methods on most metrics.