skip to main content


Title: Immersive Search: Using Virtual Reality to Examine How a Third Dimension Impacts the Searching Process
In this paper, we present results from an exploratory study to investigate users’ behaviors and preferences for three different styles of search results presentation in a virtual reality (VR) head-mounted display (HMD). Prior work in 2D displays has suggested possible benefits of presenting information in ways that exploit users’ spatial cognition abilities. We designed a VR system that displays search results in three different spatial arrangements: a list of 8 results, a 4x5 grid, and a 2x10 arc. These spatial display conditions were designed to differ in terms of the number of results displayed per page (8 vs 20) and the amount of head movement required to scan the results (list < grid < arc). Thirty-six participants completed 6 search trials in each display condition (18 total). For each trial, the participant was presented with a display of search results and asked to find a given target result or to indicate that the target was not present. We collected data about users’ behaviors with and perceptions about the three display conditions using interaction data, questionnaires, and interviews. We explore the effects of display condition and target presence on behavioral measures (e.g., completion time, head movement, paging events, accuracy) and on users’ perceptions (e.g., workload, ease of use, comfort, confidence, difficulty, and lostness). Our results suggest that there was no difference in accuracy among the display conditions, but that users completed tasks more quickly using the arc. However, users also expressed lower preferences for the arc, instead preferring the list and grid displays. Our findings extend prior research on visual search into to the area of 3-dimensional result displays for interactive information retrieval in VR HMD environments.  more » « less
Award ID(s):
1718295
NSF-PAR ID:
10188469
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
Page Range / eLocation ID:
1621 - 1624
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Objective Evaluate and model the advantage of a situation awareness (SA) supported by an augmented reality (AR) display for the ground-based joint terminal attack Controller (JTAC), in judging and describing the spatial relations between objects in a hostile zone. Background The accurate world-referenced description of relative locations of surface objects, when viewed from an oblique slant angle (aircraft, observation post) is hindered by (1) the compression of the visual scene, amplified at a lower slang angle, (2) the need for mental rotation, when viewed from a non-northerly orientation. Approach Participants viewed a virtual reality (VR)-simulated four-object scene from either of two slant angles, at each of four compass orientations, either unaided, or aided by an AR head-mounted display (AR-HMD), depicting the scene from a top-down (avoiding compression) and north-up (avoiding mental rotation) perspective. They described the geographical layout of four objects within the display. Results Compared with the control condition, that condition supported by the north-up SA display shortened the description time, particularly on non-northerly orientations (9 s, 30% benefit), and improved the accuracy of description, particularly for the more compressed scene (lower slant angle), as fit by a simple computational model. Conclusion The SA display provides large, significant benefits to this critical phase of ground-air communications in managing an attack—as predicted by the task analysis of the JTAC. Application Results impact the design of the AR-HMD to support combat ground-air communications and illustrate the magnitude by which basic cognitive principles “scale up” to realistically simulated real-world tasks such as search and rescue. 
    more » « less
  2. Virtual reality is a powerful tool for teaching 3D digital technologies in building engineering, as it facilitates the spatial perception of three-dimensional space. Spatial orientation skill is necessary for understanding 3D space. With VR, users navigate through virtually designed buildings and must be constantly aware of their position relative to other elements of the environment (orientation during navigation). In the present study, 25 building engineering students performed navigation tasks in a desktop-VR environment workshop. Performance of students using the desktop-VR was compared to a previous workshop in which navigation tasks were carried out using head-mounted displays. The Perspective Taking/Spatial Orientation Test measured spatial orientation skill. A questionnaire on user experience in the virtual environment was also administered. The gain in spatial orientation skill was 12.62%, similar to that obtained with head-mounted displays (14.23%). The desktop VR environment is an alternative to the HMD-VR environment for planning strategies to improve spatial orientation. Results from the user-experience questionnaire showed that the desktop VR environment strategy was well perceived by students in terms of interaction, 3D visualization, navigation, and sense of presence. Unlike in the HDM VR environment, student in the desktop VR environment did not report feelings of fatigue or dizziness. 
    more » « less
  3. null (Ed.)
    The number of people who own a virtual reality (VR) head-mounted display (HMD) has reached a point where researchers can readily recruit HMD owners to participate remotely using their own equipment. However, HMD owners recruited online may differ from the university community members who typically participate in VR research. HMD owners (n=220) and non-owners (n=282) were recruited through two online work sites-Amazon's Mechanical Turk and Prolific-and an undergraduate participant pool. Participants completed a survey in which they provided demographic information and completed measures of HMD use, video game use, spatial ability, and motion sickness susceptibility. In the context of the populations sampled, the results provide 1) a characterization of HMD owners, 2) a snapshot of the most commonly owned HMDs, 3) a comparison between HMD owners and non-owners, and 4) a comparison among online workers and undergraduates. Significant gender differences were found: men reported lower motion sickness susceptibility and more video game hours than women, and men outperformed women on spatial tasks. Men comprised a greater proportion of HMD owners than non-owners, but after accounting for this imbalance, HMD owners did not differ appreciably from non-owners. Comparing across recruitment platform, male undergraduates outperformed male online workers on spatial tests, and female undergraduates played fewer video game hours than female online workers. The data removal rate was higher from Amazon compared to Prolific, possibly reflecting greater dishonesty. These results provide a description of HMD users that can inform researchers recruiting remote participants through online work sites. These results also signal a need for caution when comparing in-person VR research that primarily enrolls undergraduates to online VR research that enrolls online workers. 
    more » « less
  4. Battery life is an increasingly urgent challenge for today's untethered VR and AR devices. However, the power efficiency of head-mounted displays is naturally at odds with growing computational requirements driven by better resolution, refresh rate, and dynamic ranges, all of which reduce the sustained usage time of untethered AR/VR devices. For instance, the Oculus Quest 2, under a fully-charged battery, can sustain only 2 to 3 hours of operation time. Prior display power reduction techniques mostly target smartphone displays. Directly applying smartphone display power reduction techniques, however, degrades the visual perception in AR/VR with noticeable artifacts. For instance, the "power-saving mode" on smartphones uniformly lowers the pixel luminance across the display and, as a result, presents an overall darkened visual perception to users if directly applied to VR content. Our key insight is that VR display power reduction must be cognizant of the gaze-contingent nature of high field-of-view VR displays. To that end, we present a gaze-contingent system that, without degrading luminance, minimizes the display power consumption while preserving high visual fidelity when users actively view immersive video sequences. This is enabled by constructing 1) a gaze-contingent color discrimination model through psychophysical studies, and 2) a display power model (with respect to pixel color) through real-device measurements. Critically, due to the careful design decisions made in constructing the two models, our algorithm is cast as a constrained optimization problem with a closed-form solution, which can be implemented as a real-time, image-space shader. We evaluate our system using a series of psychophysical studies and large-scale analyses on natural images. Experiment results show that our system reduces the display power by as much as 24% (14% on average) with little to no perceptual fidelity degradation. 
    more » « less
  5. Few VR applications and games implement captioning of speech and audio cues, which either inhibits or prevents access of their application by deaf or hard of hearing (DHH) users, new language learners, and other caption users. Additionally, little to no guidelines exist on how to implement live captioning on VR headsets and how it may differ from traditional television captioning. To help fill the void of information behind user preferences of different VR captioning styles, we conducted a study with eight DHH participants to test three caption movement behaviors (head-locked, lag, and appear- locked) while watching live-captioned, single-speaker presentations in VR. Participants answered a series of Likert scale and open-ended questions about their experience. Participants’ preferences were split, but most participants reported feeling comfortable with using live captions in VR and enjoyed the experience. When participants ranked the caption behaviors, there was almost an equal divide between the three types tested. IPQ results indicated each behavior had similar immersion ratings, however participants found head-locked and lag captions more user-friendly than appear-locked captions. We suggest that participants may vary in caption preference depending on how they use captions, and that providing opportunities for caption customization is best. 
    more » « less