skip to main content


Title: Interacting with Information in Immersive Virtual Environments
In this paper, we demonstrate the Information Interactions in Virtual Reality (IIVR) system designed and implemented to study how users interact with abstract information objects in immersive virtual environments in the context of information retrieval. Virtual reality displays are quickly growing as social and personal computing media, and understanding user interactions in these immersive environments is imperative. As a step towards effective information retrieval in such emerging platforms, our system is central to upcoming studies to observe how users engage in information triaging tasks in Virtual Reality (VR). In these studies, we will observe the effects of (1) information layouts and (2) types of interactions in VR. We believe this early system motivates researchers in understanding and designing meaningful interactions for future VR information retrieval applications.  more » « less
Award ID(s):
1840131
NSF-PAR ID:
10301959
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
Page Range / eLocation ID:
2600 to 2604
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Researchers in interactive information retrieval (IIR) have studied and refined 2D presentations of search results for years. Recent advances are bringing augmented reality (AR) and virtual reality (VR) to real-world systems, though the IIR community has done relatively little work to explore and understand aspects of 3D presentations of search results, effects of immersive environments, and the impacts of spatial cognition and different spatial arrangements of results displays in 3D. In the research proposed here, I outline my plan to use immerse environments to investigate how users’ spatial cognition may influence the information retrieval process. Specifically, this work will observe how spatial arrangements of search results affect users’ ability to find information in the postquery, visual search phase of the IIR process across quantitative and qualitative measures. 
    more » « less
  2. In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone. 
    more » « less
  3. Current VR/AR systems are unable to reproduce the physical sensation of fluid vessels, due to the shifting nature of fluid motion. To this end, we introduce SWISH, an ungrounded mixed-reality interface, capable of affording the users a realistic haptic sensation of fluid behaviors in vessels. The chief mechanism behind SWISH is in the use of virtual reality tracking and motor actuation to actively relocate the center of gravity of a handheld vessel, emulating the moving center of gravity of a handheld vessel that contains fluid. In addition to solving challenges related to reliable and efficient motor actuation, our SWISH designs place an emphasis on reproducibility, scalability, and availability to the maker culture. Our virtual-to-physical coupling uses Nvidia Flex's Unity integration for virtual fluid dynamics with a 3D printed augmented vessel containing a motorized mechanical actuation system. To evaluate the effectiveness and perceptual efficacy of SWISH, we conduct a user study with 24 participants, 7 vessel actions, and 2 virtual fluid viscosities in a virtual reality environment. In all cases, the users on average reported that the SWISH bucket generates accurate tactile sensations for the fluid behavior. This opens the potential for multi-modal interactions with programmable fluids in virtual environments for chemistry education, worker training, and immersive entertainment. 
    more » « less
  4. In this paper, we present results from an exploratory study to investigate users’ behaviors and preferences for three different styles of search results presentation in a virtual reality (VR) head-mounted display (HMD). Prior work in 2D displays has suggested possible benefits of presenting information in ways that exploit users’ spatial cognition abilities. We designed a VR system that displays search results in three different spatial arrangements: a list of 8 results, a 4x5 grid, and a 2x10 arc. These spatial display conditions were designed to differ in terms of the number of results displayed per page (8 vs 20) and the amount of head movement required to scan the results (list < grid < arc). Thirty-six participants completed 6 search trials in each display condition (18 total). For each trial, the participant was presented with a display of search results and asked to find a given target result or to indicate that the target was not present. We collected data about users’ behaviors with and perceptions about the three display conditions using interaction data, questionnaires, and interviews. We explore the effects of display condition and target presence on behavioral measures (e.g., completion time, head movement, paging events, accuracy) and on users’ perceptions (e.g., workload, ease of use, comfort, confidence, difficulty, and lostness). Our results suggest that there was no difference in accuracy among the display conditions, but that users completed tasks more quickly using the arc. However, users also expressed lower preferences for the arc, instead preferring the list and grid displays. Our findings extend prior research on visual search into to the area of 3-dimensional result displays for interactive information retrieval in VR HMD environments. 
    more » « less
  5. null (Ed.)
    Virtual Reality (VR) has become one of the emerging technologies over the past decade for improving the quality of life in human experiences. It has exciting and popular applications in entertainment, sports, education, and even digital documentation of notable or historical sites, allowing users to immerse themselves in an alternate reality. By combining the principles of software development and immersive VR, real-life VR experiences seek to transport users to an interactive environment where they can view, observe, and experience historical events and artifacts in a new way. There are several steps involved in VR development of cultural and historical sites that require a solid understanding for adaptable and scalable design. This paper is a review of the VR development process for notable historic preservation VR projects. This process can be used to create immersive VR experiences for other cultural sites. 
    more » « less