Owing to the increasing dynamics and complexity of construction tasks, workers often need to memorize a big amount of engineering information prior to the operations, such as spatial orientations and operational procedures. The working memory development, as a result, is critical to the performance and safety of many construction tasks. This study investigates how the format of engineering information affects human working memory based on a human-subject Virtual Reality (VR) experiment (n=90). A VR model was created to simulate a pipe maintenance task. First, participants were asked to review the task procedures in one of the following formats, including 2D isometric drawings, 3D model, and VR model. After the review session, participants were asked to perform the pipe maintenance task in the virtual environment based on their working memory. The operation accuracy and time were used as the key performance indicators of the working memory development. The experiment results indicate that the 3D and VR groups outperformed the 2D group in both operation accuracy and time, suggesting that a more immersive instruction leads to a better working memory. A further examination finds that the 2D group presented a significantly higher level of intrinsic cognitive load and extraneous cognitive load in the working memory development compared to the 3D and VR groups, indicating that different engineering information formats can cause different levels of cognitive load in working memory development, and ultimately affect the final performance. The findings are expected to inspire the design of intelligent information systems that adapt to the cognitive load of construction workers for improved working memory development.
more »
« less
Encoding Height: Egocentric Spatial Memory of Adults and Teens in a Virtual Stairwell
Commodity-level virtual reality equipment is now available to all ages. To better understand how cognitive development affects people's spatial memory in virtual reality, we assess how adults (20-29 years old) and teenagers (14-17 years old) represent their spatial memory of objects in an immersive virtual environment (IVE) where height is encoded. Despite virtual reality being a favorable conduit for the study of egocentric spatial memory, prior studies have predominately looked at objects placed at similar heights. Within a stairwell environment, participants learned the positions of nine target objects. In one condition, all objects were placed near eye height. In another, they were placed at varying heights. Our results indicate that participants' errors and latencies were similar in both environments, and across age groups. Our results have implications for the development of IVEs and the expansion of immersive technology to a more diverse, younger audience.
more »
« less
- Award ID(s):
- 1763966
- NSF-PAR ID:
- 10290443
- Date Published:
- Journal Name:
- ACM Symposium on Applied Perception 2020 (SAP '20), September 17--18, 2020, Virtual Event, USA
- Page Range / eLocation ID:
- 1 to 8
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Place and grid cells in the hippocampal formation are commonly thought to support a unified and coherent cognitive map of space. This mapping mechanism faces a challenge when a navigator is placed in a familiar environment that has been deformed from its original shape. Under such circumstances, many transformations could plausibly serve to map a navigator's familiar cognitive map to the deformed space. Previous empirical results indicate that the firing fields of rodent place and grid cells stretch or compress in a manner that approximately matches the environmental deformation, and human spatial memory exhibits similar distortions. These effects have been interpreted as evidence that reshaping a familiar environment elicits an analogously reshaped cognitive map. However, recent work has suggested an alternative explanation, whereby deformation‐induced distortions of the grid code are attributable to a mechanism that dynamically anchors grid fields to the most recently experienced boundary, thus causing history‐dependent shifts in grid phase. This interpretation raises the possibility that human spatial memory will exhibit similar history‐dependent dynamics. To test this prediction, we taught participants the locations of objects in a virtual environment and then probed their memory for these locations in deformed versions of this environment. Across three experiments with variable access to visual and vestibular cues, we observed the predicted pattern, whereby the remembered locations of objects were shifted from trial to trial depending on the boundary of origin of the participant's movement trajectory. These results provide evidence for a dynamic anchoring mechanism that governs both neuronal firing and spatial memory.
-
Scene Perception and Visuospatial Memory Converge at the Anterior Edge of Visually Responsive Cortexmore » « less
To fluidly engage with the world, our brains must simultaneously represent both the scene in front of us and our memory of the immediate surrounding environment (i.e., local visuospatial context). How does the brain's functional architecture enable sensory and mnemonic representations to closely interface while also avoiding sensory-mnemonic interference? Here, we asked this question using first-person, head-mounted virtual reality and fMRI. Using virtual reality, human participants of both sexes learned a set of immersive, real-world visuospatial environments in which we systematically manipulated the extent of visuospatial context associated with a scene image in memory across three learning conditions, spanning from a single FOV to a city street. We used individualized, within-subject fMRI to determine which brain areas support memory of the visuospatial context associated with a scene during recall (Experiment 1) and recognition (Experiment 2). Across the whole brain, activity in three patches of cortex was modulated by the amount of known visuospatial context, each located immediately anterior to one of the three scene perception areas of high-level visual cortex. Individual subject analyses revealed that these anterior patches corresponded to three functionally defined place memory areas, which selectively respond when visually recalling personally familiar places. In addition to showing activity levels that were modulated by the amount of visuospatial context, multivariate analyses showed that these anterior areas represented the identity of the specific environment being recalled. Together, these results suggest a convergence zone for scene perception and memory of the local visuospatial context at the anterior edge of high-level visual cortex.
SIGNIFICANCE STATEMENT As we move through the world, the visual scene around us is integrated with our memory of the wider visuospatial context. Here, we sought to understand how the functional architecture of the brain enables coexisting representations of the current visual scene and memory of the surrounding environment. Using a combination of immersive virtual reality and fMRI, we show that memory of visuospatial context outside the current FOV is represented in a distinct set of brain areas immediately anterior and adjacent to the perceptually oriented scene-selective areas of high-level visual cortex. This functional architecture would allow efficient interaction between immediately adjacent mnemonic and perceptual areas while also minimizing interference between mnemonic and perceptual representations. -
A primary goal of the Virtual Reality ( VR ) community is to build fully immersive and presence-inducing environments with seamless and natural interactions. To reach this goal, researchers are investigating how to best directly use our hands to interact with a virtual environment using hand tracking. Most studies in this field require participants to perform repetitive tasks. In this article, we investigate if results of such studies translate into a real application and game-like experience. We designed a virtual escape room in which participants interact with various objects to gather clues and complete puzzles. In a between-subjects study, we examine the effects of two input modalities (controllers vs. hand tracking) and two grasping visualizations (continuously tracked hands vs. virtual hands that disappear when grasping) on ownership, realism, efficiency, enjoyment, and presence. Our results show that ownership, realism, enjoyment, and presence increased when using hand tracking compared to controllers. Visualizing the tracked hands during grasps leads to higher ratings in one of our ownership questions and one of our enjoyment questions compared to having the virtual hands disappear during grasps as is common in many applications. We also confirm some of the main results of two studies that have a repetitive design in a more realistic gaming scenario that might be closer to a typical user experience.more » « less
-
more » « less
Abstract People use environmental knowledge to maintain a sense of direction in daily life. This knowledge is typically measured by having people point to unseen locations (judgments of relative direction) or navigate efficiently in the environment (shortcutting). Some people can estimate directions precisely, while others point randomly. Similarly, some people take shortcuts not experienced during learning, while others mainly follow learned paths. Notably, few studies have directly tested the correlation between pointing and shortcutting performance. We compared pointing and shortcutting in two experiments, one using desktop virtual reality (VR) (N = 57) and one using immersive VR (N = 48). Participants learned a new environment by following a fixed route and were then asked to point to unseen locations and navigate to targets by the shortest path. Participants’ performance was clustered into two groups using K-means clustering. One (lower ability) group pointed randomly and showed low internal consistency across trials in pointing, but were able to find efficient routes, and their pointing and efficiency scores were not correlated. The others (higher ability) pointed precisely, navigated by efficient routes, and their pointing and efficiency scores were correlated. These results suggest that with the same egocentric learning experience, the correlation between pointing and shortcutting depends on participants’ learning ability, and internal consistency and discriminating power of the measures. Inconsistency and limited discriminating power can lead to low correlations and mask factors driving human variation. Psychometric properties, largely under-reported in spatial cognition, can advance our understanding of individual differences and cognitive processes for complex spatial tasks.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3385955.3407938
