skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Linking global top-down views to first-person views in the brain
Humans and other animals have a remarkable capacity to translate their position from one spatial frame of reference to another. The ability to seamlessly move between top-down and first-person views is important for navigation, memory formation, and other cognitive tasks. Evidence suggests that the medial temporal lobe and other cortical regions contribute to this function. To understand how a neural system might carry out these computations, we used variational autoencoders (VAEs) to reconstruct the first-person view from the top-down view of a robot simulation, and vice versa. Many latent variables in the VAEs had similar responses to those seen in neuron recordings, including location-specific activity, head direction tuning, and encoding of distance to local objects. Place-specific responses were prominent when reconstructing a first-person view from a top-down view, but head direction–specific responses were prominent when reconstructing a top-down view from a first-person view. In both cases, the model could recover from perturbations without retraining, but rather through remapping. These results could advance our understanding of how brain regions support viewpoint linkages and transformations.  more » « less
Award ID(s):
2024633
PAR ID:
10465604
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Proceedings of the National Academy of Sciences
Volume:
119
Issue:
45
ISSN:
0027-8424
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Many attachments to a scanning electron microscope (SEM), such as energy dispersive x‐ray spectroscopy, extend its function significantly. Typically, the application of such attachments requires that the specimen has a planar surface at a specific orientation. It is a challenge to make the plane of a microscale specimen satisfy the orientation requirement since they are visible only in an SEM. An in‐situ procedure is needed to adjust specimen orientation by using stage rotation and tilting functions, in the process of which the key is to determine the initial orientation. This study proposed and tested top‐down and side‐view approaches to determine the orientation of a planar surface inside an SEM. In the top‐down one, the projected area is monitored on SEM images as stage rotation and tilt angles are adjusted. When the surface normal is along the electron beam direction, the area has a maximum value. In the side‐view approach, the stage is adjusted so that the projection appears to be a straight horizontal line on the SEM image. Once the orientation of the specimen for top‐down or side‐view observation is determined, the original can be calculated, and a desired orientation can be realized by manipulating the stage. The procedures have been tested by analyzing planar surfaces of spherical particles in Al‐Cu‐Fe alloy in the form of facets. The measured angles between two surfaces are consistent with those expected from crystallographic consideration within 2.7° and 1.7° for the top‐down and side‐view approaches, respectively. Research HighlightsTop‐down and side‐view approaches have been proposed and tested for in‐situ determination of specimen planar surface orientation in a Scanning Electron Microscope.The measured angles between two surfaces are consistent with those expected from crystallographic consideration within 2.7° and 1.7° for the top‐down and side‐view approaches, respectively. 
    more » « less
  2. null (Ed.)
    Third-person is a popular perspective for video games, but virtual reality (VR) seems to be primarily experienced from a first-person point of view (POV). While a first-person POV generally offers the highest presence; a third-person POV allows users to see their avatar; which allows for a better bond, and the higher vantage point generally increases spatial awareness and navigation. Third-person locomotion is generally implemented using a controller or keyboard, with users often sitting down; an approach that is considered to offer a low presence and embodiment. We present a novel thirdperson locomotion method that enables a high avatar embodiment by integrating skeletal tracking with head-tilt based input to enable omnidirectional navigation beyond the confines of available tracking space. By interpreting movement relative to an avatar, the user will always keep facing the camera which optimizes skeletal tracking and keeps required instrumentation minimal (1 depth camera). A user study compares the performance, usability, VR sickness incidence and avatar embodiment of our method to using a controller for a navigation task that involves interacting with objects. Though a controller offers a higher performance and usability, our locomotion method offered a significantly higher avatar embodiment. 
    more » « less
  3. Researchers have adopted remote methods, such as online surveys and video conferencing, to overcome challenges in conducting in-person usability testing, such as participation, user representation, and safety. However, remote user evaluation on hardware testbeds is limited, especially for blind participants, as such methods restrict access to observations of user interactions. We employ smart glasses in usability testing with blind people and share our lessons from a case study conducted in blind participants’ homes (N=12), where the experimenter can access participants’ activities via dual video conferencing: a third-person view via a laptop camera and a first-person view via smart glasses worn by the participant. We show that smart glasses hold potential for observing participants’ interactions with smartphone testbeds remotely; on average 58.7% of the interactions were fully captured via the first-person view compared to 3.7% via the third-person. However, this gain is not uniform across participants as it is susceptible to head movements orienting the ear towards a sound source, which highlights the need for a more inclusive camera form factor. We also share our lessons learned when it comes to dealing with lack of screen readers, a rapidly draining battery, and Internet connectivity in remote studies with blind participants. 
    more » « less
  4. Abstract A common approach to interpreting spiking activity is based on identifying the firing fields—regions in physical or configuration spaces that elicit responses of neurons. Common examples include hippocampal place cells that fire at preferred locations in the navigated environment, head direction cells that fire at preferred orientations of the animal’s head, view cells that respond to preferred spots in the visual field, etc. In all these cases, firing fields were discovered empirically, by trial and error. We argue that the existence and a number of properties of the firing fields can be established theoretically, through topological analyses of the neuronal spiking activity. In particular, we use Leray criterion powered by persistent homology theory, Eckhoff conditions and Region Connection Calculus to verify consistency of neuronal responses with a single coherent representation of space. 
    more » « less
  5. Full windshield displays (WSDs) have the potential to present imagery across the windshield. Current knowledge on display location has not investigated translucent displays at high eccentricities from the driver's forward view. A simulator study (n=26) was conducted aiming to, (a) investigate the effects of Head-Up Display (HUD) location across the entire windshield on driving performance, and (b) better understand how the visual demand for a complex HUD imagery differs from that for a Head-Down Display (HDD). Lane-keeping was poorer when HUD imagery was furthest from the driver (and for the HDD compared to the HUD). Equally, counts of "unacceptable" driving behaviour were greater for displays furthest from the driver's forward view. Furthermore, drivers preferred HUD imagery that was closer to them. The results indicate that HUD evaluations should account for image location, because of how driver gaze location can impact lateral driving performance. 
    more » « less