skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: ASSIST: Personalized Indoor Navigation via Multimodal Sensors and High-Level Semantic Information
Blind & visually impaired (BVI) individuals and those with Autism Spectrum Disorder (ASD) each face unique challenges in navigating unfamiliar indoor environments. In this paper, we propose an indoor positioning and navigation system that guides a user from point A to point B indoors with high accuracy while augmenting their situational awareness. This system has three major components: location recognition (a hybrid indoor localization app that uses Bluetooth Low Energy beacons and Google Tango to provide high accuracy), object recognition (a body-mounted camera to provide the user momentary situational awareness of objects and people), and semantic recognition (map-based annotations to alert the user of static environmental characteristics). This system also features personalized interfaces built upon the unique experiences that both BVI and ASD individuals have in indoor wayfinding and tailors its multimodal feedback to their needs. Here, the technical approach and implementation of this system are discussed, and the results of human subject tests with both BVI and ASD individuals are presented. In addition, we discuss and show the system’s user-centric interface and present points for future work and expansion.  more » « less
Award ID(s):
1737533 1137172 1160046
PAR ID:
10110459
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Leal-Taixé L., Roth S. (eds) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science
Volume:
11134
Page Range / eLocation ID:
128-143
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Assistive Technology)
    null (Ed.)
    This paper describes the interface and testing of an indoor navigation app - ASSIST - that guides blind & visually impaired (BVI) individuals through an indoor environment with high accuracy while augmenting their understanding of the surrounding environment. ASSIST features personalized interfaces by considering the unique experiences that BVI individuals have in indoor wayfinding and offers multiple levels of multimodal feedback. After an overview of the technical approach and implementation of the first prototype of the ASSIST system, the results of two pilot studies performed with BVI individuals are presented – a performance study to collect data on mobility (walking speed, collisions, and navigation errors) while using the app, and a usability study to collect user evaluation data on the perceived helpfulness, safety, ease-of-use, and overall experience while using the app. Our studies show that ASSIST is useful in providing users with navigational guidance, improving their efficiency and (more significantly) their safety and accuracy in wayfinding indoors. Findings and user feed-back from the studies confirm some of the previous results, while also providing some new insights into the creation of such an app, including the use of customized user interfaces and expanding the types of information provided. 
    more » « less
  2. ; ; ; (, IJCAI AI for Social Good Workshop)
    This paper describes the interface and testing of an indoor navigation app - ASSIST - that guides blind & visually impaired (BVI) individuals through an indoor environment with high accuracy while augmenting their understanding of the surrounding environment. ASSIST features personalized inter-faces by considering the unique experiences that BVI individuals have in indoor wayfinding and offers multiple levels of multimodal feedback. After an overview of the technical approach and implementation of the first prototype of the ASSIST system, the results of two pilot studies performed with BVI individuals are presented. Our studies show that ASSIST is useful in providing users with navigational guidance, improving their efficiency and (more significantly) their safety and accuracy in wayfinding indoors. 
    more » « less
  3. ; ; (, The 21st International ACM SIGACCESS Conference on Computers and Accessibility)
    GPS accuracy is poor in indoor environments and around buildings. Thus, reading and following signs still remains the most common mechanism for providing and receiving wayfinding information in such spaces. This puts individuals who are blind or visually impaired (BVI) at a great disadvantage. This work designs, implements, and evaluates a wayfinding system and smartphone application called CityGuide that can be used by BVI individuals to navigate their surroundings beyond what is possible with just a GPS-based system. CityGuide enables an individual to query and get turn-by-turn shortest route directions from an indoor location to an outdoor location. CityGuide leverages recently developed indoor wayfinding solutions in conjunction with GPS signals to provide a seamless indoor-outdoor navigation and wayfinding system that guides a BVI individual to their desired destination through the shortest route. Evaluations of CityGuide with BVI human subjects navigating between an indoor starting point to an outdoor destination within an unfamiliar university campus scenario showed it to be effective in reducing end-to-end navigation times and distances of almost all participants. 
    more » « less
  4. ; ; ; (, IEEE)
    Indoor navigation in complex building environments poses significant challenges, particularly for individuals who are unfamiliar with their surroundings. Mixed reality (MR) technologies have emerged as a promising solution to enhance situational awareness and facilitate navigation within indoor spaces. However, there is a lack of spatial data for indoor environments, including outdated floor plans and limited real-time operational data. This paper presents the development of a mixed-reality application for indoor building navigation and evacuation. The application uses feature extraction for location sensing and situational awareness to provide accurate and reliable navigation in any indoor environment using Microsoft HoloLens. The application can track the user's position and orientation and give the user-specific information on how to evacuate the building. This information is then used to generate navigation instructions for the user. We demonstrate how this mixed reality HoloLens application can provide spatially contextualized 3D visualizations that promote spatial knowledge acquisition and situational awareness. These 3D visualizations are developed as an emergency evacuation and navigation tool to aid the building occupants in safe and quick evacuation. Experimental results demonstrate the effectiveness of the application, providing 3D visualizations of multilevel spaces and aiding individuals in understanding their position and evacuation path during emergencies. We believe that adopting mixed reality technologies, such as the HoloLens, can greatly enhance individuals' ability to navigate large-scale environments during emergencies by promoting spatial knowledge acquisition and supporting cognitive mapping. 
    more » « less
  5. ; ; (, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW))
    Emergency response in indoor building evacuation is essential for effective rescue and safety management. First responders often lack the situational awareness capability to quickly assess the layout of a building upon initial entry. For occupants of the building, situational awareness becomes more important in cases of active shooter events or circumstances of fire and smoke. One of the challenges is to provide user-specific personalized evacuation routes in real-time. In multilevel building environments, the complexity of the architecture creates problems for both visual and mental representation of the 3D spaces. This paper presents three cutting edge Augmented Reality Instructional (ARI) modules that overcome the visual limitations associated with the traditional, static 2D methods of communicating evacuation plans for multilevel buildings. Using existing building features, the authors demonstrate how the three modules provide contextualized 3D visualizations that promote and support spatial knowledge acquisition and cognitive mapping thereby enhancing situational awareness. These ARI visualizations are developed for first responders and building occupants to help increase emergency preparedness and mitigate the evacuation related risks in multilevel building rescues and safety management. Specifically, the paper describes the design and implementation of the ARI modules and reports the results of the pilot studies conducted to evaluate their perceived usefulness, ease-of-use, and usability. The results suggest the desirability of further heuristic examination of three-dimensional situational awareness-based ARI application effectiveness in multilevel building evacuations. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email