skip to main content


Title: Developing Future Wearable Interfaces for Human-Drone Teams through a Virtual Drone Search Game
Autonomous robotic vehicles (i.e., drones) are potentially transformative for search and rescue (SAR). This paper works toward wearable interfaces, through which humans team with multiple drones. We introduce the Virtual Drone Search Game as a first step in creating a mixed reality simulation for humans to practice drone teaming and SAR techniques. Our goals are to (1) evaluate input modalities for the drones, derived from an iterative narrowing of the design space, (2) improve our mixed reality system for designing input modalities and training operators, and (3) collect data on how participants socially experience the virtual drones with which they work. In our study, 17 participants played the game with two input modalities (Gesture condition, Tap condition) in counterbalanced order. Results indicated that participants performed best with the Gesture condition. Participants found the multiple controls challenging, and future studies might include more training of the devices and game. Participants felt like a team with the drones and found them moderately agentic. In our future work, we will extend this testing to a more externally valid mixed reality game.  more » « less
Award ID(s):
1651532
NSF-PAR ID:
10276923
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Date Published:
Journal Name:
International journal of humancomputer studies
Volume:
147
Issue:
102573
ISSN:
1071-5819
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The proliferation of unmanned aerial systems (i.e., drones) can provide great value to the future of search and rescue. However, with the increase adoption of such systems, issues around hybrid human-drone team coordination and planning will arise. To address these early challenges, we provide insights into the development of testbeds in the form of mixed reality games with simulated drones. This research presents an architecture to address challenges and opportunities in using drones for search and rescue. On this architecture, we develop a mixed reality game in which human players engage with the physical world and with gameplay that is purely virtual. We expect the architecture to be useful to a range of researchers an practitioners, forming the basis for investigating and training within this unique, new domain. 
    more » « less
  2. Integrating drones into construction sites can introduce new risks to workers who already work in hazardous environments. Consequently, several recent studies have investigated the safety challenges and solutions associated with this technology integration in construction. However, there is a knowledge gap about effectively communicating such safety challenges to construction professionals and students who may work alongside drones on job sites. In this study, a 360-degree virtual reality (VR) environment was created as a training platform to communicate the safety challenges of worker-drone interactions on construction jobsites. This pilot study assesses the learning effectiveness and user experience of the developed 360 VR worker-drone safety training, which provides an immersive device-agnostic learning experience. The result indicates that such 360 VR learning material could significantly increase the safety knowledge of users while delivering an acceptable user experience in most of its assessment criteria. The outcomes of this study will serve as a valuable resource for improving future worker-drone safety training materials. 
    more » « less
  3. Search and rescue (SAR) operations are often nearly computer-technology-free due to the fragility and connectivity needs of current information communication technology (ICT). In this design fiction, we envision a world where SAR uses augmented reality (AR) and the surplus labor of volunteers during crisis response efforts. Unmanned aerial vehicles, crowdsourced mapping platforms, and concepts from video game mapping technologies can all be mixed to keep SAR operations complexity-free while incorporating ICTs. Our scenario describes a near-future SAR operation with currently available technology being assembled and deployed without issue. After our scenario, we discuss socio-technical barriers for technology use like technical fragility and overwhelming complexity. We also discuss how to work around those barriers and how to use video games as a testbed for SAR technology. We hope to inspire more resilient ICT design that is accessible without training. 
    more » « less
  4. Drones have become fixtures in commerce, safety efforts, and in homes as a leisure activity. Researchers have started to explore how drones can support people with disabilities in piloting and serve as assistive devices. Our work focuses on people with vision impairment and investigates what motivates them to fly drones. We administered a survey to visually impaired adults that gauged general interest in drone piloting and previous experience with drones. From the 59 survey responses, we interviewed 13 participants to elaborate on how they envision using drones and how different feedback and modes of piloting can make the flying experience more accessible. We found that our participants had overarching interests in aviation, trying new technology, environment exploration, and finding collaborative activities to do with their sighted family members, which extended to an interest in piloting drones. This research helps lay groundwork for design scenarios and accessible features for future drones. 
    more » « less
  5. It is now possible to deploy swarms of drones with populations in the thousands. There is growing interest in using such swarms for defense, and it has been natural to program them with bio-mimetic motion models such as flocking or swarming. However, these motion models evolved to survive against predators, not enemies with modern firearms. This paper presents experimental data that compares the survivability of several motion models for large numbers of drones. This project tests drone swarms in Virtual Reality (VR), because it is prohibitively expensive, technically complex, and potentially dangerous to fly a large swarm of drones in a testing environment. We model the behavior of drone swarms flying along parametric paths in both tight and scattered formations. We add random motion to the general motion plan to confound path prediction and targeting. We describe an implementation of these flight paths as game levels in a VR environment. We then allow players to shoot at the drones and evaluate the difference between flocking and swarming behavior on drone survivability. 
    more » « less