More Like this

1. Robot-generated Mixed Reality Gestures Improve Human-Robot Interaction

   https://doi.org/10.1007/978-3-030-90525-5_69  

   Tran, Nhan ; Grant, Trevor ; Phung, Thao ; Hirshfield, Leanne ; Wickens, Christopher ; Williams, Tom ( January 2022 , International Conference on Social Robotics (Brief Research Reports))

   We investigate the effectiveness of robot-generated mixed reality gestures. Our findings demonstrate how these gestures increase user effectiveness by decreasing user response time, and that robots can pair long referring expressions with mixed reality gestures without cognitively overloading users. 

   more » « less
2. Mixed Reality Deictic Gesture for Multi-Modal Robot Communication

   https://doi.org/10.1109/HRI.2019.8673275  

   Williams, Tom ; Bussing, Matthew ; Cabrol, Sebastian ; Boyle, Elizabeth ; Tran, Nhan ( March 2019 , 14th {ACM/IEEE} International Conference on Human-Robot Interaction)

   In previous work, researchers have repeatedly demonstrated that robots' use of deictic gestures enables effective and natural human-robot interaction. However, new technologies such as augmented reality head mounted displays enable environments in which mixed-reality becomes possible, and in such environments, physical gestures become but one category among many different types of mixed reality deictic gestures. In this paper, we present the first experimental exploration of the effectiveness of mixed reality deictic gestures beyond physical gestures. Specifically, we investigate human perception of videos simulating the display of allocentric gestures, in which robots circle their targets in users' fields of view. Our results suggest that this is an effective communication strategy, both in terms of objective accuracy and subjective perception, especially when paired with complex natural language references. 

   more » « less
3. What's The Point?: Tradeoffs Between Effectiveness and Social Perception When Using Mixed Reality to Enhance Gesturally Limited Robots

   https://doi.org/10.1145/3434073.3444676  

   Hamilton, Jared ; Phung, Thao ; Tran, Nhan ; Williams, Tom ( March 2021 , ACM/IEEE International Conference on Human-Robot Interaction)

   null (Ed.)

   Mixed Reality visualizations provide a powerful new approach for enabling gestural capabilities on non-humanoid robots. This paper explores two different categories of mixed-reality deictic gestures for armless robots: a virtual arrow positioned over a target referent (a non-ego-sensitive allocentric gesture) and a virtual arm positioned over the gesturing robot (an ego-sensitive allocentric gesture). Specifically, we present the results of a within-subjects Mixed Reality HRI experiment (N=23) exploring the trade-offs between these two types of gestures with respect to both objective performance and subjective social perceptions. Our results show a clear trade-off between performance and social perception, with non-ego-sensitive allocentric gestures enabling faster reaction time and higher accuracy, but ego-sensitive gestures enabling higher perceived social presence, anthropomorphism, and likability. 

   more » « less
4. Tradeoffs Between Effectiveness and Social Perception When Using Mixed Reality to Supplement Gesturally Limited Robots

   https://doi.org/10.31219/osf.io/ye2w5  

   Hamilton, Jared ; Tran, Nhan ; Williams, Tom ( January 2020 , International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction)

   Mixed reality visualizations provide a powerful new approach for enabling gestural capabilities for non-humanoid robots. This paper explores two different categories of mixed-reality deictic gestures for armless robots: a virtual arrow positioned over a target referent (a non-ego-sensitive allocentric gesture) and a virtual arrow positioned over the robot (an ego-sensitive allocentric gesture). We explore the trade-offs between these two types of gestures, with respect to both objective performance and subjective social perceptions. We conducted a 24-participant within-subjects experiment in which a HoloLens-wearing participant interacted with a robot that used these two types of gestures to refer to objects at two different distances. Our results demonstrate a clear trade-off between performance and social perception: non-ego-sensitive allocentric gestures led to quicker reaction time and higher accuracy, but ego-sensitive gesture led to higher perceived social presence, anthropomorphism, and likability. These results present a challenging design decision to creators of mixed reality robotic systems 

   more » « less
5. Crossing Reality: Comparing Physical and Virtual Robot Deixis

   https://doi.org/10.1145/3568162.3576972  

   Han, Zhao ; Zhu, Yifei ; Phan, Albert ; Garza, Fernando Sandoval ; Castro, Amia ; Williams, Tom ( March 2023 , ACM/IEEE International Conference on Human-Robot Interaction)

   Augmented Reality (AR) technologies present an exciting new medium for human-robot interactions, enabling new opportunities for both implicit and explicit human-robot communication. For example, these technologies enable physically-limited robots to execute non-verbal interaction patterns such as deictic gestures despite lacking the physical morphology necessary to do so. However, a wealth of HRI research has demonstrated real benefits to physical embodiment (compared to, e.g., virtual robots on screens), suggesting AR augmentation of virtual robot parts could face challenges.In this work, we present empirical evidence comparing the use of virtual (AR) and physical arms to perform deictic gestures that identify virtual or physical referents. Our subjective and objective results demonstrate the success of mixed reality deictic gestures in overcoming these potential limitations, and their successful use regardless of differences in physicality between gesture and referent. These results help to motivate the further deployment of mixed reality robotic systems and provide nuanced insight into the role of mixed-reality technologies in HRI contexts. 

   more » « less