An official website of the United States government
Social VR has increased in popularity due to its affordances for rich, embodied, and nonverbal communication. However, nonverbal communication remains inaccessible for blind and low vision people in social VR. We designed accessible cues with audio and haptics to represent three nonverbal behaviors: eye contact, head shaking, and head nodding. We evaluated these cues in real-time conversation tasks where 16 blind and low vision participants conversed with two other users in VR. We found that the cues were effective in supporting conversations in VR. Participants had statistically significantly higher scores for accuracy and confidence in detecting attention during conversations with the cues than without. We also found that participants had a range of preferences and uses for the cues, such as learning social norms. We present design implications for handling additional cues in the future, such as the challenges of incorporating AI. Through this work, we take a step towards making interpersonal embodied interactions in VR fully accessible for blind and low vision people.
more »
« less
Technology-Driven Alteration of Nonverbal Cues and its Effects on Negotiation
Abstract A person’s appearance, identity, and other nonverbal cues can substantially influence how one is perceived by a negotiation counterpart, potentially impacting the outcome of the negotiation. With recent advances in technology, it is now possible to alter such cues through real-time video communication. In many cases, a person’s physical presence can explicitly be replaced by 2D/3D representations in live interactive media. In other cases, technologies such as deepfake can subtly and implicitly alter many nonverbal cues—including a person’s appearance and identity—in real time. In this article, we look at some state-of-the-art technological advances that can enable such explicit and implicit alterations of nonverbal cues. We also discuss the implications of such technology for the negotiation landscape and highlight ethical considerations that warrant deep, ongoing attention from stakeholders.
more »
« less
- Award ID(s):
- 1750380
- PAR ID:
- 10636575
- Publisher / Repository:
- DOI PREFIX: 10.1162
- Date Published:
- Journal Name:
- Negotiation Journal
- Volume:
- 37
- Issue:
- 1
- ISSN:
- 0748-4526
- Format(s):
- Medium: X Size: p. 35-47
- Size(s):
- p. 35-47
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract This study focuses on the individual and joint contributions of two nonverbal channels (i.e., face and upper body) in avatar mediated-virtual environments. 140 dyads were randomly assigned to communicate with each other via platforms that differentially activated or deactivated facial and bodily nonverbal cues. The availability of facial expressions had a positive effect on interpersonal outcomes. More specifically, dyads that were able to see their partner’s facial movements mapped onto their avatars liked each other more, formed more accurate impressions about their partners, and described their interaction experiences more positively compared to those unable to see facial movements. However, the latter was only true when their partner’s bodily gestures were also available and not when only facial movements were available. Dyads showed greater nonverbal synchrony when they could see their partner’s bodily and facial movements. This study also employed machine learning to explore whether nonverbal cues could predict interpersonal attraction. These classifiers predicted high and low interpersonal attraction at an accuracy rate of 65%. These findings highlight the relative significance of facial cues compared to bodily cues on interpersonal outcomes in virtual environments and lend insight into the potential of automatically tracked nonverbal cues to predict interpersonal attitudes.more » « less
-
As deep-learning based image and video manipulation technology advances, the future of truth and information looks bleak. In particular, Deepfakes, wherein a person’s face can be transferred onto the face of someone else, pose a serious threat for potential spread of convincing misinformation that is drastic and ubiquitous enough to have catastrophic real-world consequences. To prevent this, an effective detection tool for manipulated media is needed. However, the detector cannot just be good, it has to evolve with the technology to keep pace with or even outpace the enemy. At the same time, it must defend against different attack types to which deep learning systems are vulnerable. To that end, in this paper, we review various methods of both attack and defense on AI systems, as well as modes of evolution for such a system. Then, we put forward a potential system that combines the latest technologies in multiple areas as well as several novel ideas to create a detection algorithm that is robust against many attacks and can learn over time with unprecedented effectiveness and efficiency.more » « less
-
null (Ed.)The paper reports ongoing research aimed at advancing knowledge on the role of expressive, charismatic gestures in education. More specifically, the objectives of the work described in the paper are to(a) measure expressive nonverbal cues from motion captured data and video images of successful instructors at Purdue University, (b) synthesize the identified nonverbal cues using 3D animation technology, and (c) study their effects on students. Current progress towards these goals and initial findings are discussed.more » « less
-
Abstract Innovations in artificial intelligence are enabling a new class of applications that can negotiate with people through chat or spoken language. Developed in close collaboration with behavioral science research, these algorithms can detect, mimic, and leverage human psychology, enabling them to undertake such functions as the detection of common mistakes made by novice negotiators. These algorithms can simulate the cognitive processes that shape human negotiations and make use of these models to influence negotiated outcomes. This article reviews some of the scientific advances enabling this technology and discusses how it is being used to advance negotiation research, teaching, and practice.more » « less
