Search for: All records

Immersive virtual reality presents a rich opportunity for learning signed languages, given the immersive environment’s ability to represent three-dimensional information. We developed a proof- of-concept American Sign Language (ASL) learning in immersive virtual reality (VR), named ASL Champ! Twelve hearing non- or novice signers played one full level of the game, during which they were asked to provide concurrent think-aloud (CTA) commentary, narrating their experience as they played in real time. We conducted a sentiment analysis from recordings of the CTA and subsequent open-ended questions and qualitatively assessed the narrations for salient themes. The analysis revealed speci!c aspects of the users’ experiences that were most likely to lead to positive or negative expressions during the CTA and the question session. The factors that had the most impact on user sentiment were the success of the sign recognition in the game and the extent to which users found the game intuitive or self-explanatory. We also found that users with more technology anxiety were more positive about the game. We also qualitatively examined user comments, revealing their real-time game experiences. This work provides insights into which aspects of an ASL learning VR game are most important for user experiences. We conclude with takeaway recommendations for future virtual or augmented reality sign language learning games. 

With improved and more easily accessible technology, immersive virtual reality (VR) head-mounted devices have become more ubiquitous. As signing avatar technology improves, virtual reality presents a new and relatively unexplored application for signing avatars. This paper discusses two primary ways that signed language can be represented in immersive virtual spaces: 1) Third-person, in which the VR user sees a character who communicates in signed language; and 2) First-person, in which the VR user produces signed content themselves, tracked by the head-mounted device and visible to the user herself (and/or to other users) in the virtual environment. We will discuss the unique affordances granted by virtual reality and how signing avatars might bring accessibility and new opportunities to virtual spaces. We will then discuss the limitations of signed content in virtual reality concerning virtual signers shown from both third- and first-person perspectives. 

The use of virtual humans (i.e., avatars) holds the potential for interactive, automated interaction in domains such as remote communication, customer service, or public announcements. For signed language users, signing avatars could potentially provide accessible content by sharing information in the signer's preferred or native language. As the development of signing avatars has gained traction in recent years, researchers have come up with many different methods of creating signing avatars. The resulting avatars vary widely in their appearance, the naturalness of their movements, and facial expressions—all of which may potentially impact users' acceptance of the avatars. We designed a study to test the effects of these intrinsic properties of different signing avatars while also examining the extent to which people's own language experiences change their responses to signing avatars. We created video stimuli showing individual signs produced by (1) a live human signer (Human), (2) an avatar made using computer-synthesized animation (CS Avatar), and (3) an avatar made using high-fidelity motion capture (Mocap avatar). We surveyed 191 American Sign Language users, including Deaf ( N = 83), Hard-of-Hearing ( N = 34), and Hearing ( N = 67) groups. Participants rated the three signers on multiple dimensions, which were then combined to form ratings of Attitudes, Impressions, Comprehension, and Naturalness. Analyses demonstrated that the Mocap avatar was rated significantly more positively than the CS avatar on all primary variables. Correlations revealed that signers who acquire sign language later in life are more accepting of and likely to have positive impressions of signing avatars. Finally, those who learned ASL earlier were more likely to give lower, more negative ratings to the CS avatar, but we did not see this association for the Mocap avatar or the Human signer. Together, these findings suggest that movement quality and appearance significantly impact users' ratings of signing avatars and show that signed language users with earlier age of ASL acquisition are the most sensitive to movement quality issues seen in computer-generated avatars. We suggest that future efforts to develop signing avatars consider retaining the fluid movement qualities integral to signed languages.