Given that most cues exchanged during a social interaction are nonverbal (e.g., facial expressions, hand gestures, body language), individuals who are blind are at a social disadvantage compared to their sighted peers. Very little work has explored sensory augmentation in the context of social assistive aids for individuals who are blind. The purpose of this study is to explore the following questions related to visual-to-vibrotactile mapping of facial action units (the building blocks of facial expressions): (1) How well can individuals who are blind recognize tactile facial action units compared to those who are sighted? (2) How well can individuals who are blind recognize emotions from tactile facial action units compared to those who are sighted? These questions are explored in a preliminary pilot test using absolute identification tasks in which participants learn and recognize vibrotactile stimulations presented through the Haptic Chair, a custom vibrotactile display embedded on the back of a chair. Study results show that individuals who are blind are able to recognize tactile facial action units as well as those who are sighted. These results hint at the potential for tactile facial action units to augment and expand access to social interactions for individuals who are blind.
more »
« less
A Vibrothermal Haptic Display for Socio-emotional Communication
Touch plays a vital role in maintaining human relationships through social and emotional communication. The proposed haptic display prototype generates stimuli in vibrotactile and thermal modalities toward simulating social touch cues between remote users. High-dimensional spatiotemporal vibrotactile-thermal (vibrothermal) patterns were evaluated with ten participants. The device can be wirelessly operated to enable remote communication. In the future, such patterns can be used to richly simulate social touch cues. A research study was conducted in two parts: first, the identification accuracy of vibrothermal patterns was explored; and second, the relatability of vibrothermal patterns to social touch experienced during social interactions was evaluated. Results revealed that while complex patterns were difficult to identify, simpler patterns, such as SINGLE TAP and HOLD, were highly identifiable and highly relatable to social touch cues. Directional patterns were less identifiable and less relatable to the social touch cues experienced during social interaction.
more »
« less
- Award ID(s):
- 1828010
- NSF-PAR ID:
- 10344396
- Date Published:
- Journal Name:
- 23rd International Conference on Human-Computer Interaction (HCII 2021) - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence
- Volume:
- 13095
- Page Range / eLocation ID:
- 17–30
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Synopsis An organism’s ability to integrate transient environmental cues experienced during development into molecular and physiological responses forms the basis for adaptive shifts in phenotypic trajectories. During temperature-dependent sex determination (TSD), thermal cues during discrete periods in development coordinate molecular changes that ultimately dictate sexual fate and contribute to patterns of inter- and intra-sexual variation. How these mechanisms interface with dynamic thermal environments in nature remain largely unknown. By deploying thermal loggers in wild nests of the American alligator (Alligator mississippiensis) over two consecutive breeding seasons, we observed that 80% of nests exhibit both male- and female-promoting thermal cues during the thermosensitive period, and of these nests, all exhibited both male- and female-promoting temperatures within the span of a single day. These observations raise a critical question—how are opposing environmental cues integrated into sexually dimorphic transcriptional programs across short temporal scales? To address this question, alligator embryos were exposed to fluctuating temperatures based on nest thermal profiles and sampled over the course of a daily thermal fluctuation. We examined the expression dynamics of upstream genes in the temperature-sensing pathway and find that post-transcriptional alternative splicing and transcript abundance of epigenetic modifier genes JARID2 and KDM6B respond rapidly to thermal fluctuations while transcriptional changes of downstream effector genes, SOX9 and DMRT1, occur on a delayed timescale. Our findings reveal how the basic mechanisms of TSD operate in an ecologically relevant context. We present a hypothetical hierarchical model based on our findings as well as previous studies, in which temperature-sensitive alternative splicing incrementally influences the epigenetic landscape to affect the transcriptional activity of key sex-determining genes.more » « less
-
more » « less
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance.
Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task.
Methods Participants ( N = 16) were provided combinations of torque assistance and vibrotactile cues at pre-specified time points in late swing and early stance while walking on a self-paced treadmill. Participants were either given explicit instructions ( N = 8) or were allowed to freely interpret (N=8) how to coordinate with cues.
Results For the free interpretation group, the data support an 8% increase in stride length and 14% increase in speed with exoskeleton torque across cue timing, as well as a 5% increase in stride length and 7% increase in speed with only vibrotactile cues. When given explicit instructions, participants modulated speed according to cue timing—increasing speed by 17% at cues in late swing and decreasing speed 11% at cues in early stance compared to no cue when exoskeleton torque was off. When torque was on, participants with explicit instructions had reduced changes in speed.
Conclusion These findings support that the presence of torque mitigates how cues were used and highlights the importance of explicit instructions for haptic cuing. Interpreting cues while walking with an exoskeleton may increase cognitive load, influencing overall human-exoskeleton performance for novice users.
Application Interactions between haptic feedback and exoskeleton use during gait can inform future feedback designs to support coordination between users and exoskeletons.
-
null (Ed.)We study student experiences of social VR for remote instruction, with students attending class from home. The study evaluates student experiences when: (1) viewing remote lectures with VR headsets, (2) viewing with desktop displays, (3) presenting with VR headsets, and (4) reflecting on several weeks of VR-based class attendance. Students rated factors such as presence, social presence, simulator sickness, communication methods, avatar and application features, and tradeoffs with other remote approaches. Headset-based viewing and presenting produced higher presence than desktop viewing, but had less-clear impact on overall experience and on most social presence measures. We observed higher attentional allocation scores for headset-based presenting than for both viewing methods. For headset VR, there were strong negative correlations between simulator sickness (primarily reported as general discomfort) and ratings of co-presence, overall experience, and some other factors. This suggests that comfortable users experienced substantial benefits of headset viewing and presenting, but others did not. Based on the type of virtual environment, student ratings, and comments, reported discomfort appears related to physical ergonomic factors or technical problems. Desktop VR appears to be a good alternative for uncomfortable students, and students report that they prefer a mix of headset and desktop viewing. We additionally provide insight from students and a teacher about possible improvements for VR class technology, and we summarize student opinions comparing viewing and presenting in VR to other remote class technologies.more » « less
-
Touch as a modality in social communication has been getting more attention with recent developments in wearable technology and an increase in awareness of how limited physical contact can lead to touch starvation and feelings of depression. Although several mediated touch methods have been developed for conveying emotional support, the transfer of emotion through mediated touch has not been widely studied. This work addresses this need by exploring emotional communication through a novel wearable haptic system. The system records physical touch patterns through an array of force sensors, processes the recordings using novel gesture-based algorithms to create actuator control signals, and generates mediated social touch through an array of voice coil actuators. We conducted a human subject study ( N = 20) to understand the perception and emotional components of this mediated social touch for common social touch gestures, including poking, patting, massaging, squeezing, and stroking. Our results show that the speed of the virtual gesture significantly alters the participants' ratings of valence, arousal, realism, and comfort of these gestures with increased speed producing negative emotions and decreased realism. The findings from the study will allow us to better recognize generic patterns from human mediated touch perception and determine how mediated social touch can be used to convey emotion. Our system design, signal processing methods, and results can provide guidance in future mediated social touch design.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1007/978-3-030-90963-5_2
