skip to main content

Title: A Vibrothermal Haptic Display for Socio-emotional Communication
Touch plays a vital role in maintaining human relationships through social and emotional communication. The proposed haptic display prototype generates stimuli in vibrotactile and thermal modalities toward simulating social touch cues between remote users. High-dimensional spatiotemporal vibrotactile-thermal (vibrothermal) patterns were evaluated with ten participants. The device can be wirelessly operated to enable remote communication. In the future, such patterns can be used to richly simulate social touch cues. A research study was conducted in two parts: first, the identification accuracy of vibrothermal patterns was explored; and second, the relatability of vibrothermal patterns to social touch experienced during social interactions was evaluated. Results revealed that while complex patterns were difficult to identify, simpler patterns, such as SINGLE TAP and HOLD, were highly identifiable and highly relatable to social touch cues. Directional patterns were less identifiable and less relatable to the social touch cues experienced during social interaction.
; ;
Award ID(s):
Publication Date:
Journal Name:
23rd International Conference on Human-Computer Interaction (HCII 2021) - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence
Page Range or eLocation-ID:
Sponsoring Org:
National Science Foundation
More Like this
  1. Synopsis An organism’s ability to integrate transient environmental cues experienced during development into molecular and physiological responses forms the basis for adaptive shifts in phenotypic trajectories. During temperature-dependent sex determination (TSD), thermal cues during discrete periods in development coordinate molecular changes that ultimately dictate sexual fate and contribute to patterns of inter- and intra-sexual variation. How these mechanisms interface with dynamic thermal environments in nature remain largely unknown. By deploying thermal loggers in wild nests of the American alligator (Alligator mississippiensis) over two consecutive breeding seasons, we observed that 80% of nests exhibit both male- and female-promoting thermal cues during the thermosensitive period, and of these nests, all exhibited both male- and female-promoting temperatures within the span of a single day. These observations raise a critical question—how are opposing environmental cues integrated into sexually dimorphic transcriptional programs across short temporal scales? To address this question, alligator embryos were exposed to fluctuating temperatures based on nest thermal profiles and sampled over the course of a daily thermal fluctuation. We examined the expression dynamics of upstream genes in the temperature-sensing pathway and find that post-transcriptional alternative splicing and transcript abundance of epigenetic modifier genes JARID2 and KDM6B respond rapidly to thermal fluctuationsmore »while transcriptional changes of downstream effector genes, SOX9 and DMRT1, occur on a delayed timescale. Our findings reveal how the basic mechanisms of TSD operate in an ecologically relevant context. We present a hypothetical hierarchical model based on our findings as well as previous studies, in which temperature-sensitive alternative splicing incrementally influences the epigenetic landscape to affect the transcriptional activity of key sex-determining genes.« less
  2. Given that most cues exchanged during a social interaction are nonverbal (e.g., facial expressions, hand gestures, body language), individuals who are blind are at a social disadvantage compared to their sighted peers. Very little work has explored sensory augmentation in the context of social assistive aids for individuals who are blind. The purpose of this study is to explore the following questions related to visual-to-vibrotactile mapping of facial action units (the building blocks of facial expressions): (1) How well can individuals who are blind recognize tactile facial action units compared to those who are sighted? (2) How well can individuals who are blind recognize emotions from tactile facial action units compared to those who are sighted? These questions are explored in a preliminary pilot test using absolute identification tasks in which participants learn and recognize vibrotactile stimulations presented through the Haptic Chair, a custom vibrotactile display embedded on the back of a chair. Study results show that individuals who are blind are able to recognize tactile facial action units as well as those who are sighted. These results hint at the potential for tactile facial action units to augment and expand access to social interactions for individuals who are blind.
  3. Objective

    This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance.


    An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task.


    Participants ( N = 16) were provided combinations of torque assistance and vibrotactile cues at pre-specified time points in late swing and early stance while walking on a self-paced treadmill. Participants were either given explicit instructions ( N = 8) or were allowed to freely interpret (N=8) how to coordinate with cues.


    For the free interpretation group, the data support an 8% increase in stride length and 14% increase in speed with exoskeleton torque across cue timing, as well as a 5% increase in stride length and 7% increase in speed with only vibrotactile cues. When given explicit instructions, participants modulated speed according to cue timing—increasing speed by 17% at cues in late swing and decreasing speed 11% at cues in early stance compared to no cue when exoskeleton torque was off. When torque was on, participants with explicit instructions had reduced changes in speed.


    These findings support that the presence of torque mitigates how cues were usedmore »and highlights the importance of explicit instructions for haptic cuing. Interpreting cues while walking with an exoskeleton may increase cognitive load, influencing overall human-exoskeleton performance for novice users.


    Interactions between haptic feedback and exoskeleton use during gait can inform future feedback designs to support coordination between users and exoskeletons.

    « less
  4. We study student experiences of social VR for remote instruction, with students attending class from home. The study evaluates student experiences when: (1) viewing remote lectures with VR headsets, (2) viewing with desktop displays, (3) presenting with VR headsets, and (4) reflecting on several weeks of VR-based class attendance. Students rated factors such as presence, social presence, simulator sickness, communication methods, avatar and application features, and tradeoffs with other remote approaches. Headset-based viewing and presenting produced higher presence than desktop viewing, but had less-clear impact on overall experience and on most social presence measures. We observed higher attentional allocation scores for headset-based presenting than for both viewing methods. For headset VR, there were strong negative correlations between simulator sickness (primarily reported as general discomfort) and ratings of co-presence, overall experience, and some other factors. This suggests that comfortable users experienced substantial benefits of headset viewing and presenting, but others did not. Based on the type of virtual environment, student ratings, and comments, reported discomfort appears related to physical ergonomic factors or technical problems. Desktop VR appears to be a good alternative for uncomfortable students, and students report that they prefer a mix of headset and desktop viewing. We additionally providemore »insight from students and a teacher about possible improvements for VR class technology, and we summarize student opinions comparing viewing and presenting in VR to other remote class technologies.« less
  5. Applegate, Roger (Ed.)
    Abstract - Odocoileus virginianus (White-tailed Deer) are social animals that thrive in rural and urban settings. Scraping behavior is an olfactory reproductive communication used by White-tailed Deer to establish breeding networks. Male scraping is a complex scent-marking behavior which advertises sociosexual status and location to potential females as well as to competing males. Female scraping behavior is thought to be an estrus signal alerting males during times of optimal fertility. This study describes a new method to examine White-tailed Deer mating systems using social network analyses of scraping behavior using an urban population of White-tailed Deer as a model. First, we validated the scraping behavior at our study site in Tougaloo, MS, during the 2019–2020 breeding season. Using remote monitoring, we continuously documented scraping behaviors over 8 different scrape-site locations and found similar behavioral, temporal, and spatial patterns in our urban breeding network as reported in rural and captive deer studies. Next, we describe methods detailing how social network analyses can reveal sociality, dominance, importance, and social structure within male scraping networks. Using centrality measures, we were able to rank dominant male influencers, anticipate social conflict among rivals, and made predictions regarding the spread of communicable diseases through a malemore »scraping network. We also detail network analyses combining both male and female scraping behavior to reveal a glimpse into the complexity of breeding networks. Using network measures, we were able to rank males based on competitiveness and female preference. Lastly, we generated a theoretical breeding network to explore female sociability, competitiveness, preference, and mate choice. Taken together, this work describes a new method using scraping network analysis to investigate the complexity of White-tailed Deer breeding networks. This work also demonstrates the future applications of this method for predicting the spread of communicable diseases and for predicting mate selection within White-tailed Deer mating systems.« less