skip to main content

Explore Research Products in the NSF-PAR

  • NSF Public Access
  • Search Results
  • Investigating Therapist Vocal Nonverbal Behavior for Applications in Robot-Mediated Therapies for Individuals Diagnosed with Autism
Title: Investigating Therapist Vocal Nonverbal Behavior for Applications in Robot-Mediated Therapies for Individuals Diagnosed with Autism
Socially assistive robots (SARs) are being utilized for delivering a variety of healthcare services to patients. The design of these human-robot interactions (HRIs) for healthcare applications have primarily focused on the interaction flow and verbal behaviors of a SAR. To date, there has been minimal focus on investigating how SAR nonverbal behaviors should be designed according to the context of the SAR’s communication goals during a HRI. In this paper, we present a methodology to investigate nonverbal behavior during specific human-human healthcare interactions so that they can be applied to a SAR. We apply this methodology to study the context-dependent vocal nonverbal behaviors of therapists during discrete trial training (DTT) therapies delivered to children with autism. We chose DTT because it is a therapy commonly being delivered by SARs and modeled after human-human interactions. Results from our study led to the following recommendations for the design of the vocal nonverbal behavior of SARs during a DTT therapy: 1) the consequential error correction should have a lower pitch and intensity than the discriminative stimulus but maintain a similar speaking rate; and 2) the consequential reinforcement should have a higher pitch and intensity than the discriminative stimulus but a slower speaking rate.  more » « less
Award ID(s):
1948224
NSF-PAR ID:
10221653
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
International Conference on Social Robotics
Page Range / eLocation ID:
416-427
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , Paladyn, Journal of Behavioral Robotics)
    null (Ed.)
    Abstract Autism spectrum disorder (ASD) is a lifelong developmental condition that affects an individual’s ability to communicate and relate to others. Despite such challenges, early intervention during childhood development has shown to have positive long-term benefits for individuals with ASD. Namely, early childhood development of communicative speech skills has shown to improve future literacy and academic achievement. However, the delivery of such interventions is often time-consuming. Socially assistive robots (SARs) are a potential strategic technology that could help support intervention delivery for children with ASD and increase the number of individuals that healthcare professionals can positively affect. For SARs to be effectively integrated in real-world treatment for individuals with ASD, they should follow current evidence-based practices used by therapists such as Applied Behavior Analysis (ABA). In this work, we present a study that investigates the efficacy of applying well-known ABA techniques to a robot-mediated listening comprehension intervention delivered to children with ASD at a university-based ABA clinic. The interventions were delivered in place of human therapists to teach study participants a new skill as a part of their overall treatment plan. All the children participating in the intervention improved in the skill being taught by the robot and enjoyed interacting with the robot, as evident by high occurrences of positive affect as well as engagement during the sessions. One of the three participants has also reached mastery of the skill via the robot-mediated interventions. 
    more » « less
  2. ; ; ; ( , PLOS ONE)
    Coleman, Melissa J. (Ed.)
    Although male vocalizations during opposite- sex interaction have been heavily studied as sexually selected signals, the understanding of the roles of female vocal signals produced in this context is more limited. During intersexual interactions between mice, males produce a majority of ultrasonic vocalizations (USVs), while females produce a majority of human-audible squeaks, also called broadband vocalizations (BBVs). BBVs may be produced in conjunction with defensive aggression, making it difficult to assess whether males respond to BBVs themselves. To assess the direct effect of BBVs on male behavior, we used a split-cage paradigm in which high rates of male USVs were elicited by female presence on the other side of a barrier, but which precluded extensive male-female contact and the spontaneous production of BBVs. In this paradigm, playback of female BBVs decreased USV production, which recovered after the playback period. Trials in which female vocalizations were prevented by the use of female bedding alone or of anesthetized females as stimuli also showed a decrease in response to BBV playback. No non-vocal behaviors declined during playback, although digging behavior increased. Similar to BBVs, WNs also robustly suppressed USV production, albeit to a significantly larger extent. USVs suppression had two distinct temporal components. When grouped in 5-second bins, USVs interleaved with bursts of stimulus BBVs. USV suppression also adapted to BBV playback on the order of minutes. Adaptation occurred more rapidly in males that were housed individually as opposed to socially for a week prior to testing, suggesting that the adaptation trajectory is sensitive to social experience. These findings suggest the possibility that vocal interaction between male and female mice, with males suppressing USVs in response to BBVs, may influence the dynamics of communicative behavior. 
    more » « less
  3. ; ; ; ( , PLOS Biology)
    Sengupta, Piali (Ed.)
    We present a high-throughput optogenetic illumination system capable of simultaneous closed-loop light delivery to specified targets in populations of moving Caenorhabditis elegans . The instrument addresses three technical challenges: It delivers targeted illumination to specified regions of the animal’s body such as its head or tail; it automatically delivers stimuli triggered upon the animal’s behavior; and it achieves high throughput by targeting many animals simultaneously. The instrument was used to optogenetically probe the animal’s behavioral response to competing mechanosensory stimuli in the the anterior and posterior gentle touch receptor neurons. Responses to more than 43,418 stimulus events from a range of anterior–posterior intensity combinations were measured. The animal’s probability of sprinting forward in response to a mechanosensory stimulus depended on both the anterior and posterior stimulation intensity, while the probability of reversing depended primarily on the anterior stimulation intensity. We also probed the animal’s response to mechanosensory stimulation during the onset of turning, a relatively rare behavioral event, by delivering stimuli automatically when the animal began to turn. Using this closed-loop approach, over 9,700 stimulus events were delivered during turning onset at a rate of 9.2 events per worm hour, a greater than 25-fold increase in throughput compared to previous investigations. These measurements validate with greater statistical power previous findings that turning acts to gate mechanosensory evoked reversals. Compared to previous approaches, the current system offers targeted optogenetic stimulation to specific body regions or behaviors with many fold increases in throughput to better constrain quantitative models of sensorimotor processing. 
    more » « less
  4. ; ; ; ; ( , ACM)
    An overarching goal of Artificial Intelligence (AI) is creating autonomous, social agents that help people. Two important challenges, though, are that different people prefer different assistance from agents and that preferences can change over time. Thus, helping behaviors should be tailored to how an individual feels during the interaction. We hypothesize that human nonverbal behavior can give clues about users' preferences for an agent's helping behaviors, augmenting an agent's ability to computationally predict such preferences with machine learning models. To investigate our hypothesis, we collected data from 194 participants via an online survey in which participants were recorded while playing a multiplayer game. We evaluated whether the inclusion of nonverbal human signals, as well as additional context (e.g., via game or personality information), led to improved prediction of user preferences between agent behaviors compared to explicitly provided survey responses. Our results suggest that nonverbal communication -- a common type of human implicit feedback -- can aid in understanding how people want computational agents to interact with them. 
    more » « less
  5. ; ; ( , 2022 IEEE 3rd International Conference on Human-Machine Systems (ICHMS))
    Objective: Haptic perception is an important component of bidirectional human-machine interactions that allow users to better interact with their environment. Artificial haptic sensation along an individual’s hand can be evoked via noninvasive electrical nerve stimulation; however, continuous stimulation can result in adaptation of sensory perception over time. In this study, we sought to quantify the adaptation profile via the change in perceived sensation intensity over time. Approach: Noninvasive stimulation of the peripheral nerve bundles evoked haptic perception using a 2x5 electrode grid placed along the medial side of the upper arm near the median and ulnar nerves. An electrode pair that evoked haptic sensation along the forearm and hand was selected. During a trial of 110-s of continuous stimulation, a constant stimulus amplitude just below the motor threshold was delivered. Each subject was instructed to press on a force transducer producing a force amplitude matched with the perceived intensity of haptic sensation. Main Findings: A force decay (i.e., intensity of sensation) was observed in all 7 subjects. Variations in the rate of decay and the start of decay across subjects were also observed. Significance: The preliminary findings established the sensory adaptation profile of peripheral nerve stimulation. Accounting for these subject-specific profiles of adaptation can allow for more stable communication between a robotic device and a user. Additionally, sensory adaptation characterization can promote the development of new stimulation strategies that can mitigate these observed adaptations, allowing for a better and more stable human-machine interaction experience. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email