skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Multisensory integration facilitates perceptual restoration of an interrupted call in a species of frog
Abstract Communication signals by both human and non-human animals are often interrupted in nature. One advantage of multimodal cues is to maintain the salience of interrupted signals. We studied a frog that naturally can have silent gaps within its call. Using video/audio-playbacks, we presented females with interrupted mating calls with or without a simultaneous dynamic (i.e., inflating and deflating) vocal sac and tested whether multisensory cues (noise and/or dynamic vocal sac) inserted into the gap can compensate an interrupted call. We found that neither inserting white noise into the silent gap of an interrupted call nor displaying the dynamic vocal sac in that same gap restored the attraction of the call equivalent to that of a complete call. Simultaneously presenting a dynamic vocal sac along with noise in the gap, however, compensated the interrupted call, making it as attractive as a complete call. Our results demonstrate that the dynamic visual sac compensates for noise interference. Such novel multisensory integration suggests that multimodal cues can provide insurance against imperfect sender coding in a noisy environment, and the communication benefits to the receiver from multisensory integration may be an important selective force favoring multimodal signal evolution.  more » « less
Award ID(s):
1914652
PAR ID:
10419445
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Editor(s):
Jennions, Michael D
Date Published:
Journal Name:
Behavioral Ecology
Volume:
33
Issue:
4
ISSN:
1045-2249
Page Range / eLocation ID:
876 to 883
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Behavioral Ecology)
    Candolin, Ulrika (Ed.)
    Abstract Females of many species choose mates using multiple sensory modalities. Multimodal noise may arise, however, in dense aggregations of animals communicating via multiple sensory modalities. Some evidence suggests multimodal signals may not always improve receiver decision-making performance. When sensory systems process input from multimodal signal sources, multimodal noise may arise and potentially complicate decision-making due to the demands on cognitive integration tasks. We tested female túngara frog, Physalaemus (=Engystomops) pustulosus, responses to male mating signals in noise from multiple sensory modalities (acoustic and visual). Noise treatments were partitioned into three categories: acoustic, visual, and multimodal. We used natural calls from conspecifics and heterospecifics for acoustic noise. Robotic frogs were employed as either visual signal components (synchronous vocal sac inflation with call) or visual noise (asynchronous vocal sac inflation with call). Females expressed a preference for the typically more attractive call in the presence of unimodal noise. However, during multimodal signal and noise treatments (robofrogs employed with background noise), females failed to express a preference for the typically attractive call in the presence of conspecific chorus noise. We found that social context and temporal synchrony of multimodal signaling components are important for multimodal communication. Our results demonstrate that multimodal signals have the potential to increase the complexity of the sensory scene and reduce the efficacy of female decision making. 
    more » « less
  2. ; ; ; ; ; ; (, Journal of experimental biology)
    null (Ed.)
    Communication systems often include a variety of components, including those that span modalities, which may facilitate detection and decision-making. For example, female tungara frogs and fringe-lipped bats generally rely on acoustic mating signals to find male tungara frogs in a mating or foraging context, respectively. However, two additional cues (vocal sac inflation and water ripples) can enhance detection and choice behavior. To date, we do not know the natural variation and covariation of these three components. To address this, we made detailed recordings of calling males, including call amplitude, vocal sac volume and water ripple height, in 54 frogs (2430 calls). We found that all three measures correlated, with the strongest association between the vocal sac volume and call amplitude. We also found that multimodal models predicted the mass of calling males better than unimodal models. These results demonstrate how multimodal components of a communication system relate to each other and provide an important foundation for future studies on how receivers integrate and compare complex displays. 
    more » « less
  3. ; ; ; (, IEEE)
    The human-robot interaction (HRI) field has rec- ognized the importance of enabling robots to interact with teams. Human teams rely on effective communication for suc- cessful collaboration in time-sensitive environments. Robots can play a role in enhancing team coordination through real-time assistance. Despite significant progress in human-robot teaming research, there remains an essential gap in how robots can effectively communicate with action teams using multimodal interaction cues in time-sensitive environments. This study addresses this knowledge gap in an experimental in-lab study to investigate how multimodal robot communication in action teams affects workload and human perception of robots. We explore team collaboration in a medical training scenario where a robotic crash cart (RCC) provides verbal and non-verbal cues to help users remember to perform iterative tasks and search for supplies. Our findings show that verbal cues for object search tasks and visual cues for task reminders reduce team workload and increase perceived ease of use and perceived usefulness more effectively than a robot with no feedback. Our work contributes to multimodal interaction research in the HRI field, highlighting the need for more human-robot teaming research to understand best practices for integrating collaborative robots in time-sensitive environments such as in hospitals, search and rescue, and manufacturing applications. 
    more » « less
  4. ; ; ; ; ; ; (, Advanced Materials Technologies)
    Abstract Silent speech interfaces offer an alternative and efficient communication modality for individuals with voice disorders and when the vocalized speech communication is compromised by noisy environments. Despite the recent progress in developing silent speech interfaces, these systems face several challenges that prevent their wide acceptance, such as bulkiness, obtrusiveness, and immobility. Herein, the material optimization, structural design, deep learning algorithm, and system integration of mechanically and visually unobtrusive silent speech interfaces are presented that can realize both speaker identification and speech content identification. Conformal, transparent, and self‐adhesive electromyography electrode arrays are designed for capturing speech‐relevant muscle activities. Temporal convolutional networks are employed for recognizing speakers and converting sensing signals into spoken content. The resulting silent speech interfaces achieve a 97.5% speaker classification accuracy and 91.5% keyword classification accuracy using four electrodes. The speech interface is further integrated with an optical hand‐tracking system and a robotic manipulator for human‐robot collaboration in both assembly and disassembly processes. The integrated system achieves the control of the robot manipulator by silent speech and facilitates the hand‐over process by hand motion trajectory detection. The developed framework enables natural robot control in noisy environments and lays the ground for collaborative human‐robot tasks involving multiple human operators. 
    more » « less
  5. ; ; ; ; ; ; ; (, Journal of Experimental Biology)
    Noise is a common problem in animal communication. We know little, however, about how animals communicate in noise using multimodal signals. Multimodal signals are hypothesized to be favoured by evolution because they increase the efficacy of detection/discrimination in noisy environments. We tested the hypothesis that female túngara frogs’ responses to attractive male advertisement calls are improved in noise when a visual signal component is added to the available choices. We tested this at two levels of decision complexity (two and three choices). In a two-choice test, the presence of noise did not reduce female preferences for attractive calls. The visual component of a calling male, associated with an unattractive call, also did not reduce preference for attractive calls in the absence of noise. In the presence of noise, however, females were more likely to choose an unattractive call coupled with the visual component. In three-choice tests, the presence of noise alone reduced female responses to attractive calls and this was not strongly affected by the presence or absence of visual components. The responses in these experiments fail to support the multimodal signal efficacy hypothesis. Instead, the data suggest that audio-visual perception and cognitive processing, related to mate choice decisions, are dependent on the complexity of the sensory scene. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email