More Like this

1. More than labels: neural representations of emotion words are widely distributed across the brain

   https://doi.org/10.1093/scan/nsae043  

   Lee, Kent_M; Satpute, Ajay_B (June 2024, Social Cognitive and Affective Neuroscience)

   Abstract Although emotion words such as “anger,” “disgust,” “happiness,” or “pride” are often thought of as mere labels, increasing evidence points to language as being important for emotion perception and experience. Emotion words may be particularly important for facilitating access to the emotion concepts. Indeed, deficits in semantic processing or impaired access to emotion words interfere with emotion perception. Yet, it is unclear what these behavioral findings mean for affective neuroscience. Thus, we examined the brain areas that support processing of emotion words using representational similarity analysis of functional magnetic resonance imaging data (N = 25). In the task, participants saw 10 emotion words (e.g. “anger,” “happiness”) while in the scanner. Participants rated each word based on its valence on a continuous scale ranging from 0 (Pleasant/Good) to 1 (Unpleasant/Bad) scale to ensure they were processing the words. Our results revealed that a diverse range of brain areas including prefrontal, midline cortical, and sensorimotor regions contained information about emotion words. Notably, our results overlapped with many regions implicated in decoding emotion experience by prior studies. Our results raise questions about what processes are being supported by these regions during emotion experience. 

   more » « less
2. The Power of Voice to Convey Emotion in Multimedia Instructional Messages

   Alyssa Lawson and Richard Mayer (October 2021, International journal of artificial intelligence in education)

   This study examines an aspect of the role of emotion in multimedia learning, i.e., whether participants can recognize the instructor’s positive or negative emotion based on hearing short clips involving only the instructor’s voice just as well as also seeing an embodied onscreen agent. Participants viewed 16 short video clips from a statistics lecture in which an animated instructor, conveying a happy, content, frustrated, or bored emotion, stands next to a slide as she lectures (agent present) or uses only her voice (agent absent). For each clip, participants rated the instructor on five-point scales for how happy, content, frustrated, and bored the instructor seemed. First, for happy, content, and bored instructors, participants were just as accurate in rating emotional tone based on voice only as with voice plus onscreen agent. This supports the voice hypothesis, which posits that voice is a powerful source of social-emotional information. Second, participants rated happy and content instructors higher on happy and content scales and rated frustrated and bored instructors higher on frustrated and bored scales. This supports the positivity hypothesis, which posits that people are particularly sensitive to the positive or negative tone of multimedia instructional messages. 

   more » « less
3. The Effects of Body Gestures and Gender on Viewer’s Perception of Animated Pedagogical Agent’s Emotions

   https://doi.org/https://doi.org/10.1007/978-3-030-49062-1_11  

   Cheng J., Adamo (July 2020, Kurosu M. (eds) Human-Computer Interaction. Multimodal and Natural Interaction. HCII 2020. Lecture Notes in Computer Science)

   The goal of this research is to develop Animated Pedagogical Agents (APA) that can convey clearly perceivable emotions through speech, facial expressions and body gestures. In particular, the two studies reported in the paper investigated the extent to which modifications to the range of movement of 3 beat gestures, e.g., both arms synchronous outward gesture, both arms synchronous forward gesture, and upper body lean, and the agent‘s gender have significant effects on viewer’s perception of the agent’s emotion in terms of valence and arousal. For each gesture the range of movement was varied at 2 discrete levels. The stimuli of the studies were two sets of 12-s animation clips generated using fractional factorial designs; in each clip an animated agent who speaks and gestures, gives a lecture segment on binomial probability. 50% of the clips featured a female agent and 50% of the clips featured a male agent. In the first study, which used a within-subject design and metric conjoint analysis, 120 subjects were asked to watch 8 stimuli clips and rank them according to perceived valence and arousal (from highest to lowest). In the second study, which used a between-subject design, 300 participants were assigned to two groups of 150 subjects each. One group watched 8 clips featuring the male agent and one group watched 8 clips featuring the female agent. Each participant was asked to rate perceived valence and arousal for each clip using a 7-point Likert scale. Results from the two studies suggest that the more open and forward the gestures the agent makes, the higher the perceived valence and arousal. Surprisingly, agents who lean their body forward more are not perceived as having higher arousal and valence. Findings also show that female agents’ emotions are perceived as having higher arousal and more positive valence that male agents’ emotions. 

   more » « less
4. Facial expressions of emotions by people with visual impairment and blindness via video conferencing

   https://doi.org/10.1177/02646196231163491  

   Nam_Kim, Hyung (April 2023, British Journal of Visual Impairment)

   Many people including those with visual impairment and blindness take advantage of video conferencing tools to meet people. Video conferencing tools enable them to share facial expressions that are considered as one of the most important aspects of human communication. This study aims to advance knowledge of how those with visual impairment and blindness share their facial expressions of emotions virtually. This study invited a convenience sample of 28 adults with visual impairment and blindness to Zoom video conferencing. The participants were instructed to pose facial expressions of basic human emotions (anger, fear, disgust, happiness, surprise, neutrality, calmness, and sadness), which were video recorded. The facial expressions were analyzed using the Facial Action Coding System (FACS) that encodes the movement of specific facial muscles called Action Units (AUs). This study found that there was a particular set of AUs significantly engaged in expressing each emotion, except for sadness. Individual differences were also found in AUs influenced by the participants’ visual acuity levels and emotional characteristics such as valence and arousal levels. The research findings are anticipated to serve as the foundation of knowledge, contributing to developing emotion-sensing technologies for those with visual impairment and blindness. 

   more » « less
5. Emotional valence and arousal induced by auditory stimuli among individuals with visual impairment

   https://doi.org/10.1177/02646196211042257  

   Kim, Hyung_Nam; Sutharson, Sam_Jotham (September 2021, British Journal of Visual Impairment)

   Despite significant vision loss, humans can still recognize various emotional stimuli via a sense of hearing and express diverse emotional responses, which can be sorted into two dimensions, arousal and valence. Yet, many research studies have been focusing on sighted people, leading to lack of knowledge about emotion perception mechanisms of people with visual impairment. This study aims at advancing knowledge of the degree to which people with visual impairment perceive various emotions – high/low arousal and positive/negative emotions. A total of 30 individuals with visual impairment participated in interviews where they listened to stories of people who became visually impaired, encountered and overcame various challenges, and they were instructed to share their emotions. Participants perceived different kinds and intensities of emotions, depending on their demographic variables such as living alone, loneliness, onset of visual impairment, visual acuity, race/ethnicity, and employment status. The advanced knowledge of emotion perceptions in people with visual impairment is anticipated to contribute toward better designing social supports that can adequately accommodate those with visual impairment. 

   more » « less