skip to main content


Title: A Computational View of the Emotional Regulation of Disgust using Multimodal Sensors
Emotion regulation can be characterized by different activities that attempt to alter an emotional response, whether behavioral, physiological or neurological. The two most widely adopted strategies, cognitive reappraisal and expressive suppression are explored in this study, specifically in the context of disgust. Study participants (N = 21) experienced disgust via video exposure, and were instructed to either regulate their emotions or express them freely. If regulating, they were required to either cognitively reappraise or suppress their emotional experiences while viewing the videos. Video recordings of the participants' faces were taken during the experiment and electrocardiogram (ECG), electromyography (EMG), and galvanic skin response (GSR) readings were also collected for further analysis. We compared the participants behavioral (facial musculature movements) and physiological (GSR and heart rate) responses as they aimed to alter their emotional responses and computationally determined that when responding to disgust stimuli, the signals recorded during suppression and free expression were very similar, whereas those recorded during cognitive reappraisal were significantly different. Thus, in the context of this study, from a signal analysis perspective, we conclude that emotion regulation via cognitive reappraisal significantly alters participants' physiological responses to disgust, unlike regulation via suppression.  more » « less
Award ID(s):
1846076
NSF-PAR ID:
10214020
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)
Page Range / eLocation ID:
567 to 571
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    While negative affect (NA) typically increases risk for binge eating, the ultimate impact of NA may depend on a person's ability to regulate their emotions. In this daily, longitudinal study, we examined whether emotion regulation (ER) modified the strength of NA‐dysregulated eating associations.

    Methods

    Women (N = 311) from the Michigan State University Twin Registry first reported dimensional binge eating symptoms and broad ER difficulties (e.g., limited emotional awareness, difficulty controlling emotional impulses). Participants then rated use of adaptive (cognitive reappraisal, social sharing, situation modification, and acceptance) and maladaptive (rumination, expressive suppression, and self‐criticism) ER strategies, emotional eating (EE), objective binge eating (OBE), and NA once daily for 49 consecutive days.

    Results

    There were several main effects of ER on binge‐eating pathology in both between‐person (i.e., comparing women who differed on average) and within‐person (i.e., examining fluctuations in variables day‐to‐day) analyses. Between‐person, greater broad ER difficulties, greater maladaptive strategy use, and lower adaptive strategy use were all associated with greater binge‐eating pathology. Within‐person, greater maladaptive strategy use was associated with greater odds of OBE on that day and on the following day. However, neither broad ER difficulties nor use of specific strategies moderated associations between NA and dysregulated eating in between‐ or within‐person analyses.

    Conclusions

    While ER is independently associated with risk for dysregulated eating, it may not fully mitigate the impact of NA. Additional strategies (e.g., decreasing environmental stressors and increasing social support) may be needed to minimize NA and its impact on dysregulated eating.

    Public Significance

    Negative affect (NA; e.g., sadness, guilt) increases dysregulated eating risk. Because NA is sometimes unavoidable, we examined whether emotion regulation (ER; i.e., how a person responds to their emotions) might impact whether NA leads to dysregulated eating. Although more effective ER was associated with less dysregulated eating overall, ER did not impact the association between NA and dysregulated eating. Other approaches may therefore be needed to mitigate NA‐dysregulated eating associations.

     
    more » « less
  2. Many people including those with visual impairment and blindness take advantage of video conferencing tools to meet people. Video conferencing tools enable them to share facial expressions that are considered as one of the most important aspects of human communication. This study aims to advance knowledge of how those with visual impairment and blindness share their facial expressions of emotions virtually. This study invited a convenience sample of 28 adults with visual impairment and blindness to Zoom video conferencing. The participants were instructed to pose facial expressions of basic human emotions (anger, fear, disgust, happiness, surprise, neutrality, calmness, and sadness), which were video recorded. The facial expressions were analyzed using the Facial Action Coding System (FACS) that encodes the movement of specific facial muscles called Action Units (AUs). This study found that there was a particular set of AUs significantly engaged in expressing each emotion, except for sadness. Individual differences were also found in AUs influenced by the participants’ visual acuity levels and emotional characteristics such as valence and arousal levels. The research findings are anticipated to serve as the foundation of knowledge, contributing to developing emotion-sensing technologies for those with visual impairment and blindness.

     
    more » « less
  3. Meta-analyses have not shown emotions to be significant predictors of deception. Criticisms of this conclusion argued that individuals must be engaged with each other in higher stake situations for such emotions to manifest, and that these emotions must be evaluated in their verbal context (Frank and Svetieva in J Appl Res Memory Cognit 1:131–133, 10.1016/j.jarmac.2012.04.006, 2012). This study examined behavioral synchrony as a marker of engagement in higher stakes truthful and deceptive interactions, and then compared the differences in facial expressions of fear, contempt, disgust, anger, and sadness not consistent with the verbal content. Forty-eight pairs of participants were randomly assigned to interviewer and interviewee, and the interviewee was assigned to steal either a watch or a ring and to lie about the item they stole, and tell the truth about the other, under conditions of higher stakes of up to $30 rewards for successful deception, and $0 plus having to write a 15-min essay for unsuccessful deception. The interviews were coded for expression of emotions using EMFACS (Friesen and Ekman in EMFACS-7; emotional facial action coding system, 1984). Synchrony was demonstrated by the pairs of participants expressing overlapping instances of happiness (AU6 + 12). A 3 (low, moderate, high synchrony) × 2 (truth, lie) mixed-design ANOVA found that negative facial expressions of emotion were a significant predictor of deception, but only when they were not consistent with the verbal content, in the moderate and high synchrony conditions. This finding is consistent with data and theorizing that shows that with higher stakes, or with higher engagement, emotions can be a predictor of deception. 
    more » « less
  4. The overall goal of our research is to develop a system of intelligent multimodal affective pedagogical agents that are effective for different types of learners (Adamo et al., 2021). While most of the research on pedagogical agents tends to focus on the cognitive aspects of online learning and instruction, this project explores the less-studied role of affective (or emotional) factors. We aim to design believable animated agents that can convey realistic, natural emotions through speech, facial expressions, and body gestures and that can react to the students’ detected emotional states with emotional intelligence. Within the context of this goal, the specific objective of the work reported in the paper was to examine the extent to which the agents’ facial micro-expressions affect students’ perception of the agents’ emotions and their naturalness. Micro-expressions are very brief facial expressions that occur when a person either deliberately or unconsciously conceals an emotion being felt (Ekman &Friesen, 1969). Our assumption is that if the animated agents display facial micro expressions in addition to macro expressions, they will convey higher expressive richness and naturalness to the viewer, as “the agents can possess two emotional streams, one based on interaction with the viewer and the other based on their own internal state, or situation” (Queiroz et al. 2014, p.2).The work reported in the paper involved two studies with human subjects. The objectives of the first study were to examine whether people can recognize micro-expressions (in isolation) in animated agents, and whether there are differences in recognition based on the agent’s visual style (e.g., stylized versus realistic). The objectives of the second study were to investigate whether people can recognize the animated agents’ micro-expressions when integrated with macro-expressions, the extent to which the presence of micro + macro-expressions affect the perceived expressivity and naturalness of the animated agents, the extent to which exaggerating the micro expressions, e.g. increasing the amplitude of the animated facial displacements affects emotion recognition and perceived agent naturalness and emotional expressivity, and whether there are differences based on the agent’s design characteristics. In the first study, 15 participants watched eight micro-expression animations representing four different emotions (happy, sad, fear, surprised). Four animations featured a stylized agent and four a realistic agent. For each animation, subjects were asked to identify the agent’s emotion conveyed by the micro-expression. In the second study, 234 participants watched three sets of eight animation clips (24 clips in total, 12 clips per agent). Four animations for each agent featured the character performing macro-expressions only, four animations for each agent featured the character performing macro- + micro-expressions without exaggeration, and four animations for each agent featured the agent performing macro + micro-expressions with exaggeration. Participants were asked to recognize the true emotion of the agent and rate the emotional expressivity ad naturalness of the agent in each clip using a 5-point Likert scale. We have collected all the data and completed the statistical analysis. Findings and discussion, implications for research and practice, and suggestions for future work will be reported in the full paper. ReferencesAdamo N., Benes, B., Mayer, R., Lei, X., Meyer, Z., &Lawson, A. (2021). Multimodal Affective Pedagogical Agents for Different Types of Learners. In: Russo D., Ahram T., Karwowski W., Di Bucchianico G., Taiar R. (eds) Intelligent Human Systems Integration 2021. IHSI 2021. Advances in Intelligent Systems and Computing, 1322. Springer, Cham. https://doi.org/10.1007/978-3-030-68017-6_33Ekman, P., &Friesen, W. V. (1969, February). Nonverbal leakage and clues to deception. Psychiatry, 32(1), 88–106. https://doi.org/10.1080/00332747.1969.11023575 Queiroz, R. B., Musse, S. R., &Badler, N. I. (2014). Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces. Presence, 23(2), 191-208. http://dx.doi.org/10.1162/

     
    more » « less
  5. Abstract

    Friends and therapists often encourage people in distress to say how they feel (i.e., name their emotions) with the hope that identifying their emotions will help them cope. Although lay and some psychological theories posit that emotion naming should facilitate subsequent emotion regulation, there is little research directly testing this question. Here, we report on two experimental studies that test how naming the emotions evoked by aversive images impacts subsequent regulation of those emotions. In study 1 (N= 80), participants were randomly assigned into one of four between-subjects conditions in which they either (i) passively observed aversive images, (ii) named the emotions that these images made them feel, (iii) regulated their emotions by reappraising the meaning of images, or (iv) both named and regulated their emotions. Analyses of self-reported negative affect revealed that emotion naming impeded emotion regulation via reappraisal. Participants who named their emotions before reappraising reported feelingworsethan those who regulated without naming. Study 2 (N= 60) replicated these findings in a within-participants design, demonstrated that emotion naming also impeded regulation via mindful acceptance, and showed that observed effects were unrelated to a measure of social desirability, thereby mitigating the concern of experimenter demand. Together, these studies show that the impact of emotion naming on emotion regulation opposes common intuitions: instead of facilitating emotion regulation via reappraisal or acceptance, constructing an instance of a specific emotion category by giving it a name may “crystalize” one’s affective experience and make it more resistant to modification.

     
    more » « less