More Like this

1. The neurobiology of interoception and affect

   https://doi.org/10.1016/j.tics.2024.01.009  

   Feldman, MJ; Bliss-Moreau, E; Lindquist, KA (July 2024, Trends in Cognitive Sciences)

   Drayton, L (Ed.)

   Scholars have argued for centuries that affective states involve interoception, or representations of the state of the body. Yet, we lack a mechanistic understanding of how signals from the body are transduced, transmitted, compressed, and integrated by the brains of humans to produce affective states. We suggest that to understand how the body contributes to affect, we first need to understand information flow through the nervous system’s interoceptive pathways. We outline such a model and discuss how unique anatomical and physiological aspects of interoceptive pathways may give rise to the qualities of affective experiences in general and valence and arousal in particular. We conclude by considering implications and future directions for research on interoception, affect, emotions, and human mental experiences. 

   more » « less
2. Modeling Uncertainty in Predicting Emotional Attributes from Spontaneous Speech

   https://doi.org/10.1109/ICASSP40776.2020.9054237  

   Sridhar, Kusha; Busso, Carlos (May 2020, IEEE international conference on acoustics, speech and signal processing (ICASSP 2020))

   A challenging task in affective computing is to build reliable speech emotion recognition (SER) systems that can accurately predict emotional attributes from spontaneous speech. To increase the trust in these SER systems, it is important to predict not only their accuracy, but also their confidence. An intriguing approach to predict uncertainty is Monte Carlo (MC) dropout, which obtains pre- dictions from multiple feed-forward passes through a deep neural network (DNN) by using dropout regularization in both training and inference. This study evaluates this approach with regression models to predict emotional attribute scores for valence, arousal and dom- inance. The analysis illustrates that predicting uncertainty in this problem is possible, where the performance is higher for samples in the test set with lower uncertainty. The study evaluates uncertainty estimation as a function of the emotional attributes, showing that samples with extreme values have lower uncertainty. Finally, we demonstrate the benefits of uncertainty estimation with reject option, where a classifier can decline to give a prediction when its confi- dence is low. By rejecting only 25% of the test set with the highest uncertainty, we achieve relative performance gains of 7.34% for arousal, 13.73% for valence and 8.79% for dominance. 

   more » « less
3. Common Functional Brain States Encode both Perceived Emotion and the Psychophysiological Response to Affective Stimuli

   https://doi.org/10.1034/s41598-33621-6  

   Bush, Keith A.; Privratsky, Anthony; Gardner, Jonathan; Zielinski, Melissa J.; Kilts, Clinton D. (October 2018, Scientific reports)

   Multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data has critically advanced the neuroanatomical understanding of affect processing in the human brain. Central to these advancements is the brain state, a temporally-succinct fMRI-derived pattern of neural activation, which serves as a processing unit. Establishing the brain state’s central role in affect processing, however, requires that it predicts multiple independent measures of affect. We employed MVPA-based regression to predict the valence and arousal properties of visual stimuli sampled from the International Affective Picture System (IAPS) along with the corollary skin conductance response (SCR) for demographically diverse healthy human participants (n = 19). We found that brain states significantly predicted the normative valence and arousal scores of the stimuli as well as the attendant individual SCRs. In contrast, SCRs significantly predicted arousal only. The prediction effect size of the brain state was more than three times greater than that of SCR. Moreover, neuroanatomical analysis of the regression parameters found remarkable agreement with regions long-established by fMRI univariate analyses in the emotion processing literature. Finally, geometric analysis of these parameters also found that the neuroanatomical encodings of valence and arousal are orthogonal as originally posited by the circumplex model of dimensional emotion. 

   more » « less
4. Investigating the Generalizability of Physiological Characteristics of Anxiety

   https://doi.org/10.1109/BIBM58861.2023.10385292  

   Zhou, Emily; Soleymani, Mohammad; Matarić, Maja J (December 2023, IEEE)

   Recent works have demonstrated the effectiveness of machine learning (ML) techniques in detecting anxiety and stress using physiological signals, but it is unclear whether ML models are learning physiological features specific to stress. To address this ambiguity, we evaluated the generalizability of physiological features that have been shown to be correlated with anxiety and stress to high-arousal emotions. Specifically, we examine features extracted from electrocardiogram (ECG) and electrodermal (EDA) signals from the following three datasets: Anxiety Phases Dataset (APD), Wearable Stress and Affect Detection (WESAD), and the Continuously Annotated Signals of Emotion (CASE) dataset. We aim to understand whether these features are specific to anxiety or general to other high-arousal emotions through a statistical regression analysis, in addition to a within-corpus, cross-corpus, and leave-one-corpus-out cross-validation across instances of stress and arousal. We used the following classifiers: Support Vector Machines, LightGBM, Random Forest, XGBoost, and an ensemble of the aforementioned models. We found that models trained on an arousal dataset perform relatively well on a previously unseen stress dataset, and vice versa. Our experimental results suggest that the evaluated models may be identifying emotional arousal instead of stress. This work is the first cross-corpus evaluation across stress and arousal from ECG and EDA signals, contributing new findings about the generalizability of stress detection. 

   more » « less
5. Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions

   https://doi.org/10.1038/s41598-019-52891-2  

   Siddharth, Siddharth; Jung, Tzyy-Ping; Sejnowski, Terrence J. (November 2019, Scientific Reports)

   Abstract Most of the research in the field of affective computing has focused on detecting and classifying human emotions through electroencephalogram (EEG) or facial expressions. Designing multimedia content to evoke certain emotions has been largely motivated by manual rating provided by users. Here we present insights from the correlation of affective features between three modalities namely, affective multimedia content, EEG, and facial expressions. Interestingly, low-level Audio-visual features such as contrast and homogeneity of the video and tone of the audio in the movie clips are most correlated with changes in facial expressions and EEG. We also detect the regions associated with the human face and the brain (in addition to the EEG frequency bands) that are most representative of affective responses. The computational modeling between the three modalities showed a high correlation between features from these regions and user-reported affective labels. Finally, the correlation between different layers of convolutional neural networks with EEG and Face images as input provides insights into human affection. Together, these findings will assist in (1) designing more effective multimedia contents to engage or influence the viewers, (2) understanding the brain/body bio-markers of affection, and (3) developing newer brain-computer interfaces as well as facial-expression-based algorithms to read emotional responses of the viewers. 

   more » « less