An official website of the United States government
For over a century, psychology has focused on uncovering mental processes of a single individual. However, humans rarely navigate the world in isolation. The most important determinants of successful development, mental health, and our individual traits and preferences arise from interacting with other individuals. Social interaction underpins who we are, how we think, and how we behave. Here we discuss the key methodological challenges that have limited progress in establishing a robust science of how minds interact and the new tools that are beginning to overcome these challenges. A deep understanding of the human mind requires studying the context within which it originates and exists: social interaction.
more »
« less
HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence
As social robots and other intelligent machines enter the home, artificial emotional intelligence (AEI) is taking center stage to address users’ desire for deeper, more meaningful human-machine interaction. To accomplish such efficacious interaction, the next-generation AEI need comprehensive human emotion models for training. Unlike theory of emotion, which has been the historical focus in psychology, emotion models are a descriptive tools. In practice, the strongest models need robust coverage, which means defining the smallest core set of emotions from which all others can be derived. To achieve the desired coverage, we turn to word embeddings from natural language processing. Using unsupervised clustering techniques, our experiments show that with as few as 15 discrete emotion categories, we can provide maximum coverage across six major languages–Arabic, Chinese, English, French, Spanish, and Russian. In support of our findings, we also examine annotations from two large-scale emotion recognition datasets to assess the validity of existing emotion models compared to human perception at scale. Because robust, comprehensive emotion models are foundational for developing real-world affective computing applications, this work has broad implications in social robotics, human-machine interaction, mental healthcare, and computational psychology.
more »
« less
- Award ID(s):
- 2234195
- PAR ID:
- 10572183
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- IEEE Transactions on Affective Computing
- Volume:
- 15
- Issue:
- 3
- ISSN:
- 2371-9850
- Page Range / eLocation ID:
- 1136 to 1152
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In recent years, extensive research has emerged in affective computing on topics like automatic emotion recognition and determining the signals that characterize individual emotions. Much less studied, however, is expressiveness—the extent to which someone shows any feeling or emotion. Expressiveness is related to personality and mental health and plays a crucial role in social interaction. As such, the ability to automatically detect or predict expressiveness can facilitate significant advancements in areas ranging from psychiatric care to artificial social intelligence. Motivated by these potential applications, we present an extension of the BP4D+ data set [27] with human ratings of expressiveness and develop methods for (1) automatically predicting expressiveness from visual data and (2) defining relationships between interpretable visual signals and expressiveness. In addition, we study the emotional context in which expressiveness occurs and hypothesize that different sets of signals are indicative of expressiveness in different con-texts (e.g., in response to surprise or in response to pain). Analysis of our statistical models confirms our hypothesis. Consequently, by looking at expressiveness separately in distinct emotional contexts, our predictive models show significant improvements over baselines and achieve com-parable results to human performance in terms of correlation with the ground truth.more » « less
-
Emotions are often thought of as internal mental states centering on individuals’ subjective feelings and evaluations. This understanding is consistent with studies of emotion narratives, or the descriptions people give for experienced events that they regard as emotions. Yet these studies, and contemporary psychology more generally, often rely on observations of educated Europeans and European Americans, constraining psychological theory and methods. In this article, we present observations from an inductive, qualitative analysis of interviews conducted with the Hadza, a community of small-scale hunter-gatherers in Tanzania, and juxtapose them with a set of interviews conducted with Americans from North Carolina. Although North Carolina event descriptions largely conformed to the assumptions of eurocentric psychological theory, Hadza descriptions foregrounded action and bodily sensations, the physical environment, immediate needs, and the experiences of social others. These observations suggest that subjective feelings and internal mental states may not be the organizing principle of emotion the world around. Qualitative analysis of emotion narratives from outside of a U.S. (and western) cultural context has the potential to uncover additional diversity in meaning-making, offering a descriptive foundation on which to build a more robust and inclusive science of emotion.more » « less
-
The computational modeling of groups requires models that connect micro-level with macro-level processes and outcomes. Recent research in computational social science has started from simple models of human behavior, and attempted to link to social structures. However, these models make simplifying assumptions about human understanding of culture that are often not realistic and may be limiting in their generality. In this paper, we present work on Bayesian affect control theory as a more comprehensive, yet highly parsimonious model that integrates artificial intelligence, social psychology, and emotions into a single predictive model of human activities in groups. We illustrate these developments with examples from an ongoing research project aimed at computational analysis of virtual software development teams.more » « less
-
Training emotion recognition models has relied heavily on human annotated data, which present diversity, quality, and cost challenges. In this paper, we explore the potential of Large Language Models (LLMs), specifically GPT-4, in automating or assisting emotion annotation. We compare GPT-4 with supervised models and/or humans in three aspects: agreement with human annotations, alignment with human perception, and impact on model training. We find that common metrics that use aggregated human annotations as ground truth can underestimate GPT-4's performance, and our human evaluation experiment reveals a consistent preference for GPT-4 annotations over humans across multiple datasets and evaluators. Further, we investigate the impact of using GPT-4 as an annotation filtering process to improve model training. Together, our findings highlight the great potential of LLMs in emotion annotation tasks and underscore the need for refined evaluation methodologies.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1109/TAFFC.2023.3324902
