An official website of the United States government
Multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data has critically advanced the neuroanatomical understanding of affect processing in the human brain. Central to these advancements is the brain state, a temporally-succinct fMRI-derived pattern of neural activation, which serves as a processing unit. Establishing the brain state’s central role in affect processing, however, requires that it predicts multiple independent measures of affect. We employed MVPA-based regression to predict the valence and arousal properties of visual stimuli sampled from the International Affective Picture System (IAPS) along with the corollary skin conductance response (SCR) for demographically diverse healthy human participants (n = 19). We found that brain states significantly predicted the normative valence and arousal scores of the stimuli as well as the attendant individual SCRs. In contrast, SCRs significantly predicted arousal only. The prediction effect size of the brain state was more than three times greater than that of SCR. Moreover, neuroanatomical analysis of the regression parameters found remarkable agreement with regions long-established by fMRI univariate analyses in the emotion processing literature. Finally, geometric analysis of these parameters also found that the neuroanatomical encodings of valence and arousal are orthogonal as originally posited by the circumplex model of dimensional emotion.
more »
« less
Regression of multiple conversation aspects using dyadic physiological measurements
Dyadic physiological responses are correlated with the quality of interpersonal processes – for example, the degree of “connectedness” in education and mental health counseling. Pattern recognition algorithms could be applied to such dyadic responses to identify the states of specific dyads, but such pattern recognition has primarily focused on classification. This paper instead uses regression algorithms to estimate three conversation aspects (valence, arousal, balance) from heart rate, skin conductance, respiration, and skin temperature. Data were collected from 35 dyads who engaged in 20 minutes of conversation, divided into 10 two-minute intervals. Each interval was rated with regard to conversation valence, arousal, and balance by an observer. When regression algorithms (support vector machines and Gaussian process regression) were trained on other data from the same dyad, they were able to estimate valence, arousal and balance with lower errors than a simple baseline estimator. However, when algorithms were trained on data from other dyads, errors were not lower than those of the baseline estimator. Overall, results indicate that, as long as training data from the same dyad are available, autonomic nervous system responses can be combined with regression algorithms to estimate multiple dyadic conversation aspects with some accuracy. This has applications in education and mental health counseling, though fundamental issues remain to be addressed before the technology is used in practice.
more »
« less
- Award ID(s):
- 2151464
- PAR ID:
- 10547229
- Publisher / Repository:
- IEEE
- Date Published:
- Format(s):
- Medium: X
- Location:
- Orlando, FL
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Patterns of estimated neural activity derived from resting state functional magnetic resonance imaging (rs-fMRI) have been shown to predict a wide range of cognitive and behavioral outcomes in both normative and clinical populations. Yet, without links to established cognitive processes, the functional brain states associated with the resting brain will remain unexplained, and potentially confounded, markers of individual differences. In this work we demonstrate the application of multivoxel pattern classifiers (MVPCs) to predict the valence and arousal properties of spontaneous affect processing in the task-non-engaged resting state. rs-fMRI data were acquired from subjects that were held out from a subject set that underwent image-based affect induction concurrent with fMRI to train the MVPCs. We also validated these affective predictions against a well-established, independent measure of autonomic arousal, skin conductance response. These findings suggest a new neuroimaging methodology for resting state analysis in which models, trained on cognition-specific task-based fMRI acquired from well-matched cohorts, capably predict hidden cognitive processes operating within the resting brain.more » « less
-
null (Ed.)Background and Context: Researchers and practitioners have begun to incorporate collaboration in programming because of its reported instructional and professional benefits. However, younger students need guidance on how to collaborate in environments that require substantial interpersonal interaction and negotiation. Previous research indicates that feedback fosters students’ productive collaboration. Objective: This study employs an intervention to explore the role instructor-directed feedback plays on elementary students’ dyadic collaboration during 2-computer pair programming. Method: We used a multi-study design, collecting video data on students’ dyadic collaboration. Study 1 qualitatively explored dyadic collaboration by coding video transcripts of four dyads which guided the design of Study 2 that examined conversation of six dyads using MANOVA and non-parametric tests. Findings: Result from Study 2 showed that students receiving feed- back used productive conversation categories significantly higher than the control condition in the sample group considered. Results are discussed in terms of group differences in specific conversation categories. Implications: Our study highlights ways to support students in pair programming contexts so that they can maximize the benefits afforded through these experiences.more » « less
-
Trackers for activity and physical fitness have become ubiquitous. Although recent work has demonstrated significant relationships between mental effort and physiological data such as skin temperature, heart rate, and electrodermal activity, we have yet to demonstrate their efficacy for the forecasting of mental effort such that a useful mental effort tracker can be developed. Given prior difficulty in extracting relationships between mental effort and physiological responses that are repeatable across individuals, we make the case that fusing self-report measures with physiological data within an internet or smartphone application may provide an effective method for training a useful mental effort tracking system. In this case study, we utilized over 90 h of data from a single participant over the course of a college semester. By fusing the participant’s self-reported mental effort in different activities over the course of the semester with concurrent physiological data collected with the Empatica E4 wearable sensor, we explored questions around how much data were needed to train such a device, and which types of machine-learning algorithms worked best. We concluded that although baseline models such as logistic regression and Markov models provided useful explanatory information on how the student’s physiology changed with mental effort, deep-learning algorithms were able to generate accurate predictions using the first 28 h of data for training. A system that combines long short-term memory and convolutional neural networks is recommended in order to generate smooth predictions while also being able to capture transitions in mental effort when they occur in the individual using the device.more » « less
-
Stressful conversation is a frequently occurring stressor in our daily life. Stressors not only adversely affect our physical and mental health but also our relationships with family, friends, and coworkers. In this paper, we present a model to automatically detect stressful conversations using wearable physiological and inertial sensors. We conducted a lab and a field study with cohabiting couples to collect ecologically valid sensor data with temporally-precise labels of stressors. We introduce the concept of stress cycles, i.e., the physiological arousal and recovery, within a stress event. We identify several novel features from stress cycles and show that they exhibit distinguishing patterns during stressful conversations when compared to physiological response due to other stressors. We observe that hand gestures also show a distinct pattern when stress occurs due to stressful conversations. We train and test our model using field data collected from 38 participants. Our model can determine whether a detected stress event is due to a stressful conversation with an F1-score of 0.83, using features obtained from only one stress cycle, facilitating intervention delivery within 3.9 minutes since the start of a stressful conversation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
