More Like this

1. Emotional Responses to Naturalistic and AI-generated Affective Pictures: A Systematic Comparison

   https://doi.org/10.1101/2025.11.03.686276  

   Gilbert, Faith; Yang, Fan; Smith, Ethan; Gardy, Sarah; Farkas, Andrew; Ding, Mingzhou; Fang, Ruogu; Keil, Andreas (November 2025, bioRxiv (accepted by Emotion))

   Abstract Pictures depicting naturalistic scenes are widely used in studies of human emotion. However, the practical use of affective pictures is limited by several factors, including the difficulty of obtaining content-diverse, high-quality, openly accessible, and standardized stimuli that are necessary for specific research questions. The use of artificially generated (AI) pictures could address this limitation, but it is unclear if AI-generated pictures evoke reliable emotional responses. The present study sought to address these challenges by comparing emotional responses to AI-generated pictures with responses to original, standardized, pictures. In two study iterations, standardized pictures containing pleasant, neutral, and unpleasant content were selected from the International Affective Picture System (IAPS) and other sources. Then, a matched AI-counterpart was created for each original picture using generative deep neural networks. A total of 109 participants viewed the picture sets while pupil diameter and electroencephalogram (EEG) were recorded. Evaluative ratings of hedonic valence and emotional arousal were also collected. For both AI and original exemplars, pictures depicting emotional content elicited stronger responses than neutral content for ratings and EEG-derived variables, with weaker effect sizes for the AI-generated pictures. Furthermore, picture-level analyses found that ratings and EEG measures were strongly correlated between matched AI and original pictures. Pupil data also showed the expected content effects in study iteration 2, but not in iteration 1. Together, this initial study suggests that AI-generated picture sets can effectively elicit well-established self-reported affect and physiological responses, presenting a promising avenue for future studies of human emotion. 

   more » « less
2. Role of the Pulvinar in Visual Affective Scene Processing

   https://doi.org/10.1167/jov.22.14.3415  

   Cui, Lihan; Liang, Yun; Bo, Ke; Keil, Andreas; Ding, Mingzhou (December 2022, Journal of Vision)

   The pulvinar is the largest nucleus of the thalamus. The proposed functions of the pulvinar range from working memory to attention to emotion. In this study, we examined the role of the pulvinar in visual affective scene processing by analyzing two independent fMRI datasets. In one dataset, simultaneous EEG-fMRI data were recorded from 20 human participants viewing pleasant, unpleasant, and neutral scenes from the International Affective Picture System (IAPS). In the other dataset, fMRI data were recorded from 30 human participants performing an emotion reappraisal task, in which they were cued to anticipate and then viewed unpleasant and neutral scenes from the IAPS. Analyzing single-trial BOLD responses using machine learning and AI-inspired decoding techniques, we found that linear machine learning methods such as SVM were not able to consistently detect the differences in neural representations of different categories of affective scenes in the pulvinar, whereas a deep neural network (DNN) based model was able to consistently do so with significantly above-chance decoding accuracies. In particular, for the unpleasant-vs-neutral decoding analysis, the DNN model was able to yield a decoding accuracy of 58.6% for the first dataset and 64.5% for the second dataset. A weight map analysis on both datasets further revealed that the medial pulvinar, especially the right medial pulvinar, is the most important contributor to the DNN decoding performance. These results demonstrated that the pulvinar contributes to the processing of visual affective scenes and AI-inspired techniques can detect functional relationships not readily detected by the conventional machine learning methods. 

   more » « less
3. The Origin of Affect-specific Neural Representations of Emotional Scenes in Early Visual Cortex

   https://doi.org/10.1167/jov.25.9.1989  

   Chen, Yujun; Keil, Andreas; Ding, Mingzhou (July 2025, Journal of Vision)

   Recent fMRI studies in human subjects have found affect-specific neural representations of emotional scenes in early visual cortex. The origin of these representations is debated. One group of hypotheses suggests that these representations result from reentrant feedback from anterior emotion-modulating structures (e.g., the amygdala), whereas another group of hypotheses states that sensory cortex, including retinotopic visual cortex, may itself code for the emotional qualities of visual stimuli, without the necessity for feedback processing. We examined this problem by employing a neural encoding model that can generate synthetic fMRI responses to natural images in early visual cortex. The model works by linearly mapping features extracted by convolutional neural networks onto voxel-wise BOLD responses in different visual areas and is trained on the Natural Scenes Dataset. Dividing the images in the International Affective Picture System into three broad categories: pleasant, neutral and unpleasant, we found that in early visual cortex, the neural patterns evoked by the emotional images cannot be decoded from that evoked by the neutral images, in contrast with the findings from recent fMRI studies in human subjects. Because the model-generated responses are free from emotion-modulated reentrant feedback, this finding can be seen as lending support to the reentry hypothesis. Interestingly, when face stimuli from the AffectNet were shown to the model, the neural patterns evoked by emotional faces in early visual cortex can be significantly decoded from that evoked by neutral faces, suggesting that the early visual cortex may contribute differently to the emotional processing of faces versus scenes. 

   more » « less
4. Rhythmic Sampling and Competition of Target and Distractor in a Motion Detection Task

   https://doi.org/10.7554/eLife.106140.1  

   Xiong, Changhao; Petro, Nathan M; Bo, Ke; Cui, Lihan; Keil, Andreas; Ding, Mingzhou (April 2025, eLife Sciences Publications Ltd.)

   Abstract It has been suggested that the visual system samples attended information rhythmically. Does rhythmic sampling also apply to distracting information? How do attended information and distracting information compete temporally for neural representations? We recorded electroencephalography from participants who detected instances of coherent motion in a random dot kinematogram (RDK; the target stimulus), overlayed on different categories (pleasant, neutral, and unpleasant) of affective images from the International Affective System (IAPS) (the distractor). The moving dots were flickered at 4.29 Hz whereas the IAPS pictures were flickered at 6 Hz. The time course of spectral power at 4.29 Hz (dot response) was taken to index the temporal dynamics of target processing. The spatial pattern of the power at 6 Hz was similarly extracted and subjected to a MVPA decoding analysis to index the temporal dynamics of processing pleasant, neutral, or unpleasant distractor pictures. We found that (1) both target processing and distractor processing exhibited rhythmicity at ∼1 Hz and (2) the phase difference between the two rhythmic time courses were related to task performance, i.e., relative phase closer to π predicted a higher rate of coherent motion detection whereas relative phase closer to 0 predicted a lower rate of coherent motion detection. These results suggest that (1) in a target-distractor scenario, both attended and distracting information were sampled rhythmically and (2) the more target sampling and distractor sampling were separated in time within a sampling cycle, the less distraction effects were observed, both at the neural and the behavioral level. 

   more » « less
5. Rhythmic sampling and competition of target and distractor in a motion detection task

   https://doi.org/10.7554/eLife.106140.3  

   Xiong, Changhao; Petro, Nathan M; Bo, Ke; Cui, Lihan; Keil, Andreas; Ding, Mingzhou (November 2025, eLife)

   It has been suggested that the visual system samples attended information rhythmically. Does rhythmic sampling also apply to distracting information? How do attended information and distracting information compete temporally for neural representations? We recorded electroencephalography (EEG) from participants who detected instances of coherent motion in a random-dot kinematogram (RDK; the target), overlayed on different categories (pleasant, neutral, and unpleasant) of affective images from the International Affective Picture System (IAPS) (the distractor). The moving dots were flickered at 4.29 Hz, whereas the IAPS pictures were flickered at 6 Hz. The time course of EEG spectral power at 4.29 Hz was taken to index the temporal dynamics of target processing. The spatial pattern of the EEG spectral power at 6 Hz was similarly extracted and subjected to a moving-window MVPA decoding analysis to index the temporal dynamics of processing pleasant, neutral, or unpleasant distractor pictures. We found that (1) both target processing and distractor processing exhibited rhythmicity at ~1 Hz and (2) the phase difference between the two rhythmic time courses was related to task performance, i.e., relative phase closer to π predicted a higher rate of coherent motion detection whereas relative phase closer to 0 predicted a lower rate of coherent motion detection. These results suggest that (1) in a target-distractor scenario, both attended and distracting information were sampled rhythmically and (2) the more target sampling and distractor sampling were separated in time within a sampling cycle, the less distraction effects were observed, both at the neural and the behavioral level. 

   more » « less