An official website of the United States government
Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.
more »
« less
An fMRI dataset in response to “The Grand Budapest Hotel”, a socially-rich, naturalistic movie
Abstract Naturalistic stimuli evoke strong, consistent, and information-rich patterns of brain activity, and engage large extents of the human brain. They allow researchers to compare highly similar brain responses across subjects, and to study how complex representations are encoded in brain activity. Here, we describe and share a dataset where 25 subjects watched part of the feature film “The Grand Budapest Hotel” by Wes Anderson. The movie has a large cast with many famous actors. Throughout the story, the camera shots highlight faces and expressions, which are fundamental to understand the complex narrative of the movie. This movie was chosen to sample brain activity specifically related to social interactions and face processing. This dataset provides researchers with fMRI data that can be used to explore social cognitive processes and face processing, adding to the existing neuroimaging datasets that sample brain activity with naturalistic movies.
more »
« less
- Award ID(s):
- 1835200
- PAR ID:
- 10294407
- Date Published:
- Journal Name:
- Scientific Data
- Volume:
- 7
- Issue:
- 1
- ISSN:
- 2052-4463
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract We present a multimodal dataset of intracranial recordings, fMRI, and eye tracking in 20 participants during movie watching. Recordings consist of single neurons, local field potential, and intracranial EEG activity acquired from depth electrodes targeting the amygdala, hippocampus, and medial frontal cortex implanted for monitoring of epileptic seizures. Participants watched an 8-min long excerpt from the video “Bang! You’re Dead” and performed a recognition memory test for movie content. 3 T fMRI activity was recorded prior to surgery in 11 of these participants while performing the same task. This NWB- and BIDS-formatted dataset includes spike times, field potential activity, behavior, eye tracking, electrode locations, demographics, and functional and structural MRI scans. For technical validation, we provide signal quality metrics, assess eye tracking quality, behavior, the tuning of cells and high-frequency broadband power field potentials to familiarity and event boundaries, and show brain-wide inter-subject correlations for fMRI. This dataset will facilitate the investigation of brain activity during movie watching, recognition memory, and the neural basis of the fMRI-BOLD signal.more » « less
-
Abstract Recent analyses have found waves of neural activity traveling across entire visual cortical areas in awake animals. These traveling waves modulate excitability of local networks and perceptual sensitivity. The general computational role for these spatiotemporal patterns in the visual system, however, remains unclear. Here, we hypothesize that traveling waves endow the brain with the capacity to predict complex and naturalistic visual inputs. We present a new network model whose connections can be rapidly and efficiently trained to predict natural movies. After training, a few input frames from a movie trigger complex wave patterns that drive accurate predictions many frames into the future, solely from the network’s connections. When the recurrent connections that drive waves are randomly shuffled, both traveling waves and the ability to predict are eliminated. These results show traveling waves could play an essential computational role in the visual system by embedding continuous spatiotemporal structures over spatial maps.more » « less
-
Electrical signatures characteristic of complex neurological activity and neuropsychiatric disease are embedded in electroencephalography (EEG) signal data. To firmly establish new correlations between these brain electrical pulses and cognition, behavior, and disorders, researchers must achieve adequate statistical power to validate and mitigate uncertainties in their findings. This necessitates the usage of extensive studies involving large volumes of raw EEG data files from multiple subjects, data which must be preprocessed before conducting further analysis. While conventional processing and analysis of these raw data have been performed using isolated physical lab environments and stovepiped applications, there is a growing necessity for processing and analysis solutions that enable distributed processing of large data collections. This study presents a novel microservices approach as an alternative and complementary solution for retrieving and preprocessing EEG signal data. The approach leverages serverless technologies to deliver a highly scalable solution for processing massive amounts of EEG data. Deployed within a public cloud environment, we assess the efficacy of this method when employing various container orchestration configurations. This work demonstrates the capability for substantial enhancements in processing speeds, particularly when dealing with extensive EEG datasets.more » « less
-
null (Ed.)Abstract The bulk of social neuroscience takes a ‘stimulus-brain’ approach, typically comparing brain responses to different types of social stimuli, but most of the time in the absence of direct social interaction. Over the last two decades, a growing number of researchers have adopted a ‘brain-to-brain’ approach, exploring similarities between brain patterns across participants as a novel way to gain insight into the social brain. This methodological shift has facilitated the introduction of naturalistic social stimuli into the study design (e.g. movies) and, crucially, has spurred the development of new tools to directly study social interaction, both in controlled experimental settings and in more ecologically valid environments. Specifically, ‘hyperscanning’ setups, which allow the simultaneous recording of brain activity from two or more individuals during social tasks, has gained popularity in recent years. However, currently, there is no agreed-upon approach to carry out such ‘inter-brain connectivity analysis’, resulting in a scattered landscape of analysis techniques. To accommodate a growing demand to standardize analysis approaches in this fast-growing research field, we have developed Hyperscanning Python Pipeline, a comprehensive and easy open-source software package that allows (social) neuroscientists to carry-out and to interpret inter-brain connectivity analyses.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1038/s41597-020-00735-4
