skip to main content


Title: hallenges for using Representational Similarity Analysis to Infer Cognitive Processes: A Demonstration from Interactive Activation Models of Word Reading.
Representational Similarity Analysis (RSA) is a powerful tool for linking brain activity patterns to cognitive processes via similarity, allowing researchers to identify the neural substrates of different cognitive levels of representation. However, the ability to map between levels of representation and brain activity using similarity depends on underlying assumptions about the dynamics of cognitive processing. To demonstrate this point, we present three toy models that make different assumptions about the interactivity within the reading system, (1) discrete, feedforward, (2) cascading, feedforward and (3) fully interactive. With the temporal resolution of fMRI, only the discrete, feedforward model provides a straightforward mapping between activation similarity and level of representation. These simulations indicate the need for a cautious interpretation of RSA results, especially with processes that are highly interactive and with neuroimaging methods that have low temporal resolution. The study further suggests a role for fully-fleshed out computational models in RSA analyses.  more » « less
Award ID(s):
1752751
NSF-PAR ID:
10321541
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the Annual Conference of the Cognitive Science Society
ISSN:
1069-7977
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Listening to pleasurable music is known to engage the brain’s reward system. This has motivated many cognitive-behavioral interventions for healthy aging, but little is known about the effects of music-based intervention (MBI) on activity and connectivity of the brain’s auditory and reward systems. Here we show preliminary evidence that brain network connectivity can change after receptive MBI in cognitively unimpaired older adults. Using a combination of whole-brain regression, seed-based connectivity analysis, and representational similarity analysis (RSA), we examined fMRI responses during music listening in older adults before and after an 8-week personalized MBI. Participants rated self-selected and researcher-selected musical excerpts on liking and familiarity. Parametric effects of liking, familiarity, and selection showed simultaneous activation in auditory, reward, and default mode network (DMN) areas. Functional connectivity within and between auditory and reward networks was modulated by participant liking and familiarity ratings. RSA showed significant representations of selection and novelty at both time-points, and an increase in striatal representation of musical stimuli following intervention. An exploratory seed-based connectivity analysis comparing pre- and post-intervention showed significant increase in functional connectivity between auditory regions and medial prefrontal cortex (mPFC). Taken together, results show how regular music listening can provide an auditory channel towards the mPFC, thus offering a potential neural mechanism for MBI supporting healthy aging.

     
    more » « less
  2. Even though human experience unfolds continuously in time, it is not strictly linear; instead, it entails cascading processes building hierarchical cognitive structures. For instance, during speech perception, humans transform a continuously varying acoustic signal into phonemes, words, and meaning, and these levels all have distinct but interdependent temporal structures. Time-lagged regression usingtemporal response functions (TRFs) has recently emerged as a promising tool for disentangling electrophysiological brain responses related to such complex models of perception. Here, we introduce the Eelbrain Python toolkit, which makes this kind of analysis easy and accessible. We demonstrate its use, using continuous speech as a sample paradigm, with a freely available EEG dataset of audiobook listening. A companion GitHub repository provides the complete source code for the analysis, from raw data to group-level statistics. More generally, we advocate a hypothesis-driven approach in which the experimenter specifies a hierarchy of time-continuous representations that are hypothesized to have contributed to brain responses, and uses those as predictor variables for the electrophysiological signal. This is analogous to a multiple regression problem, but with the addition of a time dimension. TRF analysis decomposes the brain signal into distinct responses associated with the different predictor variables by estimating a multivariate TRF (mTRF), quantifying the influence of each predictor on brain responses as a function of time(-lags). This allows asking two questions about the predictor variables: (1) Is there a significant neural representation corresponding to this predictor variable? And if so, (2) what are the temporal characteristics of the neural response associated with it? Thus, different predictor variables can be systematically combined and evaluated to jointly model neural processing at multiple hierarchical levels. We discuss applications of this approach, including the potential for linking algorithmic/representational theories at different cognitive levels to brain responses through computational models with appropriate linking hypotheses. 
    more » « less
  3. Understanding the intrinsic patterns of human brain is important to make inferences about the mind and brain-behavior association. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity without the effect of vascular confounds. The blood oxygenated level-dependent (BOLD) signal of functional MRI (fMRI) reveals the spatial and temporal brain activity across different brain regions. However, it is unclear how to associate the high temporal resolution Electrophysiological measures with high spatial resolution fMRI signals. Here, we present a novel interpretable model for coupling the structure and function activity of brain based on heterogeneous contrastive graph representation. The proposed method is able to link manifest variables of the brain (i.e. MEG, MRI, fMRI and behavior performance) and quantify the intrinsic coupling strength of different modal signals. The proposed method learns the heterogeneous node and graph representations by contrasting the structural and temporal views through the mind to multimodal brain data. The first experiment with 1200 subjects from Human connectome Project (HCP) shows that the proposed method outperforms the existing approaches in predicting individual gender and enabling the location of the importance of brain regions with sex difference. The second experiment associates the structure and temporal views between the low-level sensory regions and high-level cognitive ones. The experimental results demonstrate that the dependence of structural and temporal views varied spatially through different modal variants. The proposed method enables the heterogeneous biomarkers explanation for different brain measurements. 
    more » « less
  4. Abstract

    A cognitive map is an internal representation of the external world that guides flexible behavior in a complex environment. Cognitive map theory assumes that relationships between entities can be organized using Euclidean‐based coordinates. Previous studies revealed that cognitive map theory can also be generalized to inferences about abstract spaces, such as social spaces. However, it is still unclear whether humans can construct a cognitive map by combining relational knowledge between discrete entities with multiple abstract dimensions in nonsocial spaces. Here we asked subjects to learn to navigate a novel object space defined by two feature dimensions, price and abstraction. The subjects first learned the rank relationships between objects in each feature dimension and then completed a transitive inferences task. We recorded brain activity using functional magnetic resonance imaging (fMRI) while they performed the transitive inference task. By analyzing the behavioral data, we found that the Euclidean distance between objects had a significant effect on response time (RT). The longer the one‐dimensional rank distance and two‐dimensional (2D) Euclidean distance between objects the shorter the RT. The task‐fMRI data were analyzed using both univariate analysis and representational similarity analysis. We found that the hippocampus, entorhinal cortex, and medial orbitofrontal cortex were able to represent the Euclidean distance between objects in 2D space. Our findings suggest that relationship inferences between discrete objects can be made in a 2D nonsocial space and that the neural basis of this inference is related to cognitive maps.

     
    more » « less
  5. Abstract

    One of the fundamental goals in neuroscience is to determine how the brain processes information and ultimately controls the execution of complex behaviors. Over the past four decades, there has been a steady growth in our knowledge of the morphological and functional diversity of neurons, the building blocks of the brain. These cells clearly differ not only for their anatomy and ion channel distribution, but also for the type, strength, location, and temporal pattern of activity of the many synaptic inputs they receive. Compartmental modeling programs like NEURON have become widely used in the neuroscience community to address a broad range of research questions, including how neurons integrate synaptic inputs and propagate information through complex neural networks. One of the main strengths of NEURON is its ability to incorporate user-defined information about the realistic morphology and biophysical properties of different cell types. Although the graphical user interface of the program can be used to run initial exploratory simulations, introducing a stochastic representation of synaptic weights, locations and activation times typically requires users to develop their own codes, a task that can be overwhelming for some beginner users. Here we describe NRN-EZ, an interactive application that allows users to specify complex patterns of synaptic input activity that can be integrated as part of NEURON simulations. Through its graphical user interface, NRN-EZ aims to ease the learning curve to run computational models in NEURON, for users that do not necessarily have a computer science background.

     
    more » « less