skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • hallenges for using Representational Similarity Analysis to Infer Cognitive Processes: A Demonstration from Interactive Activation Models of Word Reading.
Title: hallenges for using Representational Similarity Analysis to Infer Cognitive Processes: A Demonstration from Interactive Activation Models of Word Reading.
Representational Similarity Analysis (RSA) is a powerful tool for linking brain activity patterns to cognitive processes via similarity, allowing researchers to identify the neural substrates of different cognitive levels of representation. However, the ability to map between levels of representation and brain activity using similarity depends on underlying assumptions about the dynamics of cognitive processing. To demonstrate this point, we present three toy models that make different assumptions about the interactivity within the reading system, (1) discrete, feedforward, (2) cascading, feedforward and (3) fully interactive. With the temporal resolution of fMRI, only the discrete, feedforward model provides a straightforward mapping between activation similarity and level of representation. These simulations indicate the need for a cautious interpretation of RSA results, especially with processes that are highly interactive and with neuroimaging methods that have low temporal resolution. The study further suggests a role for fully-fleshed out computational models in RSA analyses.  more » « less
Award ID(s):
1752751
PAR ID:
10321541
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the Annual Conference of the Cognitive Science Society
ISSN:
1069-7977
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Design Computing and Cognition’24)
    Gero, JS (Ed.)
    Recent developments in using Large Language Models (LLMs) to predict and align with neural representations of language can be applied to achieving a future vision of design tools that enable detection and reconstruction of designers’ mental representations of ideas. Prior work has largely explored this relationship during passive language tasks only, e.g., reading or listening. In this work, the relationship between brain activation data (functional imaging, fMRI) during appropriate and novel word association generation and LLM (Llama-2 7b) word representations is tested using Representational Similarity Analysis (RSA). Findings suggest that LLM word representations align with brain activity captured during novel word association, but not when forming appropriate associates. Association formation is one cognitive process central to design. By demonstrating that brain activity during this task can align with LLM word representations, insights from this work encourage further investigation into this relationship during more complex design ideation processes. 
    more » « less
  2. ; ; ; (, Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Lecture Notes in Computer Science.)
    Understanding the intrinsic patterns of human brain is important to make inferences about the mind and brain-behavior association. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity without the effect of vascular confounds. The blood oxygenated level-dependent (BOLD) signal of functional MRI (fMRI) reveals the spatial and temporal brain activity across different brain regions. However, it is unclear how to associate the high temporal resolution Electrophysiological measures with high spatial resolution fMRI signals. Here, we present a novel interpretable model for coupling the structure and function activity of brain based on heterogeneous contrastive graph representation. The proposed method is able to link manifest variables of the brain (i.e. MEG, MRI, fMRI and behavior performance) and quantify the intrinsic coupling strength of different modal signals. The proposed method learns the heterogeneous node and graph representations by contrasting the structural and temporal views through the mind to multimodal brain data. The first experiment with 1200 subjects from Human connectome Project (HCP) shows that the proposed method outperforms the existing approaches in predicting individual gender and enabling the location of the importance of brain regions with sex difference. The second experiment associates the structure and temporal views between the low-level sensory regions and high-level cognitive ones. The experimental results demonstrate that the dependence of structural and temporal views varied spatially through different modal variants. The proposed method enables the heterogeneous biomarkers explanation for different brain measurements. 
    more » « less
  3. ; ; ; ; ; ; ; (, Scientific Reports)
    Abstract Listening to pleasurable music is known to engage the brain’s reward system. This has motivated many cognitive-behavioral interventions for healthy aging, but little is known about the effects of music-based intervention (MBI) on activity and connectivity of the brain’s auditory and reward systems. Here we show preliminary evidence that brain network connectivity can change after receptive MBI in cognitively unimpaired older adults. Using a combination of whole-brain regression, seed-based connectivity analysis, and representational similarity analysis (RSA), we examined fMRI responses during music listening in older adults before and after an 8-week personalized MBI. Participants rated self-selected and researcher-selected musical excerpts on liking and familiarity. Parametric effects of liking, familiarity, and selection showed simultaneous activation in auditory, reward, and default mode network (DMN) areas. Functional connectivity within and between auditory and reward networks was modulated by participant liking and familiarity ratings. RSA showed significant representations of selection and novelty at both time-points, and an increase in striatal representation of musical stimuli following intervention. An exploratory seed-based connectivity analysis comparing pre- and post-intervention showed significant increase in functional connectivity between auditory regions and medial prefrontal cortex (mPFC). Taken together, results show how regular music listening can provide an auditory channel towards the mPFC, thus offering a potential neural mechanism for MBI supporting healthy aging. 
    more » « less
  4. ; ; ; ; ; ; ; (, eLife)
    Even though human experience unfolds continuously in time, it is not strictly linear; instead, it entails cascading processes building hierarchical cognitive structures. For instance, during speech perception, humans transform a continuously varying acoustic signal into phonemes, words, and meaning, and these levels all have distinct but interdependent temporal structures. Time-lagged regression usingtemporal response functions (TRFs) has recently emerged as a promising tool for disentangling electrophysiological brain responses related to such complex models of perception. Here, we introduce the Eelbrain Python toolkit, which makes this kind of analysis easy and accessible. We demonstrate its use, using continuous speech as a sample paradigm, with a freely available EEG dataset of audiobook listening. A companion GitHub repository provides the complete source code for the analysis, from raw data to group-level statistics. More generally, we advocate a hypothesis-driven approach in which the experimenter specifies a hierarchy of time-continuous representations that are hypothesized to have contributed to brain responses, and uses those as predictor variables for the electrophysiological signal. This is analogous to a multiple regression problem, but with the addition of a time dimension. TRF analysis decomposes the brain signal into distinct responses associated with the different predictor variables by estimating a multivariate TRF (mTRF), quantifying the influence of each predictor on brain responses as a function of time(-lags). This allows asking two questions about the predictor variables: (1) Is there a significant neural representation corresponding to this predictor variable? And if so, (2) what are the temporal characteristics of the neural response associated with it? Thus, different predictor variables can be systematically combined and evaluated to jointly model neural processing at multiple hierarchical levels. We discuss applications of this approach, including the potential for linking algorithmic/representational theories at different cognitive levels to brain responses through computational models with appropriate linking hypotheses. 
    more » « less
  5. ; ; (, IEEE)
    Identifying spatiotemporal differences in brain functional dynamics corresponding to two tasks is critical for understanding how specific neural processes contribute to distinct tasks or cognitive functions. Traditional methods rely on imposing assumptions and limits on the location and timing of activities, while machine-learning-based methods generally lack offering interpretable insights. This highlights the need for new data-driven approaches to capture spatial and temporal differences in brain activity between two tasks, while also providing interpretable explanations of the neural processes underlying these differences. In this work, we formulate the problem of finding the spatial and temporal differences in the dynamics of brain function corresponding to two motor imagery (MI) tasks (left hand movement vs right hand movement) as a discriminative discrete basis problem (DDBP). We apply the data-driven asymmetric discriminative associative algorithm (ADASSO) to EEG data collected during these tasks to uncover the key functional components of the brain’s functional dynamics that differentiate between them. Results suggest that hand movements are strongly associated with high confidence activation in the motor cortex, verifying the effectiveness of the ADASSO algorithm in identifying the location and timing of cortical activities that distinguish between the two task classes. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email