An official website of the United States government
Artificial neural networks are being proposed as models of parts of the brain. The networks are compared to recordings of biological neurons, and good performance in reproducing neural responses is considered to support the model’s validity. A key question is how much this system identification approach tells us about brain computation. Does it validate one model architecture over another? We evaluate the most commonly used comparison techniques, such as a linear encoding model and centered kernel alignment, to correctly identify a model by replacing brain recordings with known ground truth models. System identification performance is quite variable; it also depends significantly on factors independent of the ground truth architecture, such as stimuli images. In addition, we show the limitations of using functional similarity scores in identifying higher-level architectural motifs.
more »
« less
System identification of neural systems: If we got it right, would we know?
Various artificial neural networks developed by engineers have been evaluated as models of the brain, such as the ventral stream in the primate visual cortex. After being trained on large datasets, the network outputs are compared to recordings of biological neurons. Good performance in reproducing neural responses is taken as validation for the model. This system identification approach is different from the traditional ways to test theories and associated models in the natural sciences. Furthermore, it lacks a clear foundation in terms of theory and empirical validation. Here we begin characterizing some of these emerging approaches: what do they tell us? To address this question, we benchmark their ability to correctly identify a model by replacing the brain recordings with recordings from a known ground truth model. We evaluate commonly used identification techniques such as neural regression (linear regression on a population of model units) and centered kernel alignment (CKA). Even in the setting where the correct model is among the candidates, we find that the performance of these approaches at system identification is quite variable; it also depends significantly on factors independent of the ground truth architecture, such as scoring function and dataset.
more »
« less
- Award ID(s):
- 2134108
- PAR ID:
- 10565468
- Publisher / Repository:
- Center for Brains, Minds and Machines (CBMM)
- Date Published:
- Format(s):
- Medium: X
- Institution:
- Massachusetts Institute of Technology
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Cognitive tasks engage multiple brain regions. Studying how these regions interact is key to understand the neural bases of cognition. Standard approaches to model the interactions between brain regions rely on univariate statistical dependence. However, newly developed methods can capture multivariate dependence. Multivariate pattern dependence (MVPD) is a powerful and flexible approach that trains and tests multivariate models of the interactions between brain regions using independent data. In this article, we introduce PyMVPD: an open source toolbox for multivariate pattern dependence. The toolbox includes linear regression models and artificial neural network models of the interactions between regions. It is designed to be easily customizable. We demonstrate example applications of PyMVPD using well-studied seed regions such as the fusiform face area (FFA) and the parahippocampal place area (PPA). Next, we compare the performance of different model architectures. Overall, artificial neural networks outperform linear regression. Importantly, the best performing architecture is region-dependent: MVPD subdivides cortex in distinct, contiguous regions whose interaction with FFA and PPA is best captured by different models.more » « less
-
Human decision-making involves the coordinated activity of multiple brain areas, acting in concert, to enable humans to make choices. Most decisions are carried out under conditions of uncertainty, where the desired outcome may not be achieved if the wrong decision is made. In these cases, humans deliberate before making a choice. The neural dynamics underlying deliberation are unknown and intracranial recordings in clinical settings present a unique opportunity to record high temporal resolution electrophysiological data from many (hundreds) brain locations during behavior. Combined with dynamic systems modeling, these allow identification of latent brain states that describe the neural dynamics during decision-making, providing insight into these neural dynamics and computations. Results show that the neural dynamics underlying risky decisions, but not decisions without risk, converge to separate subspaces depending on the subject’s preferred choice and that the degree of overlap between these subspaces declines as choice approaches, suggesting a network level representation of evidence accumulation. These results bridge the gap between regression analyses and data driven models of latent states and suggest that during risky decisions, deliberation and evidence accumulation toward a final decision are represented by the same neural dynamics, providing novel insights into the neural computations underlying human choice.more » « less
-
Recent advancements in two-photon calcium imaging have enabled scientists to record the activity of thousands of neurons with cellular resolution. This scope of data collection is crucial to understanding the next generation of neuroscience questions, but analyzing these large recordings requires automated methods for neuron segmentation. Supervised methods for neuron segmentation achieve state of-the-art accuracy and speed but currently require large amounts of manually generated ground truth training labels. We reduced the required number of training labels by designing a semi-supervised pipeline. Our pipeline used neural network ensembling to generate pseudolabels to train a single shallow U-Net. We tested our method on three publicly available datasets and compared our performance to three widely used segmentation methods. Our method outperformed other methods when trained on a small number of ground truth labels and could achieve state-of-the-art accuracy after training on approximately a quarter of the number of ground truth labels as supervised methods. When trained on many ground truth labels, our pipeline attained higher accuracy than that of state-of-the-art methods. Overall, our work will help researchers accurately process large neural recordings while minimizing the time and effort needed to generate manual labels.more » « less
-
Research in the area of internet-of-things, cyber physical- systems, and smart health often employ sensor systems at residences for continuous monitoring. Such research oriented residential monitoring systems (RRMSs) usually face two major challenges, long-term reliable operation management and validation of system functionality with minimal human effort. Targeting these two challenges, this paper describes a monitor of monitoring systems with ground-truth validation capabilities, M2G. It consists of two subsystems, the Monitor2 system and the Ground-truth validation system. The Monitor2 system encapsulates a flexible set of general-purpose components to monitor the operation and connectivity of heterogeneous sensor devices (e.g. smart watches, smart phones, microphones, beacons, etc.), a local base-station, as well as a cloud server. It provides a user-friendly interface and supports different types of RRMSs in various contexts. The system also features a ground truth validation system to support obtaining ground truth in the field. Additionally, customized alerts can be sent to remote administrators and other personnel to report any dysfunction or inaccuracy of the system in real time. M2G is applied to three very different case studies: the M2FED system which monitors family eating dynamics, an in-home wireless sensing system for monitoring nighttime agitation, and the BESI system which monitors behavioral and environmental parameters to predict health events and to provide interventions. The results indicate that M2G is a comprehensive system that (i) requires small cost in time and effort to adapt to an existing RRMS, (ii) provides reliable data collection and reduction in data loss by detecting faults in real-time, and (iii) provides a convenient and timely ground truth validation facility.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Posted Content:
- The DOI is not currently available.
