In neuroscience, computational modeling is an effective way to gain insight into cortical mechanisms, yet the construction and analysis of large-scale network models—not to mention the extraction of underlying principles—are themselves challenging tasks, due to the absence of suitable analytical tools and the prohibitive costs of systematic numerical exploration of high-dimensional parameter spaces. In this paper, we propose a data-driven approach assisted by deep neural networks (DNN). The idea is to first discover certain input-output relations, and then to leverage this information and the superior computation speeds of the well-trained DNN to guide parameter searches and to deduce theoretical understanding. To illustrate this novel approach, we used as a test case a medium-size network of integrate-and-fire neurons intended to model local cortical circuits. With the help of an accurate yet extremely efficient DNN surrogate, we revealed the statistics of model responses, providing a detailed picture of model behavior. The information obtained is both general and of a fundamental nature, with direct application to neuroscience. Our results suggest that the methodology proposed can be scaled up to larger and more complex biological networks when used in conjunction with other techniques of biological modeling.
From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction
Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess the degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.
more »
« less
- Award ID(s):
- 1845166
- NSF-PAR ID:
- 10291282
- Date Published:
- Journal Name:
- Advances in neural information processing systems
- Volume:
- 32
- ISSN:
- 1049-5258
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
Convolutional neural networks (CNNs), a class of deep learning models, have experienced recent success in modeling sensory cortices and retinal circuits through optimizing performance on machine learning tasks, otherwise known as task optimization. Previous research has shown task-optimized CNNs to be capable of providing explanations as to why the retina efficiently encodes natural stimuli and how certain retinal cell types are involved in efficient encoding. In our work, we sought to use task-optimized CNNs as a means of explaining computational mechanisms responsible for motion-selective retinal circuits. We designed a biologically constrained CNN and optimized its performance on a motion-classification task. We drew inspiration from psychophysics, deep learning, and systems neuroscience literature to develop a toolbox of methods to reverse engineer the computational mechanisms learned in our model. Through reverse engineering our model, we proposed a computational mechanism in which direction-selective ganglion cells and starburst amacrine cells, both experimentally observed retinal cell types, emerge in our model to discriminate among moving stimuli. This emergence suggests that direction-selective circuits in the retina are ecologically designed to robustly discriminate among moving stimuli. Our results and methods also provide a framework for how to build more interpretable deep learning models and how to understand them.more » « less
-
null (Ed.)Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization 1 to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited.more » « less
-
Evolution has honed predatory skills in the natural world where localizing and intercepting fast-moving prey is required. The current generation of robotic systems mimics these biological systems using deep learning. High-speed processing of the camera frames using convolutional neural networks (CNN) (frame pipeline) on such constrained aerial edge-robots gets resource-limited. Adding more compute resources also eventually limits the throughput at the frame rate of the camera as frame-only traditional systems fail to capture the detailed temporal dynamics of the environment. Bio-inspired event cameras and spiking neural networks (SNN) provide an asynchronous sensor-processor pair (event pipeline) capturing the continuous temporal details of the scene for high-speed but lag in terms of accuracy. In this work, we propose a target localization system combining event-camera and SNN-based high-speed target estimation and frame-based camera and CNN-driven reliable object detection by fusing complementary spatio-temporal prowess of event and frame pipelines. One of our main contributions involves the design of an SNN filter that borrows from the neural mechanism for ego-motion cancelation in houseflies. It fuses the vestibular sensors with the vision to cancel the activity corresponding to the predator's self-motion. We also integrate the neuro-inspired multi-pipeline processing with task-optimized multi-neuronal pathway structure in primates and insects. The system is validated to outperform CNN-only processing using prey-predator drone simulations in realistic 3D virtual environments. The system is then demonstrated in a real-world multi-drone set-up with emulated event data. Subsequently, we use recorded actual sensory data from multi-camera and inertial measurement unit (IMU) assembly to show desired working while tolerating the realistic noise in vision and IMU sensors. We analyze the design space to identify optimal parameters for spiking neurons, CNN models, and for checking their effect on the performance metrics of the fused system. Finally, we map the throughput controlling SNN and fusion network on edge-compatible Zynq-7000 FPGA to show a potential 264 outputs per second even at constrained resource availability. This work may open new research directions by coupling multiple sensing and processing modalities inspired by discoveries in neuroscience to break fundamental trade-offs in frame-based computer vision 1 .more » « less
-
BACKGROUND Charles Darwin’s Descent of Man, and Selection in Relation to Sex tackled the two main controversies arising from the Origin of Species: the evolution of humans from animal ancestors and the evolution of sexual ornaments. Most of the book focuses on the latter, Darwin’s theory of sexual selection. Research since supports his conjecture that songs, perfumes, and intricate dances evolve because they help secure mating partners. Evidence is overwhelming for a primary role of both male and female mate choice in sexual selection—not only through premating courtship but also through intimate interactions during and long after mating. But what makes one prospective mate more enticing than another? Darwin, shaped by misogyny and sexual prudery, invoked a “taste for the beautiful” without speculating on the origin of the “taste.” How to explain when the “final marriage ceremony” is between two rams? What of oral sex in bats, cloacal rubbing in bonobos, or the sexual spectrum in humans, all observable in Darwin’s time? By explaining desire through the lens of those male traits that caught his eyes and those of his gender and culture, Darwin elided these data in his theory of sexual evolution. Work since Darwin has focused on how traits and preferences coevolve. Preferences can evolve even if attractive signals only predict offspring attractiveness, but most attention has gone to the intuitive but tenuous premise that mating with gorgeous partners yields vigorous offspring. By focusing on those aspects of mating preferences that coevolve with male traits, many of Darwin’s influential followers have followed the same narrow path. The sexual selection debate in the 1980s was framed as “good genes versus runaway”: Do preferences coevolve with traits because traits predict genetic benefits, or simply because they are beautiful? To the broader world this is still the conversation. ADVANCES Even as they evolve toward ever-more-beautiful signals and healthier offspring, mate-choice mechanisms and courter traits are locked in an arms race of coercion and resistance, persuasion and skepticism. Traits favored by sexual selection often do so at the expense of chooser fitness, creating sexual conflict. Choosers then evolve preferences in response to the costs imposed by courters. Often, though, the current traits of courters tell us little about how preferences arise. Sensory systems are often tuned to nonsexual cues like food, favoring mating signals resembling those cues. And preferences can emerge simply from selection on choosing conspecifics. Sexual selection can therefore arise from chooser biases that have nothing to do with ornaments. Choice may occur before mating, as Darwin emphasized, but individuals mate multiple times and bias fertilization and offspring care toward favored partners. Mate choice can thus occur in myriad ways after mating, through behavioral, morphological, and physiological mechanisms. Like other biological traits, mating preferences vary among individuals and species along multiple dimensions. Some of this is likely adaptive, as different individuals will have different optimal mates. Indeed, mate choice may be more about choosing compatible partners than picking the “best” mate in the absolute sense. Compatibility-based choice can drive or reinforce genetic divergence and lead to speciation. The mechanisms underlying the “taste for the beautiful” determine whether mate choice accelerates or inhibits reproductive isolation. If preferences are learned from parents, or covary with ecological differences like the sensory environment, then choice can promote genetic divergence. If everyone shares preferences for attractive ornaments, then choice promotes gene flow between lineages. OUTLOOK Two major trends continue to shift the emphasis away from male “beauty” and toward how and why individuals make sexual choices. The first integrates neuroscience, genomics, and physiology. We need not limit ourselves to the feathers and dances that dazzled Darwin, which gives us a vastly richer picture of mate choice. The second is that despite persistent structural inequities in academia, a broader range of people study a broader range of questions. This new focus confirms Darwin’s insight that mate choice makes a primary contribution to sexual selection, but suggests that sexual selection is often tangential to mate choice. This conclusion challenges a persistent belief with sinister roots, whereby mate choice is all about male ornaments. Under this view, females evolve to prefer handsome males who provide healthy offspring, or alternatively, to express flighty whims for arbitrary traits. But mate-choice mechanisms also evolve for a host of other reasons Understanding mate choice mechanisms is key to understanding how sexual decisions underlie speciation and adaptation to environmental change. New theory and technology allow us to explicitly connect decision-making mechanisms with their evolutionary consequences. A century and a half after Darwin, we can shift our focus to females and males as choosers, rather than the gaudy by-products of mate choice. Mate choice mechanisms across domains of life. Sensory periphery for stimulus detection (yellow), brain for perceptual integration and evaluation (orange), and reproductive structures for postmating choice among pollen or sperm (teal). ILLUSTRATION: KELLIE HOLOSKI/ SCIENCEmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
