Convolutional neural networks (CNN) are an emerging technique in modeling neural circuits and have been shown to converge to biologically plausible functionality in cortical circuits via task-optimization. This functionality has not been observed in CNN models of retinal circuits via task-optimization.
We sought to observe this convergence in retinal circuits by designing a biologically inspired CNN model of a motion-detection retinal circuit and optimizing it to solve a motion-classification task. The learned weights and parameters indicated that the CNN converged to direction-sensitive ganglion
and amacrine cells, cell types that have been observed in biology, and provided evidence that task-optimization is a fair method of building retinal models. The analysis used to understand the functionality of our CNN also indicates that biologically constrained deep learning models are easier to reason about their underlying mechanisms than traditional deep learning models.
more »
« less
Task-Optimized Retinal-Inspired CNN Converges to Biologically-Plausible Functionality. 8th Workshop on Biological Distributed Algorithms (BDA), July 2021.
Convolutional neural networks (CNN) are an emerging technique in modeling neural circuits and have been shown to converge to biologically plausible functionality in cortical circuits via task-optimization. This functionality has not been observed in CNN models of retinal circuits via task-optimization.
We sought to observe this convergence in retinal circuits by designing a biologically inspired CNN model of a motion-detection retinal circuit and optimizing it to solve a motion-classification task. The learned weights and parameters indicated that the CNN converged to direction-sensitive ganglion
and amacrine cells, cell types that have been observed in biology, and provided evidence that task-optimization is a fair method of building retinal models. The analysis used to understand the functionality of our CNN also indicates that biologically constrained deep learning models are easier to reason about their underlying mechanisms than traditional deep learning models.
more »
« less
- NSF-PAR ID:
- 10324421
- Date Published:
- Journal Name:
- 8th Workshop on Biological Distributed Algorithms (BDA)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Convolutional neural networks (CNNs), a class of deep learning models, have experienced recent success in modeling sensory cortices and retinal circuits through optimizing performance on machine learning tasks, otherwise known as task optimization. Previous research has shown task-optimized CNNs to be capable of providing explanations as to why the retina efficiently encodes natural stimuli and how certain retinal cell types are involved in efficient encoding. In our work, we sought to use task-optimized CNNs as a means of explaining computational mechanisms responsible for motion-selective retinal circuits. We designed a biologically constrained CNN and optimized its performance on a motion-classification task. We drew inspiration from psychophysics, deep learning, and systems neuroscience literature to develop a toolbox of methods to reverse engineer the computational mechanisms learned in our model. Through reverse engineering our model, we proposed a computational mechanism in which direction-selective ganglion cells and starburst amacrine cells, both experimentally observed retinal cell types, emerge in our model to discriminate among moving stimuli. This emergence suggests that direction-selective circuits in the retina are ecologically designed to robustly discriminate among moving stimuli. Our results and methods also provide a framework for how to build more interpretable deep learning models and how to understand them.more » « less
-
Human-Robot Collaboration (HRC), which envisions a workspace in which human and robot can dynamically collaborate, has been identified as a key element in smart manufacturing. Human action recognition plays a key role in the realization of HRC as it helps identify current human action and provides the basis for future action prediction and robot planning. Despite recent development of Deep Learning (DL) that has demonstrated great potential in advancing human action recognition, one of the key issues remains as how to effectively leverage the temporal information of human motion to improve the performance of action recognition. Furthermore, large volume of training data is often difficult to obtain due to manufacturing constraints, which poses challenge for the optimization of DL models. This paper presents an integrated method based on optical flow and convolutional neural network (CNN)-based transfer learning to tackle these two issues. First, optical flow images, which encode the temporal information of human motion, are extracted and serve as the input to a two-stream CNN structure for simultaneous parsing of spatial-temporal information of human motion. Then, transfer learning is investigated to transfer the feature extraction capability of a pretrained CNN to manufacturing scenarios. Evaluation using engine block assembly confirmed the effectiveness of the developed method.more » « less
-
more » « less
An organizational feature of neural circuits is the specificity of synaptic connections. A striking example is the direction-selective (DS) circuit of the retina. There are multiple subtypes of DS retinal ganglion cells (DSGCs) that prefer motion along one of 4 preferred directions. This computation is mediated by selective wiring of a single inhibitory interneuron, the starburst amacrine cell (SAC), with each DSGC subtype preferentially receiving input from a subset of SAC processes. We hypothesize that the molecular basis of this wiring is mediated in part by unique expression profiles of DSGC subtypes. To test this, we first performed paired recordings from isolated mouse retina of both sexes to determine that postnatal day 10 (P10) represents the age at which asymmetric synapses form. Second, we performed RNA-sequencing and differential expression analysis on isolated P10 ON-OFF DSGCs tuned for either nasal or ventral motion and identified candidates which may promote direction-specific wiring. We then used a conditional knockout strategy to test the role of one candidate, the secreted synaptic organizer cerebellin-4 (Cbln4), in the development of DS tuning. Using two-photon calcium imaging, we observed a small deficit in directional tuning among ventral-preferring DSGCs lacking Cbln4, though whole-cell voltage clamp recordings did not identify a significant change in inhibitory inputs. This suggests that Cbln4 does not function primarily via a cell-autonomous mechanism to instruct wiring of DS circuits. Nevertheless, our transcriptomic analysis identified unique candidate factors for gaining insights into the molecular mechanisms that instruct wiring specificity in the DS circuit.
Significance Statement By performing mRNA transcriptome analysis on three populations of direction-selective ganglion cells - two preferring horizontal motion and one preferring vertical motion - we identified differentially expressed candidate molecules potentially involved in cell subtype-specific synaptogenesis within this circuit. We tested the role of one differentially expressed candidate, Cbln4, enriched in ventral-preferring DSGCs. Using a targeted knockout approach, the deletion of Cbln4 led to a small reduction in direction-selective tuning while maintaining dendritic morphology and normal strength and asymmetry of inhibitory synaptic transmission. Overall, we have shown that this approach can be used to identify interesting candidate molecules, and future functional studies are required to reveal the mechanisms by which these candidates influence synaptic wiring within specific circuits. -
Evolution has honed predatory skills in the natural world where localizing and intercepting fast-moving prey is required. The current generation of robotic systems mimics these biological systems using deep learning. High-speed processing of the camera frames using convolutional neural networks (CNN) (frame pipeline) on such constrained aerial edge-robots gets resource-limited. Adding more compute resources also eventually limits the throughput at the frame rate of the camera as frame-only traditional systems fail to capture the detailed temporal dynamics of the environment. Bio-inspired event cameras and spiking neural networks (SNN) provide an asynchronous sensor-processor pair (event pipeline) capturing the continuous temporal details of the scene for high-speed but lag in terms of accuracy. In this work, we propose a target localization system combining event-camera and SNN-based high-speed target estimation and frame-based camera and CNN-driven reliable object detection by fusing complementary spatio-temporal prowess of event and frame pipelines. One of our main contributions involves the design of an SNN filter that borrows from the neural mechanism for ego-motion cancelation in houseflies. It fuses the vestibular sensors with the vision to cancel the activity corresponding to the predator's self-motion. We also integrate the neuro-inspired multi-pipeline processing with task-optimized multi-neuronal pathway structure in primates and insects. The system is validated to outperform CNN-only processing using prey-predator drone simulations in realistic 3D virtual environments. The system is then demonstrated in a real-world multi-drone set-up with emulated event data. Subsequently, we use recorded actual sensory data from multi-camera and inertial measurement unit (IMU) assembly to show desired working while tolerating the realistic noise in vision and IMU sensors. We analyze the design space to identify optimal parameters for spiking neurons, CNN models, and for checking their effect on the performance metrics of the fused system. Finally, we map the throughput controlling SNN and fusion network on edge-compatible Zynq-7000 FPGA to show a potential 264 outputs per second even at constrained resource availability. This work may open new research directions by coupling multiple sensing and processing modalities inspired by discoveries in neuroscience to break fundamental trade-offs in frame-based computer vision 1 .more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
