An official website of the United States government
A new class of neuromorphic processors promises to provide fast and power-efficient execution of spiking neural networks with on-chip synaptic plasticity. This efficiency derives in part from the fine-grained parallelism as well as event-driven communication mediated by spatially and temporally sparse spike messages. Another source of efficiency arises from the close spatial proximity between synapses and the sites where their weights are applied and updated. This proximity of compute and memory elements drastically reduces expensive data movements but imposes the constraint that only local operations can be efficiently performed, similar to constraints present in biological neural circuits. Efficient weight update operations should therefore only depend on information available locally at each synapse as non-local operations that involve copying, taking a transpose, or normalizing an entire weight matrix are not efficiently supported by present neuromorphic architectures. Moreover, spikes are typically non-negative events, which imposes additional constraints on how local weight update operations can be performed. The Locally Competitive Algorithm (LCA) is a dynamical sparse solver that uses only local computations between non-spiking leaky integrator neurons, allowing for massively parallel implementations on compatible neuromorphic architectures such as Intel's Loihi research chip. It has been previously demonstrated that non-spiking LCA can be used to learn dictionaries of convolutional kernels in an unsupervised manner from raw, unlabeled input, although only by employing non-local computation and signed non-spiking outputs. Here, we show how unsupervised dictionary learning with spiking LCA (S-LCA) can be implemented using only local computation and unsigned spike events, providing a promising strategy for constructing self-organizing neuromorphic chips.
more »
« less
A 22-pJ/spike 73-Mspikes/s 130k-compartment neural array transceiver with conductance-based synaptic and membrane dynamics
Neuromorphic cognitive computing offers a bio-inspired means to approach the natural intelligence of biological neural systems in silicon integrated circuits. Typically, such circuits either reproduce biophysical neuronal dynamics in great detail as tools for computational neuroscience, or abstract away the biology by simplifying the functional forms of neural computation in large-scale systems for machine intelligence with high integration density and energy efficiency. Here we report a hybrid which offers biophysical realism in the emulation of multi-compartmental neuronal network dynamics at very large scale with high implementation efficiency, and yet with high flexibility in configuring the functional form and the network topology. The integrate-and-fire array transceiver (IFAT) chip emulates the continuous-time analog membrane dynamics of 65 k two-compartment neurons with conductance-based synapses. Fired action potentials are registered as address-event encoded output spikes, while the four types of synapses coupling to each neuron are activated by address-event decoded input spikes for fully reconfigurable synaptic connectivity, facilitating virtual wiring as implemented by routing address-event spikes externally through synaptic routing table. Peak conductance strength of synapse activation specified by the address-event input spans three decades of dynamic range, digitally controlled by pulse width and amplitude modulation (PWAM) of the drive voltage activating the log-domain linear synapse circuit. Two nested levels of micro-pipelining in the IFAT architecture improve both throughput and efficiency of synaptic input. This two-tier micro-pipelining results in a measured sustained peak throughput of 73 Mspikes/s and overall chip-level energy efficiency of 22 pJ/spike. Non-uniformity in digitally encoded synapse strength due to analog mismatch is mitigated through single-point digital offset calibration. Combined with the flexibly layered and recurrent synaptic connectivity provided by hierarchical address-event routing of registered spike events through external memory, the IFAT lends itself to efficient large-scale emulation of general biophysical spiking neural networks, as well as rate-based mapping of rectified linear unit (ReLU) neural activations.
more »
« less
- Award ID(s):
- 2208771
- PAR ID:
- 10511471
- Publisher / Repository:
- Frontiers in Neuroscience
- Date Published:
- Journal Name:
- Frontiers in Neuroscience
- Volume:
- 17
- ISSN:
- 1662-453X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The cerebral cortex contains numerous neuronal cell types, distinguished by their molecular identity as well as their electrophysiological and morphological properties. Cortical function is reliant on stereotyped patterns of synaptic connectivity and synaptic function among these neuron types, but how these patterns are established during development remains poorly understood. Selective targeting not only of different cell types but also of distinct postsynaptic neuronal domains occurs in many brain circuits and is directed by multiple mechanisms. These mechanisms include the regulation of axonal and dendritic guidance and fine-scale morphogenesis of pre- and postsynaptic processes, lineage relationships, activity dependent mechanisms and intercellular molecular determinants such as transmembrane and secreted molecules, many of which have also been implicated in neurodevelopmental disorders. However, many studies of synaptic targeting have focused on circuits in which neuronal processes target different lamina, such that cell-type-biased connectivity may be confounded with mechanisms of laminar specificity. In the cerebral cortex, each cortical layer contains cell bodies and processes from intermingled neuronal cell types, an arrangement that presents a challenge for the development of target-selective synapse formation. Here, we address progress and future directions in the study of cell-type-biased synaptic targeting in the cerebral cortex. We highlight challenges to identifying developmental mechanisms generating stereotyped patterns of intracortical connectivity, recent developments in uncovering the determinants of synaptic target selection during cortical synapse formation, and current gaps in the understanding of cortical synapse specificity.more » « less
-
Synaptic plasticity, the dynamic tuning of signal transmission strength between neurons, serves as a fundamental basis for memory and learning in biological organisms. This adaptive nature of synapses is considered one of the key features contributing to the superior energy efficiency of the brain. Here, we use molecular dynamics simulations to demonstrate synaptic-like plasticity in a subnanoporous two-dimensional membrane. We show that a train of voltage spikes dynamically modifies the membrane’s ionic permeability in a process involving competitive bicationic transport. This process is shown to be repeatable after a given resting period. Because of a combination of subnanometer pore size and the atomic thinness of the membrane, this system exhibits energy dissipation of 0.1 to 100 aJ per voltage spike, which is several orders of magnitude lower than 0.1 to 10 fJ per spike in the human synapse. We reveal the underlying physical mechanisms at molecular detail and investigate the local energetics underlying this apparent synaptic-like behavior.more » « less
-
Neuromorphic computing is considered to have the potential to overcome the limitations of traditional von Neumann architecture due to its high efficiency, low energy consumption, and fault-tolerance. Hardware components that can emulate the synaptic plasticity of neurons, i.e. artificial synaptic devices, are required by neuromorphic systems. New devices have been examined for such components, such as phase-change artificial synapse, ferroelectric artificial synapse, and memristor synapses. Among them, memristor, a two-terminal metal-insulator-metal structure that are analogous to a biological synapse with presynaptic neuron (top electrode), postsynapticneuron (bottom electrode), and synaptic cleft (memristive film), is a promising device technology because of its tunable resistance, scalability, 3D integration compatibility, low power consumption, and relatively high speed. In contrary to inorganic materials such as metal oxides, natural organic materials have attracted interest to form the memristive layer because they are renewable, biodegradable, sustainable, biocompatible, and environmentally friendly. In this paper, honey solution embedded with carbon nanotubes (CNTs) was processed into the memristive layer by a low cost solution-based process, with synaptic plasticity of the final honey-CNT memristors characterized, including forget and relearn, spike-rate-dependent plasticity, spike-voltage-dependent plasticity, short-term to long-term memory transition, paired pulse facilitation, and spatial supra-linear summation behaviors. The successful emulation of these essential biological synaptic behaviors demonstrates the potential of honey-CNT memristors as a viable hardware component in neuromorphic computing systems.more » « less
-
ABSTRACT Synapses are often precisely organized on dendritic arbors, yet the role of synaptic topography in dendritic integration remains poorly understood. Utilizing electron microscopy (EM) connectomics we investigate synaptic topography inDrosophila melanogasterlooming circuits, focusing on retinotopically tuned visual projection neurons (VPNs) that synapse onto descending neurons (DNs). Synapses of a given VPN type project to non-overlapping regions on DN dendrites. Within these spatially constrained clusters, synapses are not retinotopically organized, but instead adopt near random distributions. To investigate how this organization strategy impacts DN integration, we developed multicompartment models of DNs fitted to experimental data and using precise EM morphologies and synapse locations. We find that DN dendrite morphologies normalize EPSP amplitudes of individual synaptic inputs and that near random distributions of synapses ensure linear encoding of synapse numbers from individual VPNs. These findings illuminate how synaptic topography influences dendritic integration and suggest that linear encoding of synapse numbers may be a default strategy established through connectivity and passive neuron properties, upon which active properties and plasticity can then tune as needed.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/fnins.2023.1198306
