More Like this

1. Robust computation with rhythmic spike patterns

   https://doi.org/10.1073/pnas.1902653116  

   Frady, E. Paxon ; Sommer, Friedrich T. ( August 2019 , Proceedings of the National Academy of Sciences)

   Information coding by precise timing of spikes can be faster and more energy efficient than traditional rate coding. However, spike-timing codes are often brittle, which has limited their use in theoretical neuroscience and computing applications. Here, we propose a type of attractor neural network in complex state space and show how it can be leveraged to construct spiking neural networks with robust computational properties through a phase-to-timing mapping. Building on Hebbian neural associative memories, like Hopfield networks, we first propose threshold phasor associative memory (TPAM) networks. Complex phasor patterns whose components can assume continuous-valued phase angles and binary magnitudes can be stored and retrieved as stable fixed points in the network dynamics. TPAM achieves high memory capacity when storing sparse phasor patterns, and we derive the energy function that governs its fixed-point attractor dynamics. Second, we construct 2 spiking neural networks to approximate the complex algebraic computations in TPAM, a reductionist model with resonate-and-fire neurons and a biologically plausible network of integrate-and-fire neurons with synaptic delays and recurrently connected inhibitory interneurons. The fixed points of TPAM correspond to stable periodic states of precisely timed spiking activity that are robust to perturbation. The link established between rhythmic firing patterns and complex attractor dynamics has implications for the interpretation of spike patterns seen in neuroscience and can serve as a framework for computation in emerging neuromorphic devices. 

   more » « less
2. Realizing Behavior Level Associative Memory Learning Through Three-Dimensional Memristor-Based Neuromorphic Circuits

   https://doi.org/10.1109/TETCI.2019.2921787  

   An, Hongyu ; An, Qiyuan ; Yi, Yang ( July 2019 , IEEE Transactions on Emerging Topics in Computational Intelligence)

   Associative memory is a widespread self-learning method in biological livings, which enables the nervous system to remember the relationship between two concurrent events. The significance of rebuilding associative memory at a behavior level is not only to reveal a way of designing a brain-like self-learning neuromorphic system but also to explore a method of comprehending the learning mechanism of a nervous system. In this paper, an associative memory learning at a behavior level is realized that successfully associates concurrent visual and auditory information together (pronunciation and image of digits). The task is achieved by associating the large-scale artificial neural networks (ANNs) together instead of relating multiple analog signals. In this way, the information carried and preprocessed by these ANNs can be associated. A neuron has been designed, named signal intensity encoding neurons (SIENs), to encode the output data of the ANNs into the magnitude and frequency of the analog spiking signals. Then, the spiking signals are correlated together with an associative neural network, implemented with a three-dimensional (3-D) memristor array. Furthermore, the selector devices in the traditional memristor cells limiting the design area have been avoided by our novel memristor weight updating scheme. With the novel SIENs, the 3-D memristive synapse, and the proposed memristor weight updating scheme, the simulation results demonstrate that our proposed associative memory learning method and the corresponding circuit implementations successfully associate the pronunciation and image of digits together, which mimics a human-like associative memory learning behavior. 

   more » « less
3. Semantically-correlated memories in a dense associative model

   Burns, Thomas F ( July 2024 , Proceedings of Machine Learning Research)

   I introduce a novel associative memory model named Correlated Dense Associative Memory (CDAM), which integrates both auto- and hetero-association in a unified framework for continuous-valued memory patterns. Employing an arbitrary graph structure to semantically link memory patterns, CDAM is theoretically and numerically analysed, revealing four distinct dynamical modes: auto-association, narrow hetero-association, wide hetero-association, and neutral quiescence. Drawing inspiration from inhibitory modulation studies, I employ anti-Hebbian learning rules to control the range of hetero-association, extract multi-scale representations of community structures in graphs, and stabilise the recall of temporal sequences. Experimental demonstrations showcase CDAM’s efficacy in handling real-world data, replicating a classical neuroscience experiment, performing image retrieval, and simulating arbitrary finite automata. 

   more » « less
4. Reproducing Fear Conditioning of Rats with Unmanned Ground Vehicles and Neuromorphic Systems

   https://doi.org/10.1109/ISQED57927.2023.10129317  

   Zins, Noah ; An, Hongyu ( April 2023 , IEEE International Symposium on Quality Electronic Design)

   Deep learning accomplishes remarkable success through training with massively labeled datasets. However, the high demands on the datasets impede the feasibility of deep learning in edge computing scenarios and suffer the data scarcity issue. Rather than relying on labeled data, animals learn by interacting with their surroundings and memorizing the relationship between concurrent events. This learning paradigm is referred to as associative memory. The successful implementation of associative memory potentially achieves self-learning schemes analogous to animals to resolve the challenges of deep learning. The state-of-the-art implementations of associative memory are limited to small-scale and offline paradigms. Thus, in this work, we implement associative memory learning with an Unmanned Ground Vehicle (UGV) and neuromorphic chips (Intel Loihi) for an online learning scenario. Our system reproduces the classic associative memory in rats. In specific, our system successfully reproduces the fear conditioning with no pretraining procedure and labeled datasets. In our experiments, the UGV serves as a substitute for the rats. Our UGV autonomously memorizes the cause-and-effect of the light stimulus and vibration stimulus, then exhibits a movement response. During associative memory learning, the synaptic weights are updated by Hebbian learning. The Intel Loihi chip is integrated with our online learning system for processing visual signals. Its average power usages for computing logic and memory are 30 mW and 29 mW, respectively. 

   more » « less
5. Targeted memory reactivation of face-name learning depends on ample and undisturbed slow-wave sleep

   https://doi.org/10.1038/s41539-021-00119-2  

   Whitmore, Nathan W. ; Bassard, Adrianna M. ; Paller, Ken A. ( January 2022 , npj Science of Learning)

   Face memory, including the ability to recall a person’s name, is of major importance in social contexts. Like many other memory functions, it may rely on sleep. We investigated whether targeted memory reactivation during sleep could improve associative and perceptual aspects of face memory. Participants studied 80 face-name pairs, and then a subset of spoken names with associated background music was presented unobtrusively during a daytime nap. This manipulation preferentially improved name recall and face recognition for those reactivated face-name pairs, as modulated by two factors related to sleep quality; memory benefits were positively correlated with the duration of stage N3 sleep (slow-wave sleep) and negatively correlated with measures of sleep disruption. We conclude that (a) reactivation of specific face-name memories during sleep can strengthen these associations and the constituent memories, and that (b) the effectiveness of this reactivation depends on uninterrupted N3 sleep.

    

   more » « less