An official website of the United States government
Abstract Human reasoning depends on reusing pieces of information by binding them together in new ways, thereby “making infinite uses of finite means” (Alexander von Humboldt). Needed for that is a binding mechanism that enables fast composition and decomposition of tokens of information. Binding can easily be implemented in symbolic computations through parentheses and ordering of symbols. But it is a highly nontrivial operation for distributed representations, where the tokens are encoded by activity patterns in large neural networks or large language models, or more abstractly, by a high dimensional vector. Vector Symbolic Architectures (VSAs) provide partial solutions, but are lacking the flexibility of the brain in information retrieval, e.g. retrieving the tokens from a composed representation or retrieval of a composed representation by just providing some tokens as a cue. We show that a mechanism which the brain employs for binding distributed representations, Behavioral Time Scale Synaptic Plasticity (BTSP), overcomes these deficiencies. In particular, it combines binding with attractor features that make information retrieval substantially more flexible and robust. We evaluate its performance on various applications, including encoding and decoding complex visual scenes and hierarchical binding. We also show that it enhances models for natural language processing and abstract brain computation. BTSP-based binding only requires binary synaptic weights and simple local synaptic plasticity, and can therefore easily be implemented through in-memory computing or other innovative designs for energy-efficient AI implementations.
more »
« less
This content will become publicly available on November 6, 2026
Behavioral Time Scale Synaptic Plasticity (BTSP) endows Hyperdimensional Computing with brain-like information retrieval flexibility
Abstract Hyperdimensional computing (HDC) addresses massively parallel implementations of symbolic computations that are both more transparent than ANNs and LLMs and more suitable for in-memory computing on highly energy-efficient analog hardware. It captures an essential aspects of brain computations: objects, concepts, and their attributes are encoded by very sparse distributed representations. But currently known methods for binding these tokens together entail deficits in flexible information retrieval.We show that a mechanism which the brain employs for binding, Behavioral Time Scale Synaptic Plasticity (BTSP), overcomes these deficiencies by adding attractor features to high-dimensional representations. They drastically improve the capability to recover from composed representations the tokens which have been bound together in them. One arrives in this way at a functionally more powerful HDC paradigm that provides new perspectives both for understanding how brains carry out symbolic computations, and for implementing them in novel energy-efficient and massively parallel neuromorphic hardware.
more »
« less
- Award ID(s):
- 2318152
- PAR ID:
- 10711777
- Publisher / Repository:
- Research Square
- Date Published:
- Format(s):
- Medium: X
- Institution:
- Research Square
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Brain-inspired HyperDimensional Computing (HDC) is an alternative computation model working based on the observation that the human brain operates on highdimensional representations of data. Existing HDC solutions rely on expensive pre-processing algorithms for feature extraction. In this paper, we propose StocHD, a novel end-to-end hyperdimensional system that supports accurate, efficient, and robust learning over raw data. StocHD expands HDC functionality to the computing area by mathematically defining stochastic arithmetic over HDC hypervectors. StocHD enables an entire learning application (including feature extractor) to process using HDC data representation, enabling uniform, efficient, robust, and highly parallel computation. We also propose a novel fully digital and scalable Processing In-Memory (PIM) architecture that exploits the HDC memory-centric nature to support extensively parallel computation.more » « less
-
Abstract—Hyperdimensional Computing (HDC) is a neurallyinspired computation model working based on the observation that the human brain operates on high-dimensional representations of data, called hypervector. Although HDC is significantly powerful in reasoning and association of the abstract information, it is weak on features extraction from complex data such as image/video. As a result, most existing HDC solutions rely on expensive pre-processing algorithms for feature extraction. In this paper, we propose StocHD, a novel end-to-end hyperdimensional system that supports accurate, efficient, and robust learning over raw data. Unlike prior work that used HDC for learning tasks, StocHD expands HDC functionality to the computing area by mathematically defining stochastic arithmetic over HDC hypervectors. StocHD enables an entire learning application (including feature extractor) to process using HDC data representation, enabling uniform, efficient, robust, and highly parallel computation. We also propose a novel fully digital and scalable Processing In-Memory (PIM) architecture that exploits the HDC memorycentric nature to support extensively parallel computation. Our evaluation over a wide range of classification tasks shows that StocHD provides, on average, 3.3x and 6.4x (52.3x and 143.Sx) faster and higher energy efficiency as compared to state-of-the-art HDC algorithm running on PIM (NVIDIA GPU), while providing 16x higher computational robustness.more » « less
-
Processing large amounts of data, especially in learning algorithms, poses a challenge for current embedded computing systems. Hyperdimensional (HD) computing (HDC) is a brain-inspired computing paradigm that works with high-dimensional vectors called hypervectors . HDC replaces several complex learning computations with bitwise and simpler arithmetic operations at the expense of an increased amount of data due to mapping the data into high-dimensional space. These hypervectors, more often than not, cannot be stored in memory, resulting in long data transfers from storage. In this article, we propose Store-n-Learn, an in-storage computing solution that performs HDC classification and clustering by implementing encoding, training, retraining, and inference across the flash hierarchy. To hide the latency of training and enable efficient computation, we introduce the concept of batching in HDC. We also present on-chip acceleration for HDC encoding in flash planes. This enables us to exploit the high parallelism provided by the flash hierarchy and encode multiple data points in parallel in both batched and non-batched fashion. Store-n-Learn also implements a single top-level FPGA accelerator with novel implementations for HDC classification training, retraining, inference, and clustering on the encoded data. Our evaluation over 10 popular datasets shows that Store-n-Learn is on average 222× (543×) faster than CPU and 10.6× (7.3×) faster than the state-of-the-art in-storage computing solution, INSIDER for HDC classification (clustering).more » « less
-
Hyperdimensional computing (HDC) is an approach from the cognitive science literature for solving information processing tasks using data represented as high-dimensional random vectors. The technique has a rigorous mathematical backing, and is easy to implement in energy-efficient and highly parallel hardware like FPGAs and processing-in-memory architectures. The effectiveness of HDC in machine learning largely depends on how raw data is mapped to high-dimensional space. In this work, we propose NysHD, a new method for constructing this mapping that is based on the Nyström method from the literature on kernel approximation. Our approach provides a simple recipe to turn any user-defined positive-semidefinite similarity function into an equivalent mapping in HDC. There is a vast literature on the design of such functions for learning problems. Our approach provides a mechanism to import them into the HDC setting, expanding the types of problems that can be tackled using HDC. Empirical evaluation against existing HDC encoding methods shows that NysHD can achieve, on average, 11% and 17% better classification accuracy on graph and string datasets respectively.more » « less
