An official website of the United States government
Abstract Neuromorphic computing, exemplified by breakthroughs in machine vision through concepts like address-event representation and send-on-delta sampling, has revolutionised sensor technology, enabling low-latency and high dynamic range perception with minimal bandwidth. While these advancements are prominent in vision and auditory perception, their potential in machine olfaction remains under-explored, particularly in the context of fast sensing. Here, we outline the perspectives for neuromorphic principles in machine olfaction. Considering the physical characteristics of turbulent odour environments, we argue that event-driven signal processing is optimally suited to the inherent properties of olfactory signals. We highlight the lack of bandwidth limitation due to turbulent dispersal processes, the characteristic temporal and chemical sparsity, as well as the high information density of the odour landscape. Further, we critically review and discuss the literature on neuromorphic olfaction; particularly focusing on neuromorphic principles such as event generation algorithms, information encoding mechanisms, event processing schemes (spiking neural networks), and learning. We discuss that the application of neuromorphic principles may significantly enhance response time and task performance in robotic olfaction, enabling autonomous systems to perform complex tasks in turbulent environments—such as environmental monitoring, odour guided search and rescue operations, and hazard detection.
more »
« less
RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network
Neuromorphic computing systems promise high energy efficiency and low latency. In particular, when integrated with neuromorphic sensors, they can be used to produce intelligent systems for a broad range of applications. An event‐based camera is such a neuromorphic sensor, inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the event data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks (SNNs) that are expensive to train. In this work, a neural network architecture is proposed, reservoir nodes‐enabled neuromorphic vision sensing network (RN‐Net), based on dynamic temporal encoding by on‐sensor reservoirs and simple deep neural network (DNN) blocks. The reservoir nodes enable efficient temporal processing of asynchronous events by leveraging the native dynamics of the node devices, while the DNN blocks enable spatial feature processing. Combining these blocks in a hierarchical structure, the RN‐Net offers efficient processing for both local and global spatiotemporal features. RN‐Net executes dynamic vision tasks created by event‐based cameras at the highest accuracy reported to date at one order of magnitude smaller network size. The use of simple DNN and standard backpropagation‐based training rules further reduces implementation and training costs.
more »
« less
- Award ID(s):
- 1900675
- PAR ID:
- 10509229
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Advanced Intelligent Systems
- Volume:
- 6
- Issue:
- 12
- ISSN:
- 2640-4567
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
An ultra-low-power gesture and gait classification SoC is presented for rehabilitation application featuring (1) mixed-signal feature extraction and integrated low-noise amplifier eliminating expensive ADC and digital feature extraction, (2) an integrated distributed deep neural network (DNN) ASIC supporting a scalable multi-chip neural network for sensor fusion with distortion resiliency for low-cost front end modules, (3) onchip learning of DNN engine allowing in-situ training of user specific operations. A 12-channel 65nm CMOS test chip was fabricated with 1μW power per channel, less than 3ms computation latency, on-chip training for user-specific DNN model and multi-chip networking capability.more » « less
-
Spiking neural networks (SNNs) are positioned to enable spatio-temporal information processing and ultra-low power event-driven neuromorphic hardware. However, SNNs are yet to reach the same performances of conventional deep artificial neural networks (ANNs), a long-standing challenge due to complex dynamics and non-differentiable spike events encountered in training. The existing SNN error backpropagation (BP) methods are limited in terms of scalability, lack of proper handling of spiking discontinuities, and/or mismatch between the rate coded loss function and computed gradient. We present a hybrid macro/micro level backpropagation (HM2-BP) algorithm for training multi-layer SNNs. The temporal effects are precisely captured by the proposed spike-train level post-synaptic potential (S-PSP) at the microscopic level. The rate-coded errors are defined at the macroscopic level, computed and back-propagated across both macroscopic and microscopic levels. Different from existing BP methods, HM2-BP directly computes the gradient of the rate-coded loss function w.r.t tunable parameters. We evaluate the proposed HM2-BP algorithm by training deep fully connected and convolutional SNNs based on the static MNIST [14] and dynamic neuromorphic N-MNIST [26]. HM2-BP achieves an accuracy level of 99:49% and 98:88% for MNIST and N-MNIST, respectively, outperforming the best reported performances obtained from the existing SNN BP algorithms. Furthermore, the HM2-BP produces the highest accuracies based on SNNs for the EMNIST [3] dataset, and leads to high recognition accuracy for the 16-speaker spoken English letters of TI46 Corpus [16], a challenging spatio-temporal speech recognition benchmark for which no prior success based on SNNs was reported. It also achieves competitive performances surpassing those of conventional deep learning models when dealing with asynchronous spiking streams.more » « less
-
Abstract In-sensor processing of dynamic and static information of visual objects avoids exchanging redundant data between physically separated sensing and computing units, holding promise for computer vision hardware. To this end, gate-tunable photodetectors, if built in a highly scalable array form, would lend themselves to large-scale in-sensor visual processing because of their potential in volume production and hence, parallel operation. Here we present two scalable in-sensor visual processing arrays based on dual-gate silicon photodiodes, enabling parallelized event sensing and edge detection, respectively. Both arrays are built in CMOS compatible processes and operated with zero static power. Furthermore, their bipolar analog output captures the amplitude of event-driven light changes and the spatial convolution of optical power densities at the device level, a feature that helps boost their performance in classifying dynamic motions and static images. Capable of processing both temporal and spatial visual information, these retinomorphic arrays suggest a path towards large-scale in-sensor visual processing systems for high-throughput computer vision.more » « less
-
Co-localized memory and processing components are necessary for the creation of scalable neuromorphic hardware, and these components can be produced using inexpensive techniques. While memristors have emerged as promising candidates for synaptic emulation, most devices rely on complex fabrication processes that hinder large-scale deployment. This work presents a fully inkjet-printed memristor utilizing hexagonal boron nitride (hBN) and graphene inks, demonstrating a manufacturable approach to neuromorphic systems. The devices exhibit nonvolatile resistive switching with symmetric pinched hysteresis loops [Formula: see text] and tunable conductance states. Furthermore, a piece-wise nonlinear memristor model was developed and implemented in Simscape to enable system-level simulations. A [Formula: see text] reservoir computing architecture, constructed using these memristive devices, successfully demonstrated temporal pattern processing and feature extraction capabilities. The results establish printed memristors as viable building blocks for next-generation edge computing applications, combining the advantages of solution-processable materials with neuromorphic functionality.more » « less
