An official website of the United States government
Recently, Intelligent IoT (IIoT), including various sensors, has gained significant attention due to its capability of sensing, deciding, and acting by leveraging artificial neural networks (ANN). Nevertheless, to achieve acceptable accuracy and high performance in visual systems, a power-delay-efficient architecture is required. In this paper, we propose an ultra-low-power processing in-sensor architecture, namely SenTer, realizing low-precision ternary multi-layer perceptron networks, which can operate in detection and classification modes. Moreover, SenTer supports two activation functions based on user needs and the desired accuracy-energy trade-off. SenTer is capable of performing all the required computations for the MLP's first layer in the analog domain and then submitting its results to a co-processor. Therefore, SenTer significantly reduces the overhead of analog buffers, data conversion, and transmission power consumption by using only one ADC. Additionally, our simulation results demonstrate acceptable accuracy on various datasets compared to the full precision models.
more »
« less
HyperSense: Hyperdimensional Intelligent Sensing for Energy‐Efficient Sparse Data Processing
Introducing HyperSense, the co‐designed hardware and software system efficiently controls analog‐to‐digital converter (ADC) modules’ data generation rate based on object presence predictions in sensor data. Addressing challenges posed by escalating sensor quantities and data rates, HyperSense reduces redundant digital data using energy‐efficient low‐precision ADC, diminishing machine learning system costs. Leveraging neurally inspired hyperdimensional computing, HyperSense analyzes real‐time raw low‐precision sensor data, offering advantages in handling noise, memory‐centricity, and real‐time learning. The proposed HyperSense model combines high‐performance software for object detection with real‐time hardware prediction, introducing the novel concept of intelligent sensor control. Comprehensive software and hardware evaluations demonstrate the solution's superior performance, evidenced by the highest area under the curve and sharpest receiver operating characteristic curve among lightweight models. Hardware‐wise, the field programmable gate array‐based domain‐specific accelerator tailored for HyperSense achieves a 5.6× speedup compared to YOLOv4 on NVIDIA Jetson Orin while showing up to 92.1% energy saving compared to the conventional system. These results underscore HyperSense's effectiveness and efficiency, positioning it as a promising solution for intelligent sensing and real‐time data processing across diverse applications.
more »
« less
- PAR ID:
- 10516226
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Advanced Intelligent Systems
- Volume:
- 6
- Issue:
- 12
- ISSN:
- 2640-4567
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Sensors have recently become valuable tools in engineering, providing real-time data for monitoring structures and the environment. They are also emerging as new tools in education and training, offering learners real-time information to reinforce their understanding of engineering concepts. However, sensing technology’s complexity, costs, fabrication and implementation challenges often hinder engineers’ exploration. Simplifying these aspects could make sensors more accessible to engineering students. In this study, the researcher developed, fabricated, and tested an efficient low-cost wireless intelligent sensor aimed at education and research, named LEWIS1. This paper describes the hardware and software architecture of the first prototype and their use, as well as the proposed new versions, LEWIS1-β and LEWIS1-γ, which simplify both hardware and software. The capabilities of the proposed sensor are compared with those of an accurate commercial PCB sensor. This paper also demonstrates examples of outreach efforts and suggests the adoption of the newer versions of LEWIS1 as tools for education and research. The authors also investigated the number of activities and sensor-building workshops that have been conducted since 2015 using the LEWIS sensor, showing an increasing trend in the excitement of people from various professions to participate and learn sensor fabrication.more » « less
-
Abstract The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor‐based compute‐in‐memory (CIM) modules can perform vector‐matrix multiplication (VMM) in place and in parallel, and have shown great promises in DNN inference applications. However, CIM‐based model training faces challenges due to non‐linear weight updates, device variations, and low‐precision. In this work, a mixed‐precision training scheme is experimentally implemented to mitigate these effects using a bulk‐switching memristor‐based CIM module. Low‐precision CIM modules are used to accelerate the expensive VMM operations, with high‐precision weight updates accumulated in digital units. Memristor devices are only changed when the accumulated weight update value exceeds a pre‐defined threshold. The proposed scheme is implemented with a system‐onchip of fully integrated analog CIM modules and digital sub‐systems, showing fast convergence of LeNet training to 97.73%. The efficacy of training larger models is evaluated using realistic hardware parameters and verifies that CIM modules can enable efficient mix‐precision DNN training with accuracy comparable to full‐precision software‐trained models. Additionally, models trained on chip are inherently robust to hardware variations, allowing direct mapping to CIM inference chips without additional re‐training.more » « less
-
Vision Transformers (ViTs) have evolved in the field of computer vision by transitioning traditional Convolutional Neural Networks (CNNs) into attention-based architectures. This architecture processes input images as sequences of patches. ViTs achieve enhanced performance in many tasks such as image classification and object detection due to their ability to capture global dependencies within input data. While their software implementations are widely adopted, deploying ViTs on hardware introduces several challenges. These include fault tolerance in the presence of hardware failures, real-time reliability, and high computational requirements. Permanent faults that are in processing elements, interconnections, or memory subsystems lead to incorrect computations and degrading system performance. This paper proposes a fault-tolerant hardware implementation of ViTs to overcome these challenges. This hardware implementation integrates real-time fault detection and recovery mechanisms. The architecture includes four primary units: patch embedding, encoder, decoder, and Multi Layer Perceptron (MLP) which are supported by fault-tolerant components such as lightweight recompute units, a centralized Built-In Self-Test (BIST), and a learning-based decision-making system using machine learning model 'decision tree'. These units are interconnected through a centralized global buffer for efficient data transfer, ensuring seamless operation even under fault conditions.more » « less
-
Image sensors with programmable region-of-interest (ROI) readout are a new sensing technology important for energyefficient embedded computer vision. In particular, ROIs can subsample the number of pixels being readout while performing single object tracking in a video. In this paper, we develop adaptive sampling algorithms which perform joint object tracking and predictive video subsampling. We utilize an object detection consisting of either mean shift tracking or a neural network, coupled with a Kalman filter for prediction. We show that our algorithms achieve mean average precision of 0.70 or higher on a dataset of 20 videos in software. Further, we implement hardware acceleration of mean shift tracking with Kalman filter adaptive subsampling on an FPGA. Hardware results show a 23× improvement in clock cycles and latency as compared to baseline methods and achieves 38FPS real-time performance. This research points to a new domain of hardware-software co-design for adaptive video subsampling in embedded computer vision.more » « less
