An official website of the United States government
Introducing HyperSense, the co‐designed hardware and software system efficiently controls analog‐to‐digital converter (ADC) modules’ data generation rate based on object presence predictions in sensor data. Addressing challenges posed by escalating sensor quantities and data rates, HyperSense reduces redundant digital data using energy‐efficient low‐precision ADC, diminishing machine learning system costs. Leveraging neurally inspired hyperdimensional computing, HyperSense analyzes real‐time raw low‐precision sensor data, offering advantages in handling noise, memory‐centricity, and real‐time learning. The proposed HyperSense model combines high‐performance software for object detection with real‐time hardware prediction, introducing the novel concept of intelligent sensor control. Comprehensive software and hardware evaluations demonstrate the solution's superior performance, evidenced by the highest area under the curve and sharpest receiver operating characteristic curve among lightweight models. Hardware‐wise, the field programmable gate array‐based domain‐specific accelerator tailored for HyperSense achieves a 5.6× speedup compared to YOLOv4 on NVIDIA Jetson Orin while showing up to 92.1% energy saving compared to the conventional system. These results underscore HyperSense's effectiveness and efficiency, positioning it as a promising solution for intelligent sensing and real‐time data processing across diverse applications.
more »
« less
Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak
Active feedback control in magnetic confinement fusion devices is desirable to mitigate plasma instabilities and enable robust operation. Optical high-speed cameras provide a powerful, non-invasive diagnostic and can be suitable for these applications. In this study, we process high-speed camera data, at rates exceeding 100 kfps, on in situ field-programmable gate array (FPGA) hardware to track magnetohydrodynamic (MHD) mode evolution and generate control signals in real time. Our system utilizes a convolutional neural network (CNN) model, which predicts the n = 1 MHD mode amplitude and phase using camera images with better accuracy than other tested non-deep-learning-based methods. By implementing this model directly within the standard FPGA readout hardware of the high-speed camera diagnostic, our mode tracking system achieves a total trigger-to-output latency of 17.6 μs and a throughput of up to 120 kfps. This study at the High Beta Tokamak-Extended Pulse (HBT-EP) experiment demonstrates an FPGA-based high-speed camera data acquisition and processing system, enabling application in real-time machine-learning-based tokamak diagnostic and control as well as potential applications in other scientific domains.
more »
« less
- PAR ID:
- 10532678
- Publisher / Repository:
- Review of Scientific Instruments
- Date Published:
- Journal Name:
- Review of Scientific Instruments
- Volume:
- 95
- Issue:
- 7
- ISSN:
- 0034-6748
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV’s current status is complicated and rigid.We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9:23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.more » « less
-
Abstract This paper presents a software-hardware integrated approach to high-speed large-range dynamic mode imaging of atomic force microscope (AFM). High speed AFM imaging is needed to interrogate dynamic processes at nanoscale such as cellular interactions and polymer crystallization process. High-speed dynamic-modes such as tapping-mode AFM imaging is challenging as the probe tapping motion is sensitive to the highly nonlinear probe-sample interaction during the imaging process. The existing hardware-based approach via bandwidth enlargement, however, results in a substantially reduction of imaging area that can be covered. Contrarily, control (algorithm)-based approach, for example, the recently developed adaptive multiloop mode (AMLM) technique, has demonstrated its efficacy in increasing the tapping-mode imaging speed without loss of imaging size. Further improvement, however, has been limited by the hardware bandwidth and online signal processing speed and computation complexity.Thus, in this paper, the AMLM technique is further enhanced to optimize the probe tapping regulation and integrated with a field programmable gate array (FPGA) platform to further increase the imaging speed without loss of imaging quality and range. Experimental implementation of the proposed approach demonstrates that the high-quality imaging can be achieved at a high-speed scanning rate of 100 Hz and higher, and over a large imaging area of over 20 µm.more » « less
-
“Active structures” are physical structures that incorporate real-time monitoring and control. Examples include active vibration damping or blast mitigation systems. Evaluating physics-based models in real-time is generally not feasible for such systems having high-rate dynamics which require microsecond response times, but data-driven machine-learning-based models can potentially offer a solution. This paper compares the cost and performance of two FPGA-based implementations of real-time, continuously-trained models for forecasting timeseries signals with non-stationarities, with one using HighLevel Synthesis (HLS) and the other a programmable overlay architecture. The proposed model accepts a uni-variate vibration signal and seeks to forecast future samples to inform highrate controllers. The proposed forecasting method performs two concurrent neural inference operations. One inference forecasts the state of the signal f samples into the future as a function of the most recent h samples, while the other forecasts the current sample given h samples starting from h + f − 1 samples into the past. The first forecast produces the forecast while the second forecast allows the system to calculate the model’s loss and perform an immediate model update before the next sample period.more » « less
-
“Active structures” are physical structures that incorporate real-time monitoring and control. Examples includeactive vibration damping or blast mitigation systems. Evaluating physics-based models in real-time is generally not feasible for such systems having high-rate dynamics which require microsecond response times, but data-driven machine-learning-based models can potentially offer a solution. This paper compares the cost and performance of two FPGA-based implementations of real-time, continuously-trained models for forecasting timeseries signals with non-stationarities, with one using HighLevel Synthesis (HLS) and the other a programmable overlay architecture. The proposed model accepts a uni-variate vibration signal and seeks to forecast future samples to inform highrate controllers. The proposed forecasting method performs two concurrent neural inference operations. One inference forecasts the state of the signal f samples into the future as a function of the most recent h samples, while the other forecasts the current sample given h samples starting from h + f − 1 samples into the past. The first forecast produces the forecast while the second forecast allows the system to calculate the model’s loss and perform an immediate model update before the next sample period.more » « less
