An official website of the United States government
Abstract This paper aims to present a potential cybersecurity risk existing in mixed reality (MR)-based smart manufacturing applications that decipher digital passwords through a single RGB camera to capture the user’s mid-air gestures. We first created a test bed, which is an MR-based smart factory management system consisting of mid-air gesture-based user interfaces (UIs) on a video see-through MR head-mounted display. To interact with UIs and input information, the user’s hand movements and gestures are tracked by the MR system. We setup the experiment to be the estimation of the password input by users through mid-air hand gestures on a virtual numeric keypad. To achieve this goal, we developed a lightweight machine learning-based hand position tracking and gesture recognition method. This method takes either video streaming or recorded video clips (taken by a single RGB camera in front of the user) as input, where the videos record the users’ hand movements and gestures but not the virtual UIs. With the assumption of the known size, position, and layout of the keypad, the machine learning method estimates the password through hand gesture recognition and finger position detection. The evaluation result indicates the effectiveness of the proposed method, with a high accuracy of 97.03%, 94.06%, and 83.83% for 2-digit, 4-digit, and 6-digit passwords, respectively, using real-time video streaming as input with known length condition. Under the unknown length condition, the proposed method reaches 85.50%, 76.15%, and 77.89% accuracy for 2-digit, 4-digit, and 6-digit passwords, respectively.
more »
« less
Sensory Adaptation in Biomolecular Memristors Improves Reservoir Computing Performance
Despite its prevalence in neurosensory systems for pattern recognition, event detection, and learning, the effects of sensory adaptation (SA) are not explored in reservoir computing (RC). Monazomycin‐based biomolecular synapse (MzBS) devices that exhibit volatile memristance and short‐term plasticity with two strength‐dependent modes of response are studied: facilitation and facilitation‐then‐depression (i.e., SA). Their ability to perform RC tasks including digit recognition, nonlinear function learning, and aerodynamic gust classification via combination of model‐based device simulations and physical experiments where SA presence is controlled is studied. Simulations exhibiting moderate SA achieve significantly higher accuracy classifying a custom 5 × 5 binary digit set, with experimental validation achieving maximum testing accuracies of 90%. Classifications of the Modified National Institute of Standards and Technology (MNIST) handwritten digit dataset achieve a maximum testing accuracy of 94.34% in devices with SA. Fitting error of the Mackey–Glass time series is also significantly reduced by SA. Experimentally obtained pressure distributions representing gusts on an airfoil in a wind tunnel are classified by MzBS reservoirs. Reservoirs exhibiting SA achieve 100% accuracy, unlike MzBS reservoirs without SA and comparable static neural networks.
more »
« less
- PAR ID:
- 10472245
- Publisher / Repository:
- Wiley
- Date Published:
- Journal Name:
- Advanced Intelligent Systems
- Volume:
- 5
- Issue:
- 8
- ISSN:
- 2640-4567
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Driven by the development of machine learning and the development of wireless techniques, lots of research efforts have been spent on the human activity recognition (HAR). Although various deep learning algorithms can achieve high accuracy for recognizing human activities, existing works lack of a theoretical performance upper bound which is the best accuracy that is only limited by the influencing factors in wireless networks such as indoor physical environments and settings of wireless sensing devices regardless of any HAR algorithm. Without the understanding of performance upper bound, mistakenly configuring the influencing factors can reduce the HAR accuracy drastically no matter what deep learning algorithms are utilized. In this paper, we propose the HAR performance upper bound which is the minimum classification error probability that doesn't depend on any HAR algorithms and can be considered as a function of influencing factors in wireless sensing networks for CSI based human activity recognition. Since the performance upper bound can capture the impacts of influencing factors on HAR accuracy, we further analyze the influences of those factors with varying situations such as through the wall HAR and different human activities by MATLAB simulations.more » « less
-
Interface‐type (IT) metal/oxide Schottky memristive devices have attracted considerable attention over filament‐type (FT) devices for neuromorphic computing because of their uniform, filament‐free, and analog resistive switching (RS) characteristics. The most recent IT devices are based on oxygen ions and vacancies movement to alter interfacial Schottky barrier parameters and thereby control RS properties. However, the reliability and stability of these devices have been significantly affected by the undesired diffusion of ionic species. Herein, a reliable interface‐dominated memristive device is demonstrated using a simple Au/Nb‐doped SrTiO3(Nb:STO) Schottky structure. The Au/Nb:STO Schottky barrier modulation by charge trapping and detrapping is responsible for the analog resistive switching characteristics. Because of its interface‐controlled RS, the proposed device shows low device‐to‐device, cell‐to‐cell, and cycle‐to‐cycle variability while maintaining high repeatability and stability during endurance and retention tests. Furthermore, the Au/Nb:STO IT memristive device exhibits versatile synaptic functions with an excellent uniformity, programmability, and reliability. A simulated artificial neural network with Au/Nb:STO synapses achieves a high recognition accuracy of 94.72% for large digit recognition from MNIST database. These results suggest that IT resistive switching can be potentially used for artificial synapses to build next‐generation neuromorphic computing.more » « less
-
Habituation and sensitization represent nonassociative learning mechanisms in both non‐neural and neural organisms. They are essential for a range of functions from survival to adaptation in dynamic environments. Design of hardware for neuroinspired computing strives to emulate such features driven by electric bias and can also be incorporated into neural network algorithms. Herein, cellular‐like learning in oxygen‐deficient NiOxdevices is demonstrated. Both habituation learning and sensitization response can be achieved in a single device by simply controlling the magnitude of the electric field. Spontaneous memory relaxations and dynamic redistribution of oxygen vacancies under electric bias enable such learning behavior of NiOxunder sequential training. These characteristics in simple device arrays are implemented to learn alphabets as well as demonstrate simulated algorithmic use cases in digit recognition. Transition metal oxides with carefully prepared defect concentrations can be highly sensitive to electronic structure perturbations under moderate electrical stimulus and serve as building blocks for next‐generation neuroinspired computing hardware.more » « less
-
Agaian, Sos S.; DelMarco, Stephen P.; Asari, Vijayan K. (Ed.)Iris recognition is a widely used biometric technology that has high accuracy and reliability in well-controlled environments. However, the recognition accuracy can significantly degrade in non-ideal scenarios, such as off-angle iris images. To address these challenges, deep learning frameworks have been proposed to identify subjects through their off-angle iris images. Traditional CNN-based iris recognition systems train a single deep network using multiple off-angle iris image of the same subject to extract the gaze invariant features and test incoming off-angle images with this single network to classify it into same subject class. In another approach, multiple shallow networks are trained for each gaze angle that will be the experts for specific gaze angles. When testing an off-angle iris image, we first estimate the gaze angle and feed the probe image to its corresponding network for recognition. In this paper, we present an analysis of the performance of both single and multimodal deep learning frameworks to identify subjects through their off-angle iris images. Specifically, we compare the performance of a single AlexNet with multiple SqueezeNet models. SqueezeNet is a variation of the AlexNet that uses 50x fewer parameters and is optimized for devices with limited computational resources. Multi-model approach using multiple shallow networks, where each network is an expert for a specific gaze angle. Our experiments are conducted on an off-angle iris dataset consisting of 100 subjects captured at 10-degree intervals between -50 to +50 degrees. The results indicate that angles that are more distant from the trained angles have lower model accuracy than the angles that are closer to the trained gaze angle. Our findings suggest that the use of SqueezeNet, which requires fewer parameters than AlexNet, can enable iris recognition on devices with limited computational resources while maintaining accuracy. Overall, the results of this study can contribute to the development of more robust iris recognition systems that can perform well in non-ideal scenarios.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1002/aisy.202300049
