More Like this

1. Texture Discrimination using a Flexible Tactile Sensor Array on a Soft Biomimetic Finger

   https://doi.org/10.1109/SENSORS43011.2019.8956704  

   Sankar, Sriramana; Brown, Alisa; Balamurugan, Darshini; Nguyen, Harrison; Iskarous, Mark; Simcox, Talya; Kumar, Deepesh; Nakagawa, Andrei; Thakor, Nitish (October 2019, IEEE Sensors)

   null (Ed.)

   Soft robotic fingers provide enhanced flexibility and dexterity when interacting with the environment. The capability of soft fingers can be further improved by integrating them with tactile sensors to discriminate various textured surfaces. In this work, a flexible 3x3 fabric-based tactile sensor array was integrated with a soft, biomimetic finger for a texture discrimination task. The finger palpated seven different textured plates and the corresponding tactile response was converted into neuromorphic spiking patterns, mimicking the firing pattern of mechanoreceptors in the skin. Spike-based feature metrics were used to classify different textures using the support vector machine (SVM) classifier. The sensor was able to achieve an accuracy of 99.21% when two features, mean spike rate and average inter-spike interval, from each taxel were used as inputs into the classifier. The experiment showed that an inexpensive, soft, biomimetic finger combined with the flexible tactile sensor array can potentially help users perceive their environment better. 

   more » « less
2. Unsupervised Learning and Adaptive Classification of Neuromorphic Tactile Encoding of Textures

   https://doi.org/10.1109/BIOCAS.2018.8584702  

   Iskarous, Mark M.; Nguyen, Harrison H.; Osborn, Luke E.; Betthauser, Joseph L.; Thakor, Nitish V. (October 2018, Proceedings of IEEE Biomedical Circuits and Systems)

   null (Ed.)

   In this work, we investigated the classification of texture by neuromorphic tactile encoding and an unsupervised learning method. Additionally, we developed an adaptive classification algorithm to detect and characterize the presence of new texture data. The neuromorphic tactile encoding of textures from a multilayer tactile sensor was based on the physical structure and afferent spike signaling of human glabrous skin mechanoreceptors. We explored different neuromorphic spike pattern metrics and dimensionality reduction techniques in order to maximize classification accuracy while improving computational efficiency. Using a dataset composed of 3 textures, we showed that unsupervised learning of the neuromorphic tactile encoding data had high classification accuracy (mean=86.46%, sd=5 .44%). Moreover, the adaptive classification algorithm was successful at determining that there were 3 underlying textures in the training dataset. In this work, tactile information is transformed into neuromorphic spiking activity that can be used as a stimulation pattern to elicit texture sensation for prosthesis users. Furthermore, we provide the basis for identifying new textures adaptively which can be used to actively modify stimulation patterns to improve texture discrimination for the user. 

   more » « less
3. Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand

   https://doi.org/10.1126/scirobotics.aax2352  

   George, J. A.; Kluger, D. T.; Davis, T. S.; Wendelken, S. M.; Okorokova, E. V.; He, Q.; Duncan, C. C.; Hutchinson, D. T.; Thumser, Z. C.; Beckler, D. T.; et al (July 2019, Science Robotics)

   We describe use of a bidirectional neuromyoelectric prosthetic hand that conveys biomimetic sensory feedback. Electromyographic recordings from residual arm muscles were decoded to provide independent and proportional control of a six-DOF prosthetic hand and wrist—the DEKA LUKE arm. Activation of contact sensors on the prosthesis resulted in intraneural microstimulation of residual sensory nerve fibers through chronically implanted Utah Slanted Electrode Arrays, thereby evoking tactile percepts on the phantom hand. With sensory feedback enabled, the participant exhibited greater precision in grip force and was better able to handle fragile objects. With active exploration, the participant was also able to distinguish between small and large objects and between soft and hard ones. When the sensory feedback was biomimetic—designed to mimic natural sensory signals—the participant was able to identify the objects significantly faster than with the use of traditional encoding algorithms that depended on only the present stimulus intensity. Thus, artificial touch can be sculpted by patterning the sensory feedback, and biologically inspired patterns elicit more interpretable and useful percepts. 

   more » « less
4. Neuromorphic vision and tactile fusion for upper limb prosthesis control

   https://doi.org/10.1109/NER.2019.8716890  

   Hays, Mark; Osborn, Luke; Ghosh, Rohan; Iskarous, Mark; Hunt, Christopher; Thakor, Nitish V. (March 2019, IEEE Transactions on Biomedical Circuits and Systems)

   null (Ed.)

   A major issue with upper limb prostheses is the disconnect between sensory information perceived by the user and the information perceived by the prosthesis. Advances in prosthetic technology introduced tactile information for monitoring grasping activity, but visual information, a vital component in the human sensory system, is still not fully utilized as a form of feedback to the prosthesis. For able-bodied individuals, many of the decisions for grasping or manipulating an object, such as hand orientation and aperture, are made based on visual information before contact with the object. We show that inclusion of neuromorphic visual information, combined with tactile feedback, improves the ability and efficiency of both able-bodied and amputee subjects to pick up and manipulate everyday objects.We discovered that combining both visual and tactile information in a real-time closed loop feedback strategy generally decreased the completion time of a task involving picking up and manipulating objects compared to using a single modality for feedback. While the full benefit of the combined feedback was partially obscured by experimental inaccuracies of the visual classification system, we demonstrate that this fusion of neuromorphic signals from visual and tactile sensors can provide valuable feedback to a prosthetic arm for enhancing real-time function and usability. 

   more » « less
5. Hierarchical Tactile Sensation Integration from Prosthetic Fingertips Enables Multi-Texture Surface Recognition

   https://doi.org/10.3390/s21134324  

   Abd, Moaed A.; Paul, Rudy; Aravelli, Aparna; Bai, Ou; Lagos, Leonel; Lin, Maohua; Engeberg, Erik D. (July 2021, Sensors)

   Multifunctional flexible tactile sensors could be useful to improve the control of prosthetic hands. To that end, highly stretchable liquid metal tactile sensors (LMS) were designed, manufactured via photolithography, and incorporated into the fingertips of a prosthetic hand. Three novel contributions were made with the LMS. First, individual fingertips were used to distinguish between different speeds of sliding contact with different surfaces. Second, differences in surface textures were reliably detected during sliding contact. Third, the capacity for hierarchical tactile sensor integration was demonstrated by using four LMS signals simultaneously to distinguish between ten complex multi-textured surfaces. Four different machine learning algorithms were compared for their successful classification capabilities: K-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and neural network (NN). The time-frequency features of the LMSs were extracted to train and test the machine learning algorithms. The NN generally performed the best at the speed and texture detection with a single finger and had a 99.2 ± 0.8% accuracy to distinguish between ten different multi-textured surfaces using four LMSs from four fingers simultaneously. The capability for hierarchical multi-finger tactile sensation integration could be useful to provide a higher level of intelligence for artificial hands. 

   more » « less