Search for: All records

Abstract Dysphagia or difficulty swallowing is caused by the failure of neurological pathways to properly activate swallowing muscles. Current electromyography (EMG) systems for dysphagia monitoring are bulky and rigid, limiting their potential for long‐term and unobtrusive use. To address this, a machine learning‐assisted wearable EMG system is presented, utilizing self‐adhesive, skin‐conformal, semi‐transparent, and robust ionic gel electrodes. The presented electrodes possess good conductivity, superior skin contact, and good transmittance, ensuring high‐fidelity EMG sensing without impeding daily activities. Moreover, the optimized material and structural designs ensure wearing comfort and conformable skin‐electrode contact, allowing for long‐term monitoring with high accuracy. Machine learning and mel‐frequency cepstral coefficient techniques are employed to classify swallowing events based on food types and volumes. Through an analysis of electrode placement on the chin and neck, the proposed system is able to effectively distinguish between different food types and water volumes using a small number of channels, making it suitable for continuous dysphagia monitoring. This work represents an advancement in machine learning assisted EMG systems for the classification and regression of swallowing events, paving the way for more efficient, unobtrusive, and long‐term dysphagia monitoring systems. 

Abstract Silent speech interfaces offer an alternative and efficient communication modality for individuals with voice disorders and when the vocalized speech communication is compromised by noisy environments. Despite the recent progress in developing silent speech interfaces, these systems face several challenges that prevent their wide acceptance, such as bulkiness, obtrusiveness, and immobility. Herein, the material optimization, structural design, deep learning algorithm, and system integration of mechanically and visually unobtrusive silent speech interfaces are presented that can realize both speaker identification and speech content identification. Conformal, transparent, and self‐adhesive electromyography electrode arrays are designed for capturing speech‐relevant muscle activities. Temporal convolutional networks are employed for recognizing speakers and converting sensing signals into spoken content. The resulting silent speech interfaces achieve a 97.5% speaker classification accuracy and 91.5% keyword classification accuracy using four electrodes. The speech interface is further integrated with an optical hand‐tracking system and a robotic manipulator for human‐robot collaboration in both assembly and disassembly processes. The integrated system achieves the control of the robot manipulator by silent speech and facilitates the hand‐over process by hand motion trajectory detection. The developed framework enables natural robot control in noisy environments and lays the ground for collaborative human‐robot tasks involving multiple human operators. 

Silent speech interfaces decode intended speech from physiological signals without the need for vocalized sound. These systems provide an alternative modality to voice-based spoken communication, addressing limitations posed by physiological constraints and environmental interferences. This review presents a comprehensive overview of bio-integrated systems for silent speech recognition, with emphasis on their physiological relevance, state-of-the-art hardware designs, signal characteristics, and machine learning (ML)-assisted speech decoding pipelines. Based on the level of physical intrusion into the body, bio-integrated speech interfaces can be categorized into epidermal, intraoral, and surgically embedded systems. Each modality captures distinct physiological signals involved in speech production. The design of bio-integrated systems involves critical trade-offs among recognition accuracy, invasiveness, portability, and robustness. Recent advances in flexible and stretchable electronics have significantly enhanced device comfort, signal quality, and integration level across these modalities. This review also outlines recent progress in ML-assisted signal processing pipelines, including preprocessing, feature extraction, ML model architectures, and evaluation metrics. Both signal-to-text and signal-to-audio approaches are discussed. Finally, application scenarios such as assistive communication, human-machine interaction, and user authentication are introduced, followed by an outlook on current challenges and emerging research directions that position this field for transformative clinical and consumer applications. 

Soft epidermal magnetic material has demonstrated excellent flexibility and wearability across a range of emerging applications, e.g., pressure sensing, robotics, among others. In this paper, we build a flexible magnetic skin based system MagSkin for touchless Human Computer Interaction (HCI). For example, a user can wear a Bandaid-like sticker on his fingertip to operate a smartphone using classic gestures without touching the screen, i.e., tap virtual icons floating on top of the screen. The magnetic skin is specially fabricated with randomly distributed magnetic microparticles, so that any shape change of the skin will cause position and orientation changes of those particles, introducing complex variations of the magnetic field around the skin. To characterize the magnetic field of the magnetic skin, we designed a multi-dipole based model. Based on this model, we developed 6-DoF position tracking and orientation estimation algorithms to localize the skin precisely. We also designed novel gestures, e.g., finger bend and pinch, for users to make input/adjustment based on the flexibility of the material. We have conducted experiments with 15 volunteers for 18 hours, experimental results show localization accuracy is 6.2 mm in 3D, 4.5 mm in 2D, and recognition accuracy is 94% for classical gesture and 92% for novel gesture. 

While semi‐autonomous drones are increasingly used for road infrastructure inspection, their insufficient ability to independently handle complex scenarios beyond initial job planning hinders their full potential. To address this, the paper proposes a human–drone collaborative inspection approach leveraging flexible surface electromyography (sEMG) for conveying inspectors' speech guidance to intelligent drones. Specifically, this paper contributes a new data set,sEMGCommands forPilotingDrones (sCPD), and ansEMG‐basedCross‐subjectClassificationNetwork (sXCNet), for both command keyword recognition and inspector identification. sXCNet acquires the desired functions and performance through a synergetic effort of sEMG signal processing, spatial‐temporal‐frequency deep feature extraction, and multitasking‐enabled cross‐subject representation learning. The cross‐subject design permits deploying one unified model across all authorized inspectors, eliminating the need for subject‐dependent models tailored to individual users. sXCNet achieves notable classification accuracies of 98.1% on the sCPD data set and 86.1% on the public Ninapro db1 data set, demonstrating strong potential for advancing sEMG‐enabled human–drone collaboration in road infrastructure inspection.