skip to main content

Title: A pilot study of the Earable device to measure facial muscle and eye movement tasks among healthy volunteers
The Earable device is a behind-the-ear wearable originally developed to measure cognitive function. Since Earable measures electroencephalography (EEG), electromyography (EMG), and electrooculography (EOG), it may also have the potential to objectively quantify facial muscle and eye movement activities relevant in the assessment of neuromuscular disorders. As an initial step to developing a digital assessment in neuromuscular disorders, a pilot study was conducted to determine whether the Earable device could be utilized to objectively measure facial muscle and eye movements intended to be representative of Performance Outcome Assessments, (PerfOs) with tasks designed to model clinical PerfOs, referred to as mock-PerfO activities. The specific aims of this study were: To determine whether the Earable raw EMG, EOG, and EEG signals could be processed to extract features describing these waveforms; To determine Earable feature data quality, test re-test reliability, and statistical properties; To determine whether features derived from Earable could be used to determine the difference between various facial muscle and eye movement activities; and, To determine what features and feature types are important for mock-PerfO activity level classification. A total of N = 10 healthy volunteers participated in the study. Each study participant performed 16 mock-PerfOs activities, including talking, chewing, swallowing, eye closure, gazing in different directions, puffing cheeks, chewing an apple, and making various facial expressions. Each activity was repeated four times in the morning and four times at night. A total of 161 summary features were extracted from the EEG, EMG, and EOG bio-sensor data. Feature vectors were used as input to machine learning models to classify the mock-PerfO activities, and model performance was evaluated on a held-out test set. Additionally, a convolutional neural network (CNN) was used to classify low-level representations of the raw bio-sensor data for each task, and model performance was correspondingly evaluated and compared directly to feature classification performance. The model’s prediction accuracy on the Earable device’s classification ability was quantitatively assessed. Study results indicate that Earable can potentially quantify different aspects of facial and eye movements and may be used to differentiate mock-PerfO activities. Specially, Earable was found to differentiate talking, chewing, and swallowing tasks from other tasks with observed F1 scores >0.9. While EMG features contribute to classification accuracy for all tasks, EOG features are important for classifying gaze tasks. Finally, we found that analysis with summary features outperformed a CNN for activity classification. We believe Earable may be used to measure cranial muscle activity relevant for neuromuscular disorder assessment. Classification performance of mock-PerfO activities with summary features enables a strategy for detecting disease-specific signals relative to controls, as well as the monitoring of intra-subject treatment responses. Further testing is needed to evaluate the Earable device in clinical populations and clinical development settings.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ;
Li-Jessen, Nicole Yee-Key
Date Published:
Journal Name:
PLOS Digital Health
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Patients with neuromuscular disease fail to produce necessary muscle force and have trouble maintaining joint moment required to perform activities of daily living. Measuring muscle force values in patients with neuromuscular disease is important but challenging. Electromyography (EMG) can be used to obtain muscle activation values, which can be converted to muscle forces and joint torques. Surface electrodes can measure activations of superficial muscles, but fine-wire electrodes are needed for deep muscles, although it is invasive and require skilled personnel and preparation time. EMG-driven modeling with surface electrodes alone could underestimate the net torque. In this research, authors propose a methodology to predict muscle activations from deeper muscles of the upper extremity. This method finds missing muscle activation one at a time by combining an EMG-driven musculoskeletal model and muscle synergies. This method tracks inverse dynamics joint moments to determine synergy vector weights and predict muscle activation of selected shoulder and elbow muscles of a healthy subject. In addition, muscle-tendon parameter values (optimal fiber length, tendon slack length, and maximum isometric force) have been personalized to the experimental subject. The methodology is tested for a wide range of rehabilitation tasks of the upper extremity across multiple healthy subjects. Results show this methodology can determine single unmeasured muscle activation up to Pearson's correlation coefficient (R) of 0.99 (root mean squared error, RMSE = 0.001) and 0.92 (RMSE = 0.13) for the elbow and shoulder muscles, respectively, for one degree-of-freedom (DoF) tasks. For more complicated five DoF tasks, activation prediction accuracy can reach up to R = 0.71 (RMSE = 0.29).

    more » « less
  2. Scalp electroencephalography (EEG) is a neural source signal that is extensively used in neuroengineering due to its non-invasive nature and ease of collection. However, a drawback to the use of EEG is the prevalence of physiological artifacts generated by eye movements and eye blinks that contaminate the brain signals. Previously, we have proposed and validated an H ∞ -based Adaptive Noise Cancellation (ANC) technique for the real-time identification, learning and removal of eye blinks, eye motions, amplitude drifts and recording biases from EEG simultaneously. However, the standard electroocu- lography (EOG) electrode configuration requires four elec- trodes for EOG measurement, which limits its applicability for reduced-channel mobile applications, such as brain-computer interfaces (BCI). Here, we assess multiple configurations with varying number of EOG electrodes and compare the ANC effectiveness of these configurations to the ideal four-electrode configuration. From an analysis of the root mean squared error (RMSE) and differences in signal to noise ratios (SNR) between the ideal four-electrode case and the alternative configurations, it is reported that several three-electrode alternative configu- rations were effective in essentially replicating the ability to remove EOG artifacts in an experimental cohort of ten healthy subjects. For nine subjects, it was shown that only two to three EOG electrodes were needed to achieve similar performance as compared to the four-electrode case. This study demonstrates that the typical four-electrode configuration for EOG recordings for adaptive noise cancellation of ocular artifacts may not be necessary; by using the proposed new EOG configurations it is possible to improve electrode allocation efficiency for EOG measurements in mobile EEG applications. 
    more » « less
  3. IEEE (Ed.)
    This research involves developing a drone control system that functions by relating EEG and EMG from the forehead to different facial movements using recurrent neural networks (RNN) such as long-short term memory (LSTM) and gated recurrent Unit (GRU). As current drone control methods are largely limited to handheld devices, regular operators are actively engaged while flying and cannot perform any passive control. Passive control of drones would prove advantageous in various applications as drone operators can focus on additional tasks. The advantages of the chosen methods and those of some alternative system designs are discussed. For this research, EEG signals were acquired at three frontal cortex locations (fp1, fpz , fp2 ) using electrodes from an OpenBCI headband and observed for patterns of Fast Fourier Transform (FFT) frequency-amplitude distributions. Five different facial expressions were repeated while recording EEG signals of 0-60Hz frequencies with two reference electrodes placed on both earlobes. EMG noise received during EEG measurements was not filtered away but was observed to be minimal. A dataset was first created for the actions done, and later categorized by a mean average error (MAE), a statistical error deviation analysis and then classified with both an LSTM and GRU neural network by relating FFT amplitudes to the actions. On average, the LSTM network had classification accuracy of 78.6%, and the GRU network had a classification accuracy of 81.8%. 
    more » « less
  4. Abstract Cerebrovascular accidents like a stroke can affect the lower limb as well as upper extremity joints (i.e., shoulder, elbow, or wrist) and hinder the ability to produce necessary torque for activities of daily living. In such cases, muscles’ ability to generate forces reduces, thus affecting the joint’s torque production. Understanding how muscles generate forces is a key element to injury detection. Researchers have developed several computational methods to obtain muscle forces and joint torques. Electromyography (EMG) driven modeling is one of the approaches to estimate muscle forces and obtain joint torques from muscle activity measurements. Musculoskeletal models and EMG-driven models require necessary muscle-specific parameters for the calculation. The focus of this study is to investigate the EMG-driven approach along with an upper extremity musculoskeletal model to determine muscle forces of two major muscle groups, biceps brachii and triceps brachii, consisting of seven muscle-tendon units. Estimated muscle forces are used to determine the elbow joint torque. Experimental EMG signals and motion capture data are collected for a healthy subject. The musculoskeletal model is scaled to match the geometric parameters of the subject. Then, the approach calculates muscle forces and joint moment for two tasks: simple elbow flexion extension and triceps kickback. Individual muscle forces and net joint torques for both tasks are estimated. The study also has compared the effect of muscle-tendon parameters (optimal fiber length and tendon slack length) on the estimated results. 
    more » « less
  5. Obeid, Iyad ; Selesnick, Ivan ; Picone, Joseph (Ed.)
    The Neural Engineering Data Consortium has recently developed a new subset of its popular open source EEG corpus – TUH EEG (TUEG) [1]. The TUEG Corpus is the world’s largest open source corpus of EEG data and currently has over 3,300 subscribers. There are several valuable subsets of this data, including the TUH Seizure Detection Corpus (TUSZ) [2], which was featured in the Neureka 2020 Epilepsy Challenge [3]. In this poster, we present a new subset of the TUEG Corpus – the TU Artifact Corpus. This corpus contains 310 EEG files in which every artifact has been annotated. This data can be used to evaluate artifact reduction technology. Since TUEG is comprised of actual clinical data, the set of artifacts appearing in the data is rich and challenging. EEG artifacts are defined as waveforms that are not of cerebral origin and may be affected by numerous external and or physiological factors. These extraneous signals are often mistaken for seizures due to their morphological similarity in amplitude and frequency [4]. Artifacts often lead to raised false alarm rates in machine learning systems, which poses a major challenge for machine learning research. Most state-of-the-art systems use some form of artifact reduction technology to suppress these events. The corpus was annotated using a five-way classification that was developed to meet the needs of our constituents. Brief descriptions of each form of the artifact are provided in Ochal et al. [4]. The five basic tags are: • Chewing (CHEW): An artifact resulting from the tensing and relaxing of the jaw muscles. Chewing is a subset of the muscle artifact class. Chewing has the same characteristic high frequency sharp waves with 0.5 sec baseline periods between bursts. This artifact is generally diffuse throughout the different regions of the brain. However, it might have a higher level of activity in one hemisphere. Classification of a muscle artifact as chewing often depends on whether the accompanying patient report mentions any chewing, since other muscle artifacts can appear superficially similar to chewing artifact. • Electrode (ELEC): An electrode artifact encompasses various electrode related artifacts. Electrode pop is an artifact characterized by channels using the same electrode “spiking” with an electrographic phase reversal. Electrostatic is an artifact caused by movement or interference of electrodes and or the presence of dissimilar metals. A lead artifact is caused by the movement of electrodes from the patient’s head and or poor connection of electrodes. This results in disorganized and high amplitude slow waves. • Eye Movement (EYEM): A spike-like waveform created during patient eye movement. This artifact is usually found on all of the frontal polar electrodes with occasional echoing on the frontal electrodes. • Muscle (MUSC): A common artifact with high frequency, sharp waves corresponding to patient movement. These waveforms tend to have a frequency above 30 Hz with no specific pattern, often occurring because of agitation in the patient. • Shiver (SHIV): A specific and sustained sharp wave artifact that occurs when a patient shivers, usually seen on all or most channels. Shivering is a relatively rare subset of the muscle artifact class. Since these artifacts can overlap in time, a concatenated label format was implemented as a compromise between the limitations of our annotation tool and the complexity needed in an annotation data structure used to represent these overlapping events. We distribute an XML format that easily handles overlapping events. Our annotation tool [5], like most annotation tools of this type, is limited to displaying and manipulating a flat or linear annotation. Therefore, we encode overlapping events as a series of concatenated names using symbols such as: • EYEM+CHEW: eye movement and chewing • EYEM+SHIV: eye movement and shivering • CHEW+SHIV: chewing and shivering An example of an overlapping annotation is shown below in Figure 1. This release is an update of TUAR v1.0.0, which was a partially annotated database. In v1.0.0, a similar five way system was used as well as an additional “null” tag. The “null” tag covers anything that was not annotated, including instances of artifact. Only a limited number of artifacts were annotated in v1.0.0. In this updated version, every instance of an artifact is annotated; ultimately, this provides the user with confidence that any part of the record that is not annotated with one of the five classes does not contain an artifact. No new files, patients, or sessions were added in v2.0.0. However, the data was reannotated with these standards. The total number of files remains the same, but the number of artifact events increases significantly. Complete statistics will be provided on the corpus once annotation is complete and the data is released. This is expected to occur in early July – just after the IEEE SPMB submission deadline. The TUAR Corpus is an open-source database that is currently available for use by any registered member of our consortium. To register and receive access, please follow the instructions provided at this web page: The data is located here: 
    more » « less