skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.
Attention:The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, June 11 until 2:00 AM ET on Friday, June 12 due to maintenance. We apologize for the inconvenience.


Title: Hybrid brain-computer interface with motor imagery and error-related brain activity
Objective. Brain-computer interface (BCI) systems read and interpret brain activity directly from the brain. They can provide a means of communication or locomotion for patients suffering from neurodegenerative diseases or stroke. However, non-stationarity of brain activity limits the reliable transfer of the algorithms that were trained during a calibration session to real-time BCI control. One source of non-stationarity is the user's brain response to the BCI output (feedback), for instance, whether the BCI feedback is perceived as an error by the user or not. By taking such sources of non-stationarity into account, the reliability of the BCI can be improved. Approach. In this work, we demonstrate a real-time implementation of a hybrid motor imagery BCI combining the information from the motor imagery signal and the error-related brain activity simultaneously so as to gain benefit from both sources. Main results. We show significantly improved performance in real-time BCI control across 12 participants, compared to a conventional motor imagery BCI. The significant improvement is in terms of classification accuracy, target hit rate, subjective perception of control and information-transfer rate. Moreover, our offline analyses of the recorded EEG data show that the error-related brain activity provides a more reliable source of information than the motor imagery signal. Significance. This work shows for the first time, that the error-related brain activity classifier compared to the motor imagery classifier is more consistent when trained on calibration data and tested during online control, which likely explains why the proposed hybrid BCI allows for a more reliable means of communication or rehabilitation for patients in need.  more » « less
Award ID(s):
1817226 1219200 1528214
PAR ID:
10188594
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Journal of Neural Engineering
ISSN:
1741-2560
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Brain-computer interface (BCI) systems read and infer the user brain activity directly from the brain providing a means of communication and rehabilitation for patients in need. However, brain signals are known to be non-stationary and existing systems are not reliable and robust enough to be taken outside of the laboratory. Often times long calibration and recalibration of the system is required which can be tiresome and frustrating to the user. In this study, we compare the method of common spatial patterns (CSP) with two of its variants, namely, the canonical correlation analysis approach to common spatial patterns (CCACSP) and the common spatio-spectral patterns (CSSP) in detecting the motor imagery signal when trained on calibration data with sham feedback and tested in online control. We show that the motor imagery performance is significantly better with CSSP and CCACSP compared to CSP and hence, able to provide a more reliable transfer of the classifier from calibration to online control. 
    more » « less
  2. Abstract Objective.Non-invasive electroencephalograms (EEG)-based brain–computer interfaces (BCIs) play a crucial role in a diverse range of applications, including motor rehabilitation, assistive and communication technologies, holding potential promise to benefit users across various clinical spectrums. Effective integration of these applications into daily life requires systems that provide stable and reliable BCI control for extended periods. Our prior research introduced the AIRTrode, a self-adhesive (A), injectable (I), and room-temperature (RT) spontaneously-crosslinked hydrogel electrode (AIRTrode). The AIRTrode has shown lower skin-contact impedance and greater stability than dry electrodes and, unlike wet gel electrodes, does not dry out after just a few hours, enhancing its suitability for long-term application. This study aims to demonstrate the efficacy of AIRTrodes in facilitating reliable, stable and long-term online EEG-based BCI operations.Approach.In this study, four healthy participants utilized AIRTrodes in two BCI control tasks–continuous and discrete–across two sessions separated by six hours. Throughout this duration, the AIRTrodes remained attached to the participants’ heads. In the continuous task, participants controlled the BCI through decoding of upper-limb motor imagery (MI). In the discrete task, the control was based on decoding of error-related potentials (ErrPs).Main Results.Using AIRTrodes, participants demonstrated consistently reliable online BCI performance across both sessions and tasks. The physiological signals captured during MI and ErrPs tasks were valid and remained stable over sessions. Lastly, both the BCI performances and physiological signals captured were comparable with those from freshly applied, research-grade wet gel electrodes, the latter requiring inconvenient re-application at the start of the second session.Significance.AIRTrodes show great potential promise for integrating non-invasive BCIs into everyday settings due to their ability to support consistent BCI performances over extended periods. This technology could significantly enhance the usability of BCIs in real-world applications, facilitating continuous, all-day functionality that was previously challenging with existing electrode technologies. 
    more » « less
  3. Brain-computer interface (BCI) systems are proposed as a means of communication for locked-in patients. One common BCI paradigm is motor imagery in which the user controls a BCI by imagining movements of different body parts. It is known that imagining different body parts results in event-related desynchronization (ERD) in various frequency bands. Existing methods such as common spatial patterns (CSP) and its refinement filterbank common spatial patterns (FB-CSP) aim at finding features that are informative for classification of the motor imagery class. Our proposed method is a temporally adaptive common spatial patterns implementation of the commonly used filter-bank common spatial patterns method using convolutional neural networks; hence it is called TA-CSPNN. With this method we aim to: (1) make the feature extraction and classification end-to-end, (2) base it on the way CSP/FBCSP extracts relevant features, and finally, (3) reduce the number of trainable parameters compared to existing deep learning methods to improve generalizability in noisy data such as EEG. More importantly, we show that this reduction in parameters does not affect performance and in fact the trained network generalizes better for data from some participants. We show our results on two datasets, one publicly available from BCI Competition IV, dataset 2a and another in-house motor imagery dataset. 
    more » « less
  4. This systematic review addresses the plausibility of using novel feedback modalities for brain–computer interface (BCI) and attempts to identify the best feedback modality on the basis of the effectiveness or learning rate. Out of the chosen studies, it was found that 100% of studies tested visual feedback, 31.6% tested auditory feedback, 57.9% tested tactile feedback, and 21.1% tested proprioceptive feedback. Visual feedback was included in every study design because it was intrinsic to the response of the task (e.g. seeing a cursor move). However, when used alone, it was not very effective at improving accuracy or learning. Proprioceptive feedback was most successful at increasing the effectiveness of motor imagery BCI tasks involving neuroprosthetics. The use of auditory and tactile feedback resulted in mixed results. The limitations of this current study and further study recommendations are discussed. 
    more » « less
  5. Brain-computer interface (BCI) actively translates the brain signals into executable actions by establishing direct communication between the human brain and external devices. Recording brain activity through electroencephalography (EEG) is generally contaminated with both physiological and nonphysiological artifacts, which significantly hinders the BCI performance. Artifact subspace reconstruction (ASR) is a well-known statistical technique that automatically removes artifact components by determining the rejection threshold based on the initial reference EEG segment in multichannel EEG recordings. In real-world applications, the fixed threshold may limit the efficacy of the artifact correction, especially when the quality of the reference data is poor. This study proposes an adaptive online ASR technique by integrating the Hebbian/anti-Hebbian neural networks into the ASR algorithm, namely, principle subspace projection ASR (PSP-ASR) and principal subspace whitening ASR (PSW-ASR) that segmentwise self-organize the artifact subspace by updating the synaptic weights according to the Hebbian and anti-Hebbian learning rules. The effectiveness of the proposed algorithm is compared to the conventional ASR approaches on benchmark EEG dataset and three BCI frameworks, including steady-state visual evoked potential (SSVEP), rapid serial visual presentation (RSVP), and motor imagery (MI) by evaluating the root-mean-square error (RMSE), the signal-to-noise ratio (SNR), the Pearson correlation, and classification accuracy. The results demonstrated that the PSW-ASR algorithm effectively removed the EEG artifacts and retained the activity-specific brain signals compared to the PSP-ASR, standard ASR (Init-ASR), and moving-window ASR (MW-ASR) methods, thereby enhancing the SSVEP, RSVP, and MI BCI performances. Finally, our empirical results from the PSW-ASR algorithm suggested the choice of an aggressive cutoff range of c = 1-10 for activity-specific BCI applications and a moderate range of for the benchmark dataset and general BCI applications. 
    more » « less