skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Temporally Adaptive Common Spatial Patterns with Deep Convolutional Neural Networks
Brain-computer interface (BCI) systems are proposed as a means of communication for locked-in patients. One common BCI paradigm is motor imagery in which the user controls a BCI by imagining movements of different body parts. It is known that imagining different body parts results in event-related desynchronization (ERD) in various frequency bands. Existing methods such as common spatial patterns (CSP) and its refinement filterbank common spatial patterns (FB-CSP) aim at finding features that are informative for classification of the motor imagery class. Our proposed method is a temporally adaptive common spatial patterns implementation of the commonly used filter-bank common spatial patterns method using convolutional neural networks; hence it is called TA-CSPNN. With this method we aim to: (1) make the feature extraction and classification end-to-end, (2) base it on the way CSP/FBCSP extracts relevant features, and finally, (3) reduce the number of trainable parameters compared to existing deep learning methods to improve generalizability in noisy data such as EEG. More importantly, we show that this reduction in parameters does not affect performance and in fact the trained network generalizes better for data from some participants. We show our results on two datasets, one publicly available from BCI Competition IV, dataset 2a and another in-house motor imagery dataset.  more » « less
Award ID(s):
1817226 1219200 1528214
PAR ID:
10187073
Author(s) / Creator(s):
;
Date Published:
Journal Name:
2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
Page Range / eLocation ID:
4533 - 4536
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Brain-computer interface (BCI) systems read and infer the user brain activity directly from the brain providing a means of communication and rehabilitation for patients in need. However, brain signals are known to be non-stationary and existing systems are not reliable and robust enough to be taken outside of the laboratory. Often times long calibration and recalibration of the system is required which can be tiresome and frustrating to the user. In this study, we compare the method of common spatial patterns (CSP) with two of its variants, namely, the canonical correlation analysis approach to common spatial patterns (CCACSP) and the common spatio-spectral patterns (CSSP) in detecting the motor imagery signal when trained on calibration data with sham feedback and tested in online control. We show that the motor imagery performance is significantly better with CSSP and CCACSP compared to CSP and hence, able to provide a more reliable transfer of the classifier from calibration to online control. 
    more » « less
  2. Brain Connectivity (BC) features of multichannel EEG have been proposed for Motor Imagery (MI) decoding in Brain-Computer Interface applications, but the advantages of BC features vs. single-channel features are unclear. Here, we consider three BC features, i.e., Phase Locking Value (PLV), Granger Causality, and weighted Phase Lag Index, and investigate the relationship between the most central nodes in BC-based networks and the most influential EEG channels in single-channel classification based on common spatial pattern filtering. Then, we compare the accuracy of MI decoders that use BC features in source vs. sensor space. Applied to the BCI Competition VI Dataset 2a (left- vs. right-hand MI decoding), our study found that PLV in sensor space achieves the highest classification accuracy among BC features and has similar performance compared to single-channel features, while the transition from sensor to source space reduces the average accuracy of BC features. Across all BC measures, the network topology is similar in left- vs. right-hand MI tasks, and the most central nodes in BC-based networks rarely overlap with the most influential channels in single-channel classification. 
    more » « less
  3. P300-based Brain-Computer Interface (BCI) performance is vulnerable to latency jitter. To investigate the role of latency jitter on BCI system performance, we proposed the classifier-based latency estimation (CBLE) method. In our previous study, CBLE was based on least-squares (LS) and stepwise linear discriminant analysis (SWLDA) classifiers. Here, we aim to extend the CBLE method using sparse autoencoders (SAE) to compare the SAE-based CBLE method with LS- and SWLDA-based CBLE. The newly-developed SAE-based CBLE and previously used methods are also applied to a newly-collected dataset to reduce the possibility of spurious correlations. Our results showed a significant (p<0.001) negative correlation between BCI accuracy and estimated latency jitter. Furthermore, we also examined the effect of the number of electrodes on each classification technique. Our results showed that on the whole, CBLE worked regardless of the classification method and electrode count; by contrast the effect of the number of electrodes on BCI performance was classifier dependent. 
    more » « less
  4. Topological data analysis (TDA) has proven to be a potent approach for extracting intricate topological structures from complex and high-dimensional data. In this paper, we propose a TDA-based processing pipeline for analyzing multi-channel scalp EEG data. The pipeline starts with extracting both frequency and temporal information from the signals via the Hilbert–Huang Transform. The sequences of instantaneous frequency and instantaneous amplitude across all electrode channels are treated as approximations of curves in the high-dimensional space. TDA features, which represent the local topological structure of the curves, are further extracted and used in the classification models. Three sets of scalp EEG data, including one collected in a lab and two Brain–computer Interface (BCI) competition data, were used to validate the proposed methods, and compare with other state-of-art TDA methods. The proposed TDA-based approach shows superior performance and outperform the winner of the BCI competition. Besides BCI, the proposed method can also be applied to spatial and temporal data in other domains such as computer vision, remote sensing, and medical imaging. 
    more » « less
  5. Brain-computer interface (BCI) actively translates the brain signals into executable actions by establishing direct communication between the human brain and external devices. Recording brain activity through electroencephalography (EEG) is generally contaminated with both physiological and nonphysiological artifacts, which significantly hinders the BCI performance. Artifact subspace reconstruction (ASR) is a well-known statistical technique that automatically removes artifact components by determining the rejection threshold based on the initial reference EEG segment in multichannel EEG recordings. In real-world applications, the fixed threshold may limit the efficacy of the artifact correction, especially when the quality of the reference data is poor. This study proposes an adaptive online ASR technique by integrating the Hebbian/anti-Hebbian neural networks into the ASR algorithm, namely, principle subspace projection ASR (PSP-ASR) and principal subspace whitening ASR (PSW-ASR) that segmentwise self-organize the artifact subspace by updating the synaptic weights according to the Hebbian and anti-Hebbian learning rules. The effectiveness of the proposed algorithm is compared to the conventional ASR approaches on benchmark EEG dataset and three BCI frameworks, including steady-state visual evoked potential (SSVEP), rapid serial visual presentation (RSVP), and motor imagery (MI) by evaluating the root-mean-square error (RMSE), the signal-to-noise ratio (SNR), the Pearson correlation, and classification accuracy. The results demonstrated that the PSW-ASR algorithm effectively removed the EEG artifacts and retained the activity-specific brain signals compared to the PSP-ASR, standard ASR (Init-ASR), and moving-window ASR (MW-ASR) methods, thereby enhancing the SSVEP, RSVP, and MI BCI performances. Finally, our empirical results from the PSW-ASR algorithm suggested the choice of an aggressive cutoff range of c = 1-10 for activity-specific BCI applications and a moderate range of for the benchmark dataset and general BCI applications. 
    more » « less