Search for: All records

Neural networks (NN) has been adopted by brain-computer interfaces (BCI) to encode brain signals acquired using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). However, it has been found that NN models are vulnerable to adversarial examples, i.e., corrupted samples with imperceptible noise. Once attacked, it could impact medical diagnosis and patients’ quality of life. While early work focuses on interference using external devices at the time of signal acquisition, recent research shifts to collected signals, features, and learning models under various attack modes (e.g., white-, grey-, and black-box). However, existing work only considers single-modality attacks and ignores the topological relationships among different observations, e.g., samples having strong similarities. Different from previous approaches, we introduce graph neural networks (GNN) to multimodal BCI-based classification and explore its performance and robustness against adversarial attacks. This study will evaluate the robustness of NN models with and without graph knowledge on both single and multimodal data. 

Homology of human and machine vision systems demonstrates that better machine could be designed with human assistance. Similar components can be mapped from neuroimaging data to visual features for recognizing an object. However, inferring object relationships from human vision and machine vision are not clear. To measure the similarity of human and machine visual inference, this work study an inference method using Microsoft COCO dataset. The input data is manually generated, and used for a java-based inference engine, which collects semantic data in a co-occurrence matrix, and writes the data to a knowledge graph in the DOT language. Unlike the black-box property of deep neural network, the proposed method is transparent. When rendered by GraphViz tools, the visible results in the knowledge graph indicated that the COCO dataset-based machine inference is promising when compared to human inference, yielding an accuracy of 64% at best. This novel inference study on the COCO dataset reveals that homology of human and machine vision systems is promising to be bridged. Bigger dataset and more concepts may increase the accuracy in the future work.