Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain–body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.
Quantifying VR cybersickness using EEG
Current techniques for characterizing cybersickness (visually induced motion sickness) in virtual environments rely on qualitative questionnaires. For interactive graphics to create visual experiences that enhance the illusion of presence while mitigating cybersickness, interactive measures are needed to characterize cybersickness. In this paper, we acquire EEG signals from participants as they experience vection-induced cybersickness and compare those signals to a baseline. Our study shows that there is a correlation between the participant-reported cybersickness (as measured by movements of a joystick) and brain EEG signals. Through independent component analysis, we separate those signals which are a result of cybersickness from other sources (such as eye blinks). Our user study finds that there is a highly correlative and statistically significant Delta- (1.0–4.0 Hz), Theta- (4.0–7.0 Hz), and Alpha-wave (7.0–13.0 Hz) increase associated with cybersickness in immersive virtual environments across participants. Establishing a strong correlation between cybersickness and EEG-measured brain activity provides us with the first step toward interactively characterizing and mitigating cybersickness in virtual environments.
more »
« less
- Award ID(s):
- 1823321
- PAR ID:
- 10280498
- Date Published:
- Journal Name:
- Virtual Reality
- ISSN:
- 1359-4338
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
Immersion into a virtual environment (VE) goften results in adverse symptoms including nausea, dizziness, and disorientation. These symptoms are an indicator of cybersickness,which is a condition similar to motion sickness experienced in VEs. In this paper, we hypothesized that administered cognitive distraction can accelerate the rate of habituation to a VE. This acceleration, therefore, can lower severity of cybersickness in fewer amount of immersions. To evaluate the impact of cognitive distraction on reducing the effects of cybersickness, we designed a VE and carried out a human subject study with control and experimental groups created through stratified random sampling.Subjects were immersed in our VE on four separate sessions, and our experimental group received cognitive distraction throughout the immersions. Cybersickness was measured using the Simulator Sickness Questionnaire (SSQ) and Presence Questionnaire (PQ). Upon comparing the average SSQ subgroups nausea, oculomotor, and disorientation scores reported by participants for each immersion session, we observed that our experimental group exhibited decrease in cybersickness to a greater extent than that of our control group. We completed t-tests for each of these comparisons, to find that these results are statistically insignificant. We plan to continue with this work by incorporating up to 30 total participants to clarify these findings.more » « less
-
This literature review examines the existing research into cybersickness reduction with regards to head mounted display use. Cybersickness refers to a collection of negative symptoms sometimes experienced as the result of being immersed in a virtual environment, such as nausea, dizziness, or eye strain. These symptoms can prevent individuals from utilizing virtual reality (VR) technologies, so discovering new methods of reducing them is critical. Our objective in this literature review is to provide a better picture of what cybersickness reduction techniques exist, the quantity of research demonstrating their effectiveness, and the virtual scenes testing has taken place in. This will help to direct researches towards promising avenues, and illuminate gaps in the literature. Following the preferred reporting items for systematic reviews and meta-analyses statement, we obtained a batch of 1,055 papers through the use of software aids. We selected 88 papers that examine potential cybersickness reduction approaches. Our acceptance criteria required that papers examined malleable conditions that could be conceivably modified for everyday use, examined techniques in conjunction with head mounted displays, and compared cybersickness levels between two or more user conditions. These papers were sorted into categories based on their general approach to combating cybersickness, and labeled based on the presence of statistically significant results, the use of virtual vehicles, the level of visual realism, and the virtual scene contents used in evaluation of their effectiveness. In doing this we have created a snapshot of the literature to date so that researchers may better understand what approaches are being researched, and the types of virtual experiences used in their evaluation. Keywords: Virtual reality cybersickness Simulator Sickness Visually induced motion sickness reduction Systematic review Head mounted display.more » « less
-
The prospect of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) in the presence of topological information of participants is often left unexplored in most of the brain-computer interface (BCI) systems. Additionally, the usage of these modalities together in the field of multimodality analysis to support multiple brain signals toward improving BCI performance is not fully examined. This study first presents a multimodal data fusion framework to exploit and decode the complementary synergistic properties in multimodal neural signals. Moreover, the relations among different subjects and their observations also play critical roles in classifying unknown subjects. We developed a context-aware graph neural network (GNN) model utilizing the pairwise relationship among participants to investigate the performance on an auditory task classification. We explored standard and deviant auditory EEG and fNIRS data where each subject was asked to perform an auditory oddball task and has multiple trials regarded as context-aware nodes in our graph construction. In experiments, our multimodal data fusion strategy showed an improvement up to 8.40% via SVM and 2.02% via GNN, compared to the single-modal EEG or fNIRS. In addition, our context-aware GNN achieved 5.3%, 4.07% and 4.53% higher accuracy for EEG, fNIRS and multimodal data based experiments, compared to the baseline models.more » « less
-
Driven by the increasing complexity of built environments, firefighters are often exposed to extensive wayfinding information which could cause high cognitive load and ineffective or even dangerous decision making. To reduce injuries and fatal incidents in firefighters’ line of duty, this study aims at measuring the cognitive load and identifying the source of such cognitive overload in wayfinding information review. We developed a Sternberg Test to induce cognitive load on participants pertaining to working memory development, where participants were required to memorize colors, letters, numbers, directions, icons, words, and letter combinations that are relevant to wayfinding tasks. We used an Electroencephalogram (EEG) device to monitor neural activities especially in frontal, parietal, and occipital areas of brain. The fast Fourier transformation (FFT) was applied to separate the sub-band energy. The speed of response in Sternberg Test and the EEG signals were compared to show the coherence between the results of the two methods in representing the cognitive load in the review test. Results indicate that the cognitive load arises from diverse information can be measured to help customize wayfinding information for controlled cognitive load of firefighters in wayfinding tasks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1007/s10055-021-00517-2
