skip to main content


The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, June 13 until 2:00 AM ET on Friday, June 14 due to maintenance. We apologize for the inconvenience.

Title: Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain–body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ;
Publisher / Repository:
Oxford University Press
Date Published:
Journal Name:
PNAS Nexus
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Heart rate has natural fluctuations that are typically ascribed to autonomic function. Recent evidence suggests that conscious processing can affect the timing of the heartbeat. We hypothesized that heart rate is modulated by conscious processing and therefore dependent on attentional focus. To test this, we leverage the observation that neural processes can be synchronized between subjects by presenting an identical narrative stimulus. As predicted, we find significant inter-subject correlation of the heartbeat (ISC-HR) when subjects are presented with an auditory or audiovisual narrative. Consistent with the conscious processing hypothesis, we find that ISC-HR is reduced when subjects are distracted from the narrative, and that higher heart rate synchronization predicts better recall of the narrative. Finally, patients with disorders of consciousness who are listening to a story have lower ISC-HR, as compared to healthy individuals, and that individual ISC-HR might predict a patients’ prognosis.. We conclude that heart rate fluctuations are partially driven by conscious processing, depend on attentional state, and may represent a simple metric to assess conscious state in unresponsive patients. 
    more » « less
  2. Vital signs (e.g., heart and respiratory rate) are indicative for health status assessment. Efforts have been made to extract vital signs using radio frequency (RF) techniques (e.g., Wi-Fi, FMCW, UWB), which offer a non-touch solution for continuous and ubiquitous monitoring without users’ cooperative efforts. While RF-based vital signs monitoring is user-friendly, its robustness faces two challenges. On the one hand, the RF signal is modulated by the periodic chest wall displacement due to heartbeat and breathing in a nonlinear manner. It is inherently hard to identify the fundamental heart and respiratory rates (HR and RR) in the presence of higher order harmonics of them and intermodulation between HR and RR, especially when they have overlapping frequency bands. On the other hand, the inadvertent body movements may disturb and distort the RF signal, overwhelming the vital signals, thus inhibiting the parameter estimation of the physiological movement (i.e., heartbeat and breathing). In this paper, we propose DeepVS, a deep learning approach that addresses the aforementioned challenges from the non-linearity and inadvertent movements for robust RF-based vital signs sensing in a unified manner. DeepVS combines 1D CNN and attention models to exploit local features and temporal correlations. Moreover, it leverages a two-stream scheme to integrate features from both time and frequency domains. Additionally, DeepVS unifies the estimation of HR and RR with a multi-head structure, which only adds limited extra overhead (<1%) to the existing model, compared to doubling the overhead using two separate models for HR and RR respectively. Our experiments demonstrate that DeepVS achieves 80-percentile HR/RR errors of 7.4/4.9 beat/breaths per minute (bpm) on a challenging dataset, as compared to 11.8/7.3 bpm of a non-learning solution. Besides, an ablation study has been conducted to quantify the effectiveness of DeepVS. 
    more » « less
  3. null (Ed.)
    Emotion regulation can be characterized by different activities that attempt to alter an emotional response, whether behavioral, physiological or neurological. The two most widely adopted strategies, cognitive reappraisal and expressive suppression are explored in this study, specifically in the context of disgust. Study participants (N = 21) experienced disgust via video exposure, and were instructed to either regulate their emotions or express them freely. If regulating, they were required to either cognitively reappraise or suppress their emotional experiences while viewing the videos. Video recordings of the participants' faces were taken during the experiment and electrocardiogram (ECG), electromyography (EMG), and galvanic skin response (GSR) readings were also collected for further analysis. We compared the participants behavioral (facial musculature movements) and physiological (GSR and heart rate) responses as they aimed to alter their emotional responses and computationally determined that when responding to disgust stimuli, the signals recorded during suppression and free expression were very similar, whereas those recorded during cognitive reappraisal were significantly different. Thus, in the context of this study, from a signal analysis perspective, we conclude that emotion regulation via cognitive reappraisal significantly alters participants' physiological responses to disgust, unlike regulation via suppression. 
    more » « less
  4. As we comprehend narratives, our attentional engagement fluctuates over time. Despite theoretical conceptions of narrative engagement as emotion-laden attention, little empirical work has characterized the cognitive and neural processes that comprise subjective engagement in naturalistic contexts or its consequences for memory. Here, we relate fluctuations in narrative engagement to patterns of brain coactivation and test whether neural signatures of engagement predict subsequent memory. In behavioral studies, participants continuously rated how engaged they were as they watched a television episode or listened to a story. Self-reported engagement was synchronized across individuals and driven by the emotional content of the narratives. In functional MRI datasets collected as different individuals watched the same show or listened to the same story, engagement drove neural synchrony, such that default mode network activity was more synchronized across individuals during more engaging moments of the narratives. Furthermore, models based on time-varying functional brain connectivity predicted evolving states of engagement across participants and independent datasets. The functional connections that predicted engagement overlapped with a validated neuromarker of sustained attention and predicted recall of narrative events. Together, our findings characterize the neural signatures of attentional engagement in naturalistic contexts and elucidate relationships among narrative engagement, sustained attention, and event memory. 
    more » « less
  5. Abstract Objective

    The factors that influence seizure timing are poorly understood, and seizure unpredictability remains a major cause of disability. Work in chronobiology has shown that cyclical physiological phenomena are ubiquitous, with daily and multiday cycles evident in immune, endocrine, metabolic, neurological, and cardiovascular function. Additionally, work with chronic brain recordings has identified that seizure risk is linked to daily and multiday cycles in brain activity. Here, we provide the first characterization of the relationships between the cyclical modulation of a diverse set of physiological signals, brain activity, and seizure timing.


    In this cohort study, 14 subjects underwent chronic ambulatory monitoring with a multimodal wrist‐worn sensor (recording heart rate, accelerometry, electrodermal activity, and temperature) and an implanted responsive neurostimulation system (recording interictal epileptiform abnormalities and electrographic seizures). Wavelet and filter–Hilbert spectral analyses characterized circadian and multiday cycles in brain and wearable recordings. Circular statistics assessed electrographic seizure timing and cycles in physiology.


    Ten subjects met inclusion criteria. The mean recording duration was 232 days. Seven subjects had reliable electroencephalographic seizure detections (mean = 76 seizures). Multiday cycles were present in all wearable device signals across all subjects. Seizure timing was phase locked to multiday cycles in five (temperature), four (heart rate, phasic electrodermal activity), and three (accelerometry, heart rate variability, tonic electrodermal activity) subjects. Notably, after regression of behavioral covariates from heart rate, six of seven subjects had seizure phase locking to the residual heart rate signal.


    Seizure timing is associated with daily and multiday cycles in multiple physiological processes. Chronic multimodal wearable device recordings can situate rare paroxysmal events, like seizures, within a broader chronobiology context of the individual. Wearable devices may advance the understanding of factors that influence seizure risk and enable personalized time‐varying approaches to epilepsy care.

    more » « less