skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: A Recurrent Neural Network for Attenuating Non-cognitive Components of Pupil Dynamics
There is increasing interest in how the pupil dynamics of the eye reflect underlying cognitive processes and brain states. Problematic, however, is that pupil changes can be due to non-cognitive factors, for example luminance changes in the environment, accommodation and movement. In this paper we consider how by modeling the response of the pupil in real-world environments we can capture the non-cognitive related changes and remove these to extract a residual signal which is a better index of cognition and performance. Specifically, we utilize sequence measures such as fixation position, duration, saccades, and blink-related information as inputs to a deep recurrent neural network (RNN) model for predicting subsequent pupil diameter. We build and evaluate the model for a task where subjects are watching educational videos and subsequently asked questions based on the content. Compared to commonly-used models for this task, the RNN had the lowest errors rates in predicting subsequent pupil dilation given sequence data. Most importantly was how the model output related to subjects' cognitive performance as assessed by a post-viewing test. Consistent with our hypothesis that the model captures non-cognitive pupil dynamics, we found (1) the model's root-mean square error was less for lower performing subjects than for those having better performance on the post-viewing test, (2) the residuals of the RNN (LSTM) model had the highest correlation with subject post-viewing test scores and (3) the residuals had the highest discriminability (assessed via area under the ROC curve, AUC) for classifying high and low test performers, compared to the true pupil size or the RNN model predictions. This suggests that deep learning sequence models may be good for separating components of pupil responses that are linked to luminance and accommodation from those that are linked to cognition and arousal.  more » « less
Award ID(s):
1816363
PAR ID:
10295744
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Frontiers in Psychology
Volume:
12
ISSN:
1664-1078
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Journal of Ophthalmology)
    Costagliola, Ciro (Ed.)
    Purpose. This study investigated how a conscious change in ocular accommodation affects intraocular pressure (IOP) and ocular biometrics in healthy adult volunteers of different ages. Methods. Thirty-five healthy volunteers without ocular disease or past ocular surgery, and with refractive error between −3.50 and +2.50 diopters, were stratified into 20, 40, and 60 year old (y.o.) age groups. Baseline measurements of central cornea thickness, anterior chamber depth, anterior chamber angle, cornea diameter, pupil size, and ciliary muscle thickness were made by autorefraction and optical coherence tomography (OCT), while IOP was measured by pneumotonometry. Each subject’s right eye focused on a target 40 cm away. Three different tests were performed in random order: (1) 10 minutes of nonaccommodation (gazing at the target through lenses that allowed clear vision without accommodating), (2) 10 minutes of accommodation (addition of a minus 3 diopter lens), and (3) 10 minutes of alternating between accommodation and nonaccommodation (1-minute intervals). IOP was measured immediately after each test. A 20-minute rest period was provided between tests. Data from 31 subjects were included in the study. ANOVA and paired t-tests were used for statistical analyses. Results. Following alternating accommodation, IOP decreased by 0.7 mmHg in the right eye when all age groups were combined ( p  = 0.029). Accommodation or nonaccommodation alone did not decrease IOP. Compared to the 20 y.o. group, the 60 y.o. group had a thicker ciliary muscle within 75 μm of the scleral spur, a thinner ciliary muscle at 125–300 μm from the scleral spur, narrower anterior chamber angles, shallower anterior chambers, and smaller pupils during accommodation and nonaccommodation ( p ’s < 0.01). Conclusion. Alternating accommodation, but not constant accommodation, significantly decreased IOP. This effect was not lost with aging despite physical changes to the aging eye. A greater accommodative workload and/or longer test period may improve the effect. 
    more » « less
  2. ; (, eLife)
    Making a categorical judgment can systematically bias our subsequent perception of the world. We show that these biases are well explained by a self-consistent Bayesian observer whose perceptual inference process is causally conditioned on the preceding choice. We quantitatively validated the model and its key assumptions with a targeted set of three psychophysical experiments, focusing on a task sequence where subjects first had to make a categorical orientation judgment before estimating the actual orientation of a visual stimulus. Subjects exhibited a high degree of consistency between categorical judgment and estimate, which is difficult to reconcile with alternative models in the face of late, memory related noise. The observed bias patterns resemble the well-known changes in subjective preferences associated with cognitive dissonance, which suggests that the brain’s inference processes may be governed by a universal self-consistency constraint that avoids entertaining ‘dissonant’ interpretations of the evidence. 
    more » « less
  3. ; ; (, eneuro)
    Abrupt onsets reflexively shift covert spatial attention. Recent work demonstrated that trial-to-trial information about the probability of a peripheral onset modulated the magnitude of the attentional cueing effect (low probability > high probability). Although onsets were physically identical, pupil responses could have been modulated by information about the probability of the onset's appearance. Specifically, anticipatory constrictions may have preceded high-probability onsets. Here, we tested this hypothesis using centrally presented, luminance-matched onset-probability signals. For half the participants, vertical signaled high probability (0.8) of onset appearance, while horizontal signaled low probability (0.2). Contingencies were reversed for the other half. Participants fixated the onset-probability signal for 2,000 ms before the onset was briefly presented or omitted, in line with the signaled probability. To maintain engagement, participants completed a simple localization task. Preliminary evidence for an anticipatory reduction in the pupil area was obtained in Experiment 1. However, this effect disappeared in Experiment 2 with a larger replication sample. Exploratory analyses uncovered a violation of a fundamental methodological assumption: despite being task-irrelevant and perfectly luminance-matched, vertical onset-probability signals consistently generated smaller pupil areas, relative to horizontal signals in both Experiments 1 and 2. Interestingly, this “orientation effect” was stronger in the second half of the experimental session, and in a third experiment, we significantly reduced its magnitude by changing the locations of the task-relevant stimuli. In short, across three experiments (self-reported gender, 52 females, 26 males, 1 nonbinary), we show that even with perfect luminance matching, unforeseen changes in cognitive state can modulate pupillometric measurements. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; (, Human Brain Mapping)
    ABSTRACT With the increasing availability of large‐scale multimodal neuroimaging datasets, it is necessary to develop data fusion methods which can extract cross‐modal features. A general framework, multidataset independent subspace analysis (MISA), has been developed to encompass multiple blind source separation approaches and identify linked cross‐modal sources in multiple datasets. In this work, we utilized the multimodal independent vector analysis (MMIVA) model in MISA to directly identify meaningful linked features across three neuroimaging modalities—structural magnetic resonance imaging (MRI), resting state functional MRI and diffusion MRI—in two large independent datasets, one comprising of control subjects and the other including patients with schizophrenia. Results show several linked subject profiles (sources) that capture age‐associated decline, schizophrenia‐related biomarkers, sex effects, and cognitive performance. For sources associated with age, both shared and modality‐specific brain‐age deltas were evaluated for association with non‐imaging variables. In addition, each set of linked sources reveals a corresponding set of cross‐modal spatial patterns that can be studied jointly. We demonstrate that the MMIVA fusion model can identify linked sources across multiple modalities, and that at least one set of linked, age‐related sources replicates across two independent and separately analyzed datasets. The same set also presented age‐adjusted group differences, with schizophrenia patients indicating lower multimodal source levels. Linked sets associated with sex and cognition are also reported for the UK Biobank dataset. 
    more » « less
  5. ; ; (, Journal of computing in civil engineering)
    null (Ed.)
    Amid the rapid development of building information technologies, wayfinding information has become more accessible to building users and first responders. As a result, a realistic risk of cognitive load related to the wayfinding information processing starts to emerge. As cognition-driven adaptive wayfinding information systems become increasingly captivated to overcome challenges of cognition overload due to overwhelming information, a practical and non-invasive method to monitor and classify cognitive loads during the processing of wayfinding information is needed. This paper tests a Functional Near-Infrared Spectroscopy (fNIRS) based method to identify cognitive load related to wayfinding information processing. It provides a holistic fNIRS signal analytical pipeline to extract hemodynamic response features in the prefrontal cortex (PFC) for cognitive load classification. A human-subject experiment (N=15) based on the Sternberg working memory test was performed to model the relationship between fNIRS features and cognitive load. Personalized models were also evaluated to capture individual differences and identify unique contributing features to each person. The results find that fNIRS-based model can help classify cognitive load changes driven by the different levels of task difficulty with satisfactory performance (avg. accuracy rate 70.02±4.41 percent). The findings also demonstrate that personalized models, instead of universal models, are needed for classifying cognitive load based on neuroimaging data. fNIRS has demonstrated comparable advantages over other neuroimaging methods in cognitive load classification given its robustness to motion artifacts and the satisfactory predictability. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email