An official website of the United States government
Background: Researchers have recently started to validate decades-old program-comprehension models using functional magnetic resonance imaging (fMRI). While fMRI helps us to understand neural correlates of cognitive processes during program comprehension, its comparatively low temporal resolution (i.e., seconds) cannot capture fast cognitive subprocesses (i.e., milliseconds). Aims: To increase the explanatory power of fMRI measurement of programmers, we are exploring in this methodological paper the feasibility of adding simultaneous eye tracking to fMRI measurement. By developing a method to observe programmers with two complementary measures, we aim at obtaining a more comprehensive understanding of program comprehension. Method: We conducted a controlled fMRI experiment of 22 student participants with simultaneous eye tracking. Results: We have been able to successfully capture fMRI and eye-tracking data, although with limitations regarding partial data loss and spatial imprecision. The biggest issue that we experienced is the partial loss of data: for only 10 participants, we could collect a complete set of high-precision eye-tracking data. Since some participants of fMRI studies show excessive head motion, the proportion of full and high-quality data on fMRI and eye tracking is rather low. Still, the remaining data allowed us to confirm our prior hypothesis of semantic recall during program comprehension, which was not possible with fMRI alone. Conclusions: Simultaneous measurement of program comprehension with fMRI and eye tracking is promising, but with limitations. By adding simultaneous eye tracking to our fMRI study framework, we can conduct more fine-grained fMRI analyses, which in turn helps us to understand programmer behavior better.
more »
« less
Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension
Researchers have been employing psycho-physiological measures to better understand program comprehension, for example simultaneous fMRI and eye tracking to validate top-down comprehension models. In this paper, we argue that there is additional value in eye-tracking data beyond eye gaze: Pupil dilation and blink rates may offer insights into programmers' cognitive load. However, the fMRI environment may influence pupil dilation and blink rates, which would diminish their informative value. We conducted a preliminary analysis of pupil dilation and blink rates of an fMRI experiment with 22 student participants. We conclude from our preliminary analysis that the correction for our fMRI environment is challenging, but possible, such that we can use pupil dilation and blink rates to more reliably observe program comprehension.
more »
« less
- Award ID(s):
- 1755762
- PAR ID:
- 10085742
- Date Published:
- Journal Name:
- Proceedings of the Workshop on Eye Movements in Programming (EMIP'18)
- Page Range / eLocation ID:
- 1 to 5
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
After decades of research, there is still no comprehensive, validated model of program comprehension. Recently, researchers have been applying psycho-physiological measures to expand our understanding of program comprehension. In this position paper, we argue that measuring program comprehension simultaneously with functional magnetic resonance imaging (fMRI) and eye tracking is promising. However, due to the different nature of both measures in terms of response delay and temporal resolution, we need to develop suitable tools. We describe the challenges of conjoint analysis of fMRI and eye-tracking data, and we also outline possible solution strategies.more » « less
-
This research aims to explore the prediction of student learning outcomes in Augmented Reality (AR) educational settings, focusing on engineering education, by analyzing pupil dilation and problem-solving time as key indicators. In this research, we have created an innovative AR learning platform through the incorporation of eye-tracking technology into the Microsoft HoloLens 2. This enhanced learning platform enables the collection of data on pupil dilation and problem-solving duration as students engage in AR-based learning activities. In this study, we hypothesize that pupil dilation and problem-solving time could be significant predictors of student performance in the AR learning environment. The results of our study suggest that problem-solving time may be a critical factor in predicting student learning success for materials involving procedural knowledge at low difficulty levels. Additionally, both pupil dilation and problem-solving time are predictive indicators of student learning outcomes when dealing with predominantly procedural knowledge at high difficulty levels.more » « less
-
null (Ed.)There is increasing interest in how the pupil dynamics of the eye reflect underlying cognitive processes and brain states. Problematic, however, is that pupil changes can be due to non-cognitive factors, for example luminance changes in the environment, accommodation and movement. In this paper we consider how by modeling the response of the pupil in real-world environments we can capture the non-cognitive related changes and remove these to extract a residual signal which is a better index of cognition and performance. Specifically, we utilize sequence measures such as fixation position, duration, saccades, and blink-related information as inputs to a deep recurrent neural network (RNN) model for predicting subsequent pupil diameter. We build and evaluate the model for a task where subjects are watching educational videos and subsequently asked questions based on the content. Compared to commonly-used models for this task, the RNN had the lowest errors rates in predicting subsequent pupil dilation given sequence data. Most importantly was how the model output related to subjects' cognitive performance as assessed by a post-viewing test. Consistent with our hypothesis that the model captures non-cognitive pupil dynamics, we found (1) the model's root-mean square error was less for lower performing subjects than for those having better performance on the post-viewing test, (2) the residuals of the RNN (LSTM) model had the highest correlation with subject post-viewing test scores and (3) the residuals had the highest discriminability (assessed via area under the ROC curve, AUC) for classifying high and low test performers, compared to the true pupil size or the RNN model predictions. This suggests that deep learning sequence models may be good for separating components of pupil responses that are linked to luminance and accommodation from those that are linked to cognition and arousal.more » « less
-
Abstract In childhood, higher levels of temperamental fear—an early‐emerging proclivity to distress in the face of novelty—are associated with lower social responsivity and greater social anxiety. While the early emergence of temperamental fear in infancy is poorly understood, it is theorized to be driven by individual differences in reactivity and self‐regulation to novel stimuli. The current study used eye tracking to capture infants’ (N = 124) reactions to a video of a smiling stranger—a common social encounter—including infant gaze aversions from the stranger's face (indexing arousal regulation) and pupil dilation (indexing physiological reactivity), longitudinally at 2, 4, 6, and 8 months of age. Multilevel mixed‐effects models indicated that more fearful infants took more time to look away from a smiling stranger's face than less fearful infants, suggesting that high‐fear infants may have slower arousal regulation. At 2 and 4 months, more fearful infants also exhibited greater and faster pupil dilation before gaze aversions, consistent with greater physiological reactivity. Together, these findings suggest that individual differences in infants’ gaze aversions and pupil dilation can index the development of fearful temperament in early infancy, facilitating the identification of, and interventions for, risk factors to social disruptions.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3216723.3216726
