skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Using Developer Eye Movements to Externalize the Mental Model Used in Code Summarization Tasks
cognition model (i.e., bottom-up or top-down) applied during program comprehension tasks. The cognition models examine how programmers understand source code by describing the temporary information structures in the programmer’s short term memory. The two types of models that we are interested in are top-down and bottom-up. The top-down model is normally applied as-needed (i.e., the domain of the system is familiar). The bottom-up model is typically applied when a developer is not familiar with the domain or the source code. An eye-tracking study of 18 developers reading and summarizing Java methods is used as our dataset for analyzing the mental cognition model. The developers provide a written summary for methods assigned to them. In total, 63 methods are used from five different systems. The results indicate that on average, experts and novices read the methods more closely (using the bottom-up mental model) than bouncing around (using top-down). However, on average novices spend longer gaze time performing bottom-up (66s.) compared to experts (43s.)  more » « less
Award ID(s):
1730181
PAR ID:
10148765
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
11th ACM Symposium on Eye tracking Research and Applications (ETRA)
Page Range / eLocation ID:
9 pages
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, ACM/IEEE International Conference on Software Engineering (ICSE))
    An eye-tracking study of 18 developers reading and summarizing Java methods is presented. The developers provide a written summary for methods assigned to them. In total, 63 methods are used from five different systems. Previous studies on this topic use only short methods presented in isolation usually as images. In contrast, this work presents the study in the Eclipse IDE allowing access to all the source code in the system. The developer can navigate via scrolling and switching files while writing the summary. New eye-tracking infrastructure allows for this improvement in the study environment. Data collected includes eye gazes on source code, written summaries, and time to complete each summary. Unlike prior work that concluded developers focus on the signature the most, these results indicate that they tend to focus on the method body more than the signature. Moreover, both experts and novices tend to revisit control flow terms rather than reading them for a long period. They also spend a significant amount of gaze time and have higher gaze visits when they read call terms. Experts tend to revisit the body of the method significantly more frequently than its signature as the size of the method increases. Moreover, experts tend to write their summaries from source code lines that they read the most. 
    more » « less
  2. ; ; (, Proceedings of the ACM on Human-Computer Interaction)
    Understanding players' mental models are crucial for game designers who wish to successfully integrate player-AI interactions into their game. However, game designers face the difficult challenge of anticipating how players model these AI agents during gameplay and how they may change their mental models with experience. In this work, we conduct a qualitative study to examine how a pair of players develop mental models of an adversarial AI player during gameplay in the multiplayer drawing game iNNk. We conducted ten gameplay sessions in which two players (n = 20, 10 pairs) worked together to defeat an AI player. As a result of our analysis, we uncovered two dominant dimensions that describe players' mental model development (i.e., focus and style). The first dimension describes the focus of development which refers to what players pay attention to for the development of their mental model (i.e., top-down vs. bottom-up focus). The second dimension describes the differences in the style of development, which refers to how players integrate new information into their mental model (i.e., systematic vs. reactive style). In our preliminary framework, we further note how players process a change when a discrepancy occurs, which we observed occur through comparisons (i.e., compare to other systems, compare to gameplay, compare to self). We offer these results as a preliminary framework for player mental model development to help game designers anticipate how different players may model adversarial AI players during gameplay. 
    more » « less
  3. ; ; ; ; ; ; ; ; ; ; et al (, Global Biogeochemical Cycles)
    Abstract Large stocks of soil carbon (C) and nitrogen (N) in northern permafrost soils are vulnerable to remobilization under climate change. However, there are large uncertainties in present‐day greenhouse gas (GHG) budgets. We compare bottom‐up (data‐driven upscaling and process‐based models) and top‐down (atmospheric inversion models) budgets of carbon dioxide (CO2), methane (CH4) and nitrous oxide (N2O) as well as lateral fluxes of C and N across the region over 2000–2020. Bottom‐up approaches estimate higher land‐to‐atmosphere fluxes for all GHGs. Both bottom‐up and top‐down approaches show a sink of CO2in natural ecosystems (bottom‐up: −29 (−709, 455), top‐down: −587 (−862, −312) Tg CO2‐C yr−1) and sources of CH4(bottom‐up: 38 (22, 53), top‐down: 15 (11, 18) Tg CH4‐C yr−1) and N2O (bottom‐up: 0.7 (0.1, 1.3), top‐down: 0.09 (−0.19, 0.37) Tg N2O‐N yr−1). The combined global warming potential of all three gases (GWP‐100) cannot be distinguished from neutral. Over shorter timescales (GWP‐20), the region is a net GHG source because CH4dominates the total forcing. The net CO2sink in Boreal forests and wetlands is largely offset by fires and inland water CO2emissions as well as CH4emissions from wetlands and inland waters, with a smaller contribution from N2O emissions. Priorities for future research include the representation of inland waters in process‐based models and the compilation of process‐model ensembles for CH4and N2O. Discrepancies between bottom‐up and top‐down methods call for analyses of how prior flux ensembles impact inversion budgets, more and well‐distributed in situ GHG measurements and improved resolution in upscaling techniques. 
    more » « less
  4. ; (, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    The mental demands associated with operating complex whole-body powered exoskeletons are poorly understood. This study aimed to explore the overall workload associated with using a powered wholebody exoskeleton among expert and novice users, as well as the changes in workload resulting from novices adapting to exoskeleton-use over time. We used eye-tracking measures to quantify the differences in workload of six novices and five experts while they performed a levelwalking task, with and without wearing a whole-body powered exoskeleton. We found that only novices’ pupil dilation (PD) increased, while experts showed a greater proportion of downward-directed pathfixations (PF) compared to novices while wearing the exoskeleton. These results indicate that novices’ mental demands were higher, and that experts and novices exhibited distinct visuomotor strategies. Eyetracking measures may potentially be used to detect differences in workload and skill-level associated with using exoskeletons, and also considered as inputs for future adaptive exoskeleton control algorithms. 
    more » « less
  5. ; ; ; ; (, 11th ACM Symposium on Eye tracking Research and Applications (ETRA))
    Busjahn et al. [4] on the factors influencing dwell time during source code reading, where source code element type and frequency of gaze visits are studied as factors. Unlike the previous study, this study focuses on analyzing eye movement data in large open source Java projects. Five experts and thirteen novices participated in the study where the main task is to summarize methods. The results examine semantic line-level information that developers view during summarization. We find no correlation between the line length and the total duration of time spent looking on the line even though it exists between a token’s length and the total fixation time on the token reported in prior work. The first fixations inside a method are more likely to be on a method’s signature, a variable declaration, or an assignment compared to the other fixations inside a method. In addition, it is found that smaller methods tend to have shorter overall fixation duration for the entire method, but have significantly longer duration per line in the method. The analysis provides insights into how source code’s unique characteristics can help in building more robust methods for analyzing eye movements in source code and overall in building theories to support program comprehension on realistic tasks. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email