An eye-tracking study of 18 developers reading and summarizing Java methods is presented. The developers provide a written summary for methods assigned to them. In total, 63 methods are used from five different systems. Previous studies on this topic use only short methods presented in isolation usually as images. In contrast, this work presents the study in the Eclipse IDE allowing access to all the source code in the system. The developer can navigate via scrolling and switching files while writing the summary. New eye-tracking infrastructure allows for this improvement in the study environment. Data collected includes eye gazes on source code, written summaries, and time to complete each summary. Unlike prior work that concluded developers focus on the signature the most, these results indicate that they tend to focus on the method body more than the signature. Moreover, both experts and novices tend to revisit control flow terms rather than reading them for a long period. They also spend a significant amount of gaze time and have higher gaze visits when they read call terms. Experts tend to revisit the body of the method significantly more frequently than its signature as the size of the method increases. Moreover, experts tend to write their summaries from source code lines that they read the most.
more »
« less
Using Developer Eye Movements to Externalize the Mental Model Used in Code Summarization Tasks
cognition model (i.e., bottom-up or top-down) applied during program comprehension tasks. The cognition models examine how programmers understand source code by describing the temporary information structures in the programmer’s short term memory. The two types of models that we are interested in are top-down and bottom-up. The top-down model is normally applied as-needed (i.e., the domain of the system is familiar). The bottom-up model is typically applied when a developer is not familiar with the domain or the source code. An eye-tracking study of 18 developers reading and summarizing Java methods is used as our dataset for analyzing the mental cognition model. The developers provide a written summary for methods assigned to them. In total, 63 methods are used from five different systems. The results indicate that on average, experts and novices read the methods more closely (using the bottom-up mental model) than bouncing around (using top-down). However, on average novices spend longer gaze time performing bottom-up (66s.) compared to experts (43s.)
more »
« less
- Award ID(s):
- 1730181
- PAR ID:
- 10148765
- Date Published:
- Journal Name:
- 11th ACM Symposium on Eye tracking Research and Applications (ETRA)
- Page Range / eLocation ID:
- 9 pages
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Understanding players' mental models are crucial for game designers who wish to successfully integrate player-AI interactions into their game. However, game designers face the difficult challenge of anticipating how players model these AI agents during gameplay and how they may change their mental models with experience. In this work, we conduct a qualitative study to examine how a pair of players develop mental models of an adversarial AI player during gameplay in the multiplayer drawing game iNNk. We conducted ten gameplay sessions in which two players (n = 20, 10 pairs) worked together to defeat an AI player. As a result of our analysis, we uncovered two dominant dimensions that describe players' mental model development (i.e., focus and style). The first dimension describes the focus of development which refers to what players pay attention to for the development of their mental model (i.e., top-down vs. bottom-up focus). The second dimension describes the differences in the style of development, which refers to how players integrate new information into their mental model (i.e., systematic vs. reactive style). In our preliminary framework, we further note how players process a change when a discrepancy occurs, which we observed occur through comparisons (i.e., compare to other systems, compare to gameplay, compare to self). We offer these results as a preliminary framework for player mental model development to help game designers anticipate how different players may model adversarial AI players during gameplay.more » « less
-
Abstract Large stocks of soil carbon (C) and nitrogen (N) in northern permafrost soils are vulnerable to remobilization under climate change. However, there are large uncertainties in present‐day greenhouse gas (GHG) budgets. We compare bottom‐up (data‐driven upscaling and process‐based models) and top‐down (atmospheric inversion models) budgets of carbon dioxide (CO2), methane (CH4) and nitrous oxide (N2O) as well as lateral fluxes of C and N across the region over 2000–2020. Bottom‐up approaches estimate higher land‐to‐atmosphere fluxes for all GHGs. Both bottom‐up and top‐down approaches show a sink of CO2in natural ecosystems (bottom‐up: −29 (−709, 455), top‐down: −587 (−862, −312) Tg CO2‐C yr−1) and sources of CH4(bottom‐up: 38 (22, 53), top‐down: 15 (11, 18) Tg CH4‐C yr−1) and N2O (bottom‐up: 0.7 (0.1, 1.3), top‐down: 0.09 (−0.19, 0.37) Tg N2O‐N yr−1). The combined global warming potential of all three gases (GWP‐100) cannot be distinguished from neutral. Over shorter timescales (GWP‐20), the region is a net GHG source because CH4dominates the total forcing. The net CO2sink in Boreal forests and wetlands is largely offset by fires and inland water CO2emissions as well as CH4emissions from wetlands and inland waters, with a smaller contribution from N2O emissions. Priorities for future research include the representation of inland waters in process‐based models and the compilation of process‐model ensembles for CH4and N2O. Discrepancies between bottom‐up and top‐down methods call for analyses of how prior flux ensembles impact inversion budgets, more and well‐distributed in situ GHG measurements and improved resolution in upscaling techniques.more » « less
-
The mental demands associated with operating complex whole-body powered exoskeletons are poorly understood. This study aimed to explore the overall workload associated with using a powered wholebody exoskeleton among expert and novice users, as well as the changes in workload resulting from novices adapting to exoskeleton-use over time. We used eye-tracking measures to quantify the differences in workload of six novices and five experts while they performed a levelwalking task, with and without wearing a whole-body powered exoskeleton. We found that only novices’ pupil dilation (PD) increased, while experts showed a greater proportion of downward-directed pathfixations (PF) compared to novices while wearing the exoskeleton. These results indicate that novices’ mental demands were higher, and that experts and novices exhibited distinct visuomotor strategies. Eyetracking measures may potentially be used to detect differences in workload and skill-level associated with using exoskeletons, and also considered as inputs for future adaptive exoskeleton control algorithms.more » « less
-
Abstract. Understanding and quantifying the global methane (CH4) budgetis important for assessing realistic pathways to mitigate climate change.Atmospheric emissions and concentrations of CH4 continue to increase,making CH4 the second most important human-influenced greenhouse gas interms of climate forcing, after carbon dioxide (CO2). The relativeimportance of CH4 compared to CO2 depends on its shorteratmospheric lifetime, stronger warming potential, and variations inatmospheric growth rate over the past decade, the causes of which are stilldebated. Two major challenges in reducing uncertainties in the atmosphericgrowth rate arise from the variety of geographically overlapping CH4sources and from the destruction of CH4 by short-lived hydroxylradicals (OH). To address these challenges, we have established aconsortium of multidisciplinary scientists under the umbrella of the GlobalCarbon Project to synthesize and stimulate new research aimed at improvingand regularly updating the global methane budget. Following Saunois et al. (2016), we present here the second version of the living review paperdedicated to the decadal methane budget, integrating results of top-downstudies (atmospheric observations within an atmospheric inverse-modellingframework) and bottom-up estimates (including process-based models forestimating land surface emissions and atmospheric chemistry, inventories ofanthropogenic emissions, and data-driven extrapolations). For the 2008–2017 decade, global methane emissions are estimated byatmospheric inversions (a top-down approach) to be 576 Tg CH4 yr−1 (range 550–594, corresponding to the minimum and maximumestimates of the model ensemble). Of this total, 359 Tg CH4 yr−1 or∼ 60 % is attributed to anthropogenic sources, that isemissions caused by direct human activity (i.e. anthropogenic emissions; range 336–376 Tg CH4 yr−1 or 50 %–65 %). The mean annual total emission for the new decade (2008–2017) is29 Tg CH4 yr−1 larger than our estimate for the previous decade (2000–2009),and 24 Tg CH4 yr−1 larger than the one reported in the previousbudget for 2003–2012 (Saunois et al., 2016). Since 2012, global CH4emissions have been tracking the warmest scenarios assessed by theIntergovernmental Panel on Climate Change. Bottom-up methods suggest almost30 % larger global emissions (737 Tg CH4 yr−1, range 594–881)than top-down inversion methods. Indeed, bottom-up estimates for naturalsources such as natural wetlands, other inland water systems, and geologicalsources are higher than top-down estimates. The atmospheric constraints onthe top-down budget suggest that at least some of these bottom-up emissionsare overestimated. The latitudinal distribution of atmosphericobservation-based emissions indicates a predominance of tropical emissions(∼ 65 % of the global budget, < 30∘ N)compared to mid-latitudes (∼ 30 %, 30–60∘ N)and high northern latitudes (∼ 4 %, 60–90∘ N). The most important source of uncertainty in the methanebudget is attributable to natural emissions, especially those from wetlandsand other inland waters. Some of our global source estimates are smaller than those in previouslypublished budgets (Saunois et al., 2016; Kirschke et al., 2013). In particular wetland emissions are about 35 Tg CH4 yr−1 lower due toimproved partition wetlands and other inland waters. Emissions fromgeological sources and wild animals are also found to be smaller by 7 Tg CH4 yr−1 by 8 Tg CH4 yr−1, respectively. However, the overalldiscrepancy between bottom-up and top-down estimates has been reduced byonly 5 % compared to Saunois et al. (2016), due to a higher estimate of emissions from inland waters, highlighting the need for more detailed research on emissions factors. Priorities for improving the methanebudget include (i) a global, high-resolution map of water-saturated soilsand inundated areas emitting methane based on a robust classification ofdifferent types of emitting habitats; (ii) further development ofprocess-based models for inland-water emissions; (iii) intensification ofmethane observations at local scales (e.g., FLUXNET-CH4 measurements)and urban-scale monitoring to constrain bottom-up land surface models, andat regional scales (surface networks and satellites) to constrainatmospheric inversions; (iv) improvements of transport models and therepresentation of photochemical sinks in top-down inversions; and (v) development of a 3D variational inversion system using isotopic and/orco-emitted species such as ethane to improve source partitioning. The data presented here can be downloaded fromhttps://doi.org/10.18160/GCP-CH4-2019 (Saunois et al., 2020) and from theGlobal Carbon Project.more » « less
An official website of the United States government

