skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Humans in Empirical Software Engineering Studies: An Experience Report
The use of human validation in software engineering methods, tools, and processes is crucial to understanding how these artifacts actually impact the people using them. In this paper, we report our experiences on two methods of data collection we have used in software engineering empirical studies, namely online questionnaire-based data collection and in-person eye tracking data collection using eye tracking equipment. The design and instrumentation challenges we faced are discussed with possible ways to mitigate them. We conclude with some guidelines and our vision for the future in human-centric studies in software engineering.  more » « less
Award ID(s):
1855756
PAR ID:
10340703
Author(s) / Creator(s):
;
Date Published:
Journal Name:
2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)
Page Range / eLocation ID:
1286 to 1292
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Empirical software engineering)
    null (Ed.)
    For several years, the software engineering research community used eye trackers to study program comprehension, bug localization, pair programming, and other software engineering tasks. Eye trackers provide researchers with insights on software engineers’ cognitive processes, data that can augment those acquired through other means, such as on-line surveys and questionnaires. While there are many ways to take advantage of eye trackers, advancing their use requires defining standards for experimental design, execution, and reporting. We begin by presenting the foundations of eye tracking to provide context and perspective. Based on previous surveys of eye tracking for programming and software engineering tasks and our collective, extensive experience with eye trackers, we discuss when and why researchers should use eye trackers as well as how they should use them. We compile a list of typical use cases—real and anticipated—of eye trackers, as well as metrics, visualizations, and statistical analyses to analyze and report eye-tracking data. We also discuss the pragmatics of eye tracking studies. Finally, we offer lessons learned about using eye trackers to study software engineering tasks. This paper is intended to be a one-stop resource for researchers interested in designing, executing, and reporting eye tracking studies of software engineering tasks. 
    more » « less
  2. ; ; ; ; ; ; ; (, International Conference on Software Engineering (ICSE) - Demonstrations Track)
    null (Ed.)
    Eye tracking tools are used in software engineering research to study various software development activities. However, a major limitation of these tools is their inability to track gaze data for activities that involve source code editing. We present a novel solution to support eye tracking experiments for tasks involving source code edits as an extension of the iTrace community infrastructure. We introduce the iTrace-Atom plugin and gazel—a Python data processing pipeline that maps gaze information to changing source code elements and provides researchers with a way to query this dynamic data. iTrace-Atom is evaluated via a series of simulations and is over 99% accurate at high eye-tracking speeds of over 1,000Hz. iTrace and gazel completely revolutionize the way eye tracking studies are conducted in realistic settings with the presence of scrolling, context switching, and now editing. This opens the doors to support many day-to-day software engineering tasks such as bug fixing, adding new features, and refactoring. 
    more » « less
  3. ; ; ; ; ; ; (, 2019 IEEE/ACM 6th International Workshop on Eye Movements in Programming (EMIP))
    null (Ed.)
    The evolution and effort in designing and implementing iTrace, an infrastructure for integrating eye tracking into developer environments, is presented. The goal is to make eye tracking practical for various stakeholders in software engineering namely researchers, practitioners, and educators. An overview of iTrace and the general process involved in conducting an eye tracking study with human subjects using iTrace is presented in this tool demo paper. Upcoming features and ongoing plans for community involvement are also presented. 
    more » « less
  4. ; ; ; ; ; ; (, 6th International Workshop on Eye Movements in Programming (EMIP))
    The evolution and effort in designing and implementing iTrace, an infrastructure for integrating eye tracking into developer environments, is presented. The goal is to make eye tracking practical for various stakeholders in software engineering namely researchers, practitioners, and educators. An overview of iTrace and the general process involved in conducting an eye tracking study with human subjects using iTrace is presented in this tool demo paper. Upcoming features and ongoing plans for community involvement are also presented. 
    more » « less
  5. ; ; ; ; ; ; (, Proceedings of the 6th International Workshop on Eye Movements in Programming (EMIP))
    The evolution and effort in designing and implementing iTrace, an infrastructure for integrating eye tracking into developer environments, is presented. The goal is to make eye tracking practical for various stakeholders in software engineering namely researchers, practitioners, and educators. An overview of iTrace and the general process involved in conducting an eye tracking study with human subjects using iTrace is presented in this tool demo paper. Upcoming features and ongoing plans for community involvement are also presented. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email