More Like this

1. Automatically identifying valid API versions for software development tutorials on the Web

   https://doi.org/10.1002/smr.2227  

   Nishi, Manziba Akanda ; Damevski, Kostadin ( November 2019 , Journal of Software: Evolution and Process)

   Online tutorials are a valuable source of community‐created information used by numerous developers to learn new APIs and techniques. Once written, tutorials are rarely actively curated and can become dated over time. Tutorials often reference APIs that change rapidly, and deprecated classes, methods, and fields can render tutorials inapplicable to newer releases of the API. Newer tutorials may not be compatible with older APIs that are still in use.

   In this paper, we first empirically study the tutorial versioning problem, confirming its presence in popular tutorials on the Web. We subsequently propose a technique, based on similar techniques in the literature, for automatically detecting the applicable API version ranges of tutorials, given access to the official API documentation they reference. The proposed technique identifies each API mention in a tutorial and maps the mention to the corresponding API element in the official documentation. The version of the tutorial is determined by combining the version ranges of all of the constituent API mentions. Our technique's precision varies from 61% to 89% and recall varies from 42% to 84% based on different levels of granularity of API mentions and different problem constraints. We observe API methods are the most challenging to accurately disambiguate due to method overloading. As the API mentions in tutorials are often redundant, and each mention of a specific API element commonly occurs several times in a tutorial, the distance of the predicted version range from the true version range is low: 3.61 on average for the tutorials in our sample.

    

   more » « less
2. Automatically Identifying Valid API Versions for Software Development Tutorials on the Web

   Nishi, Manziba A. ; Damevski, K. ( August 2019 , Journal of software)

   Online tutorials are a valuable source of community created information used by numerous developers to learn new APIs and techniques. Once written, tutorials are rarely actively curated and can become dated over time. Tutorials often reference APIs that change rapidly, and deprecated classes, methods and fields can render tutorials inapplicable to newer releases of the API.Newer tutorials may not be compatible with older APIs that are still in use. In this paper, we first empirically study the tutorial versioning problem, confirming its presence in popular tutorials on the Web. We subsequently propose a technique, based on similar techniques in the literature, for automatically detecting the applicable API version ranges of tutorials, given access to the official API documentation they reference. The proposed technique identifies each API mention in a tutorial and maps the mention to the corresponding API element in the official documentation. The version of the tutorial is determined by combining the version ranges of all of the constituent API mentions. Our technique’s precision varies from 61% to 89% and recall varies from 42% to 84% based on different levels of granularity of API mentions and different problem constraints. We observe API methods are the most challenging to accurately disambiguate due to method overloading. As the API mentions in tutorials are often redundant, and each mention of a specific API element commonly occurs several times in a tutorial, the distance of the predicted version range from the true version range is low; 3.61 on average for the tutorials in our sample. 

   more » « less
3. A Theory of Robust API Knowledge

   https://doi.org/10.1145/3444945  

   Thayer, Kyle ; Chasins, Sarah E. ; Ko, Amy J. ( March 2021 , ACM Transactions on Computing Education)

   null (Ed.)

   Creating modern software inevitably requires using application programming interfaces (APIs). While software developers can sometimes use APIs by simply copying and pasting code examples, a lack of robust knowledge of how an API works can lead to defects, complicate software maintenance, and limit what someone can express with an API. Prior work has uncovered the many ways that API documentation fails to be helpful, though rarely describes precisely why. We present a theory of robust API knowledge that attempts to explain why, arguing that effective understanding and use of APIs depends on three components of knowledge: (1) the domain concepts the API models along with terminology, (2) the usage patterns of APIs along with rationale, and (3) facts about an API’s execution to support reasoning about its runtime behavior. We derive five hypotheses from this theory and present a study to test them. Our study investigated the effect of having access to these components of knowledge, finding that while learners requested these three components of knowledge when they were not available, whether the knowledge helped the learner use or understand the API depended on the tasks and likely the relevance and quality of the specific information provided. The theory and our evidence in support of its claims have implications for what content API documentation, tutorials, and instruction should contain and the importance of giving the right information at the right time, as well as what information API tools should compute, and even how APIs should be designed. Future work is necessary to both further test and refine the theory, as well as exploit its ideas for better instructional design. 

   more » « less
4. Effectiveness of an online homework tutorial about changing basis in quantum mechanics

   https://doi.org/10.1119/perc.2022.pr.Corsiglia  

   Corsiglia, Giaco ; Pollock, Steven J. ; Wilcox, Bethany R. ( January 2022 , Effectiveness of an online homework tutorial about changing basis in quantum mechanics)

   Changing basis is a common task when solving quantum mechanical problems. As part of a research project investigating student understanding of basis and change of basis in quantum mechanics, we developed a tutorial to support students in learning about basis in the context of spin-1/2 systems. We have since created an interactive online version of the basis tutorial as part of a freely available suite of online quantum tutorials called ACE Physics (https://acephysics.net). The ACE Physics tutorials include dynamic guidance elements and, unlike other tutorials, are intended for use outside the classroom without instructor facilitation. After extensive study in an instructor-supported environment, we assigned the ACE Physics basis tutorial as homework in two semesters of upper-division quantum mechanics, and we report on the effectiveness of the activity based on pre-/post-testing and comparison of student exam performance with a similar semester that did not include the activity. We find that the tutorial produces sufficient learning gains to justify continued assignment as a homework problem in our classes. 

   more » « less
5. Docable: Evaluating the Executability of Software Tutorials

   https://doi.org/10.1145/3368089.3409706  

   Mirhosseini, Samim ; Parnin, Chris ( January 2020 , ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering)

   The typical software tutorial includes step-by-step instructions for installing developer tools, editing files and code, and running commands. When these software tutorials are not executable, either due to missing instructions, ambiguous steps, or simply broken commands, their value is diminished. Non-executable tutorials impact developers in several ways, including frustrating learning experiences, and limiting usability of developer tools. To understand to what extent software tutorials are executable---and why they may fail---we conduct an empirical study on over 600 tutorials, including nearly 15,000 code blocks. We find a naive execution strategy achieves an overall executability rate of only 26%. Even a human-annotation-based execution strategy---while doubling executability---still yields no tutorial that can successfully execute all steps. We identify several common executability barriers, ranging from potentially innocuous causes, such as interactive prompts requiring human responses, to insidious errors, such as missing steps and inaccessible resources. We validate our findings with major stakeholders in technical documentation and discuss possible strategies for improving software tutorials, such as providing accessible alternatives for tutorial takers, and investing in automated tutorial testing to ensure continuous quality of software tutorials. 

   more » « less