Search for: All records

Computer science pedagogy, especially in the higher education and vocational training context, has long-favored the hands-on practice provided by programming tasks due to the belief that this leads to better performance on hands-on tasks at work. This assumption, however, has not been experimentally tested against other modes of engagement such as worked example-based reflection. While theory suggests that example-based reflection could be better for conceptual learning, the concern is that the lack of practice will leave students unable to implement the learned concepts in practice, thus leaving them unprepared for work. In this paper, therefore, we experimentally contrast programming practice with example-based reflection to observe their differential impact on conceptual learning and performance on a hands-on task in the context of a collaborative programming project. The industry paradigm of Mob Programming, adapted for use in an online and instructional context, is used to structure the collaboration. Keeping with the prevailing view held in pedagogy, we hypothesize that example-based reflection will lead to better conceptual learning but will be detrimental to hands-on task performance. Results support that reflection leads to conceptual learning. Additionally, however, reflection does not pose an impediment to hands-on task performance. We discuss possible explanations for this effect, thus providing an improved understanding of prior theory in this new computer science education context. We also discuss implications for the pedagogy of software engineering education, in light of this new evidence, that impacts student learning as well as work performance in the future. 

Contributing to the literature on aptitude-treatment interactions between worked examples and problem-solving, this paper addresses differential learning from the two approaches when students are positioned as domain experts learning new concepts. Our evaluation is situated in a team project that is part of an advanced software engineering course. In this course, students who possess foundational domain knowledge but are learning new concepts engage alternatively in programming followed by worked example-based reflection. They are either allowed to finish programming or are curtailed after a pre-specified time to participate in a longer worked example-based reflection. We find significant pre- to post-test learning gains in both conditions. Then, we not only find significantly more learning when students participated in longer worked example-based reflections but also a significant performance improvement on a problem-solving transfer task. These findings suggest that domain experts learning new concepts benefit more from worked example-based reflections than from problem-solving. 

When developers make changes to their code, they typically run regression tests to detect if their recent changes (re)introduce any bugs. However, many tests are flaky, and their outcomes can change non-deterministically, failing without apparent cause. Flaky tests are a significant nuisance in the development process, since they make it more difficult for developers to trust the outcome of their tests. The traditional approach to identify flaky tests is to rerun them multiple times: if a test is observed both passing and failing on the same code, it is definitely flaky. We conducted a very large empirical study looking for flaky tests by rerunning the test suites of 24 projects 10,000 times each, and found that even with this many reruns, some flaky tests were still not detected. We propose FlakeFlagger, a novel approach that collects a set of features describing the behavior of each test, and then predicts tests that are likely to be flaky based on similar behavioral features. We found that FlakeFlagger correctly labeled at least as many tests as flaky as a state-of-the-art flaky test classifier, but that FlakeFlagger reported far fewer false positives (an increase in precision from just 11% to 60%). This lower false positive rate translates directly to saved time for researchers and developers who use the classification result to guide more expensive flaky test detection processes. By investigating the information gain of each feature, we conclude that test execution time, overall test coverage, coverage of recently changed lines and usage of third party libraries are effective predictors of test flakiness. We did not find any keywords or tokens in the source code of tests that were effective in predicting test flakiness, and did not find the presence of test smells to be effective in predicting test flakiness.

This archive contains the dataset that we collected of flaky tests, along with the features that we collected from each test.

Contents:
Project_Info.csv: List of projects and their revisions studied
build-logs-<project-slug>.tgz: An archive of all of the maven build logs from each of the 10,000 runs of that project's test suite. 
failing-test-reports-<project-slug>.tgz An archive of all of the surefire XML reports for each failing test of each build of each project.
test_results.csv: Summary of the number of passing and failing runs for each test in each project. 
"Run ID" is a key into the <project-slug>.tgz archive also in this artifact, which refers to the run that we observed the test fail on.
test_features.csv: Summary of the features that each test had, as per our feature detectors described in the paper
flakeflagger-code.zip: All scripts used to generate and process these results. These scripts are also located at https://github.com/AlshammariA/FlakeFlagger

 

Over the past eleven years, the Robot Operating System (ROS), has grown from a small research project into the most popular framework for robotics development. Composed of packages released on the Rosdistro package manager, ROS aims to simplify development by providing reusable libraries, tools and conventions for building a robot. Still, developing a complete robot is a difficult task that involves bridging many technical disciplines. Experts who create computer vision packages, for instance, may need to rely on software designed by mechanical engineers to implement motor control. As building a robot requires domain expertise in software, mechanical, and electrical engineering, as well as artificial intelligence and robotics, ROS faces knowledge based barriers to collaboration. In this paper, we examine how the necessity of domain specific knowledge impacts the open source collaboration model. We create a comprehensive corpus of package metadata and dependencies over three years in the ROS ecosystem, analyze how collaboration is structured, and study the dependency network evolution. We find that the most widely used ROS packages belong to a small cluster of foundational working groups (FWGs), each organized around a different domain in robotics. We show that the FWGs are growing at a slower rate than the rest of the ecosystem, in terms of their membership and number of packages, yet the number of dependencies on FWGs is increasing at a faster rate. In addition, we mined all ROS packages on GitHub, and showed that 82% rely exclusively on functionality provided by FWGs. Finally, we investigate these highly influential groups and describe the unique model of collaboration they support in ROS. 

Dynamic conversational agent-based support for collaborative learning has shown significant positive effects on learning over no-support or static-support control conditions in prior studies. In order to understand the boundary between human-led and AI-led support for collaboration, we compare in this study an approach where the agent’s primary role is to help students regulate their own collaboration with two more typical prompting strategies that are used only during a reflection phase: one designed to provide a specific informational focus for the reflection, and the other designed to draw out evaluation, elaboration, and exploration of alternative perspectives. Significant positive effects on learning over and above just the human-led form of support are observed when either of the prompting strategies are used. 

Purpose In response to the evolving COVID-19 pandemic, many universities have transitioned to online instruction. With learning promising to be online, at least in part, for the near future, instructors may be thinking of providing online collaborative learning opportunities to their students who are increasingly isolated from their peers because of social distancing guidelines. This paper aims to provide design recommendations for online collaborative project-based learning exercises based on this research in a software engineering course at the university level. Design/methodology/approach Through joint work between learning scientists, course instructors and software engineering practitioners, instructional design best practices of alignment between the context of the learners, the learning objectives, the task and the assessment are actualized in the design of collaborative programming projects for supporting learning. The design, first segments a short real-time collaborative exercise into tasks, each with a problem-solving phase where students participate in collaborative programming, and a reflection phase for reflecting on what they learned in the task. Within these phases, a role-assignment paradigm scaffolds collaboration by assigning groups of four students to four complementary roles that rotate after each task. Findings By aligning each task with granular learning objectives, significant pre- to post-test learning from the exercise as well as each task is observed. Originality/value The roles used in the paradigm discourage divide-and-conquer tendencies often associated with collaborative projects. By requiring students to discuss conflicting ideas to arrive at a consensus implementation, their ideas are made explicit, thus providing opportunities for clarifying misconceptions through discussion and learning from the collaboration.