Search for: All records

In this paper we reflect on our decade-long journey of creating, evolving, and evaluating a number of software design concepts and technical debt management technologies. These include: a novel maintainability metric, a new model for representing design information, a suite of design anti-patterns, and a formalized model of design debt. All of these concepts are rooted in options theory, and they all share the objective of helping a software project team quantify and visualize major design principles, and address the very real maintainability challenges faced by their organizations in practice. The evolution of our research has been propelled by our continuous interactions with industrial collaborators. For each concept, technology, and supporting tool, we embarked on an ambitious program of empirical validation—in “the lab”, with industry partners, and with open source projects. We reflect on the successes of this research and on areas where significant challenges remain. In particular, we observe that improved software design education, both for students and professional developers, is the prerequisite for our research and technology to be widely adopted. During this journey, we also observed a number of gaps: between what we offer in research and what practitioners need, between management and development, and between debt detection and debt reduction. Addressing these challenges motivates our research moving forward. 

Dependencies among software entities are the basis for many software analytic research and architecture analysis tools. Dynamically typed languages, such as Python, JavaScript and Ruby, tolerate the lack of explicit type references, making certain syntactic dependencies indiscernible in source code. We call these possible dependencies, in contrast with the explicit dependencies that are directly referenced in source code. Type inference techniques have been widely studied and applied, but existing architecture analytic research and tools have not taken possible dependencies into consideration. The fundamental question is, to what extent will these missing possible dependencies impact the architecture analysis? To answer this question , we conducted an empirical study with 105 Python projects, using type inference techniques to manifest possible dependencies. Our study revealed that the architectural impact of possible dependencies is substantial-higher than that of explicit dependencies: (1) file-level possible dependencies account for at least 27.93% of all file-level dependencies, and create different dependency structures than that of explicit dependencies only, with an average difference of 30.71%; (2) adding possible dependencies significantly improves the precision (0.52%∼14.18%), recall(31.73%∼39.12%), and F1 scores (22.13%∼32.09%) of capturing co-change relations; (3) on average, a file involved in possible dependencies influences 28% more files and 42% more dependencies within architectural sub-spaces than a file involved in just explicit dependencies; (4) on average, a file involved in possible dependencies consumes 32% more maintenance effort. Consequently, maintainability scores reported by existing tools make a system written in these dynamic languages appear to be better modularized than it actually is. This evidence strongly * with the Ministry suggests that possible dependencies have a more significant impact than explicit dependencies on architecture quality, that architecture analysis and tools should assess and even emphasize the architectural impact of possible dependencies due to dynamic typing. 

Architecture degradation has a strong negative impact on software quality and can result in significant losses. Severe software degradation does not happen overnight. Software evolves continuously, through numerous issues, fixing bugs and adding new features, and architecture flaws emerge quietly and largely unnoticed until they grow in scope and significance when the system becomes difficult to maintain. Developers are largely unaware of these flaws or the accumulating debt as they are focused on their immediate tasks of address individual issues. As a consequence, the cumulative impacts of their activities, as they affect the architecture, go unnoticed. To detect these problems early and prevent them from accumulating into severe ones we propose to monitor software evolution by tracking the interactions among files revised to address issues. In particular, we propose and show how we can automatically detect active hotspots, to reveal architecture problems. We have studied hundreds of hotspots along the evolution timelines of 21 open source projects and showed that there exist just a few dominating active hotspots per project at any given time. Moreover, these dominating active hotspots persist over long time periods, and thus deserve special attention. Compared with state-of-the-art design and code smell detection tools we report that, using active hotspots, it is possible to detect signs of software degradation both earlier and more precisely.