When dealing with safety-critical systems, various regulations, standards, and guidelines stipulate stringent requirements for certification and traceability of artifacts, but typically lack \rev{details} with regards to the corresponding software engineering process. Given the industrial practice of only using semi-formal notations for describing engineering processes with the lack of proper tool mapping engineers and developers need to invest a significant amount of time and effort to ensure that all steps mandated by quality assurance are followed. The sheer size and complexity of systems and regulations make manual, timely feedback from Quality Assurance (QA) engineers infeasible. In order to address these issues, in this paper, we propose a novel framework for tracking, and ``passively'' executing processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints.
We evaluate our approach by applying it to three case studies: a safety-critical open-source community system, a safety-critical system in the air-traffic control domain, and a non-safety-critical, web-based system. Results from our analysis confirm that trace links are often corrected or completed after the work step has been considered finished, and the engineer has already moved on to another step. Thus, support for timely and automated constraint checking has significant potential to reduce rework as the engineer receives continuous feedback already during their work step.
more »
« less
This content will become publicly available on August 1, 2024
ProCon: An automated process-centric quality constraints checking framework
When dealing with safety–critical systems, various regulations, standards, and guidelines stipulate stringent requirements for certification and traceability of artifacts, but typically lack details with regards to the corresponding software engineering process. Given the industrial practice of only using semi-formal notations for describing engineering processes – with the lack of proper tool mapping – engineers and developers need to invest a significant amount of time and effort to ensure that all steps mandated by quality assurance are followed. The sheer size and complexity of systems and regulations make manual, timely feedback from Quality Assurance (QA) engineers infeasible. In order to address these issues, in this paper, we propose a novel framework for tracking, and “passively” executing processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to three case studies: a safety–critical open-source community system, a safety–critical system in the air-traffic control domain, and a non-safety–critical, web-based system. Results from our analysis confirm that trace links are often corrected or completed after the work step has been considered finished, and the engineer has already moved on to another step. Thus, support for timely and automated constraint checking has significant potential to reduce rework as the engineer receives continuous feedback already during their work step.
more »
« less
- Award ID(s):
- 1931962
- NSF-PAR ID:
- 10468151
- Publisher / Repository:
- Journal of Systems and Software
- Date Published:
- Journal Name:
- Journal of Systems and Software
- Volume:
- 202
- Issue:
- C
- ISSN:
- 0164-1212
- Page Range / eLocation ID:
- 111727
- Subject(s) / Keyword(s):
- ["Software engineering process","Traceability","Developer support","Quality assurance","Process deviation","Constraint checking"]
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Regulations, standards, and guidelines for safety-critical systems stipulate stringent traceability but do not prescribe the corresponding, detailed software engineering process. Given the industrial practice of using only semi-formal notations to describe engineering processes, processes are rarely ``executable'' and developers have to spend significant manual effort in ensuring that they follow the steps mandated by quality assurance. The size and complexity of systems and regulations makes manual, timely feedback from Quality Assurance (QA) engineers infeasible. In this paper we propose a novel framework for tracking processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to two different case studies; one open source community system and a safety-critical system in the air-traffic control domain. Results from the analysis show that trace links are often corrected or completed after the fact and thus timely and automated constraint checking support has significant potential on reducing rework.more » « less
-
Process safety is becoming a greater focus of chemical plant design and operation due to the number of incidents involving dangerous chemical accidents. Since its creation nearly 20 years ago, the Chemical Safety Board (CSB) has investigated 130 safety incidents and provided over 800 safety recommendations to operating chemical facilities. Following a gas well blowout in 2018, the CSB gave a recommendation to the American Petroleum Institute (API) to establish recommended practice on alarm management. Similarly, in 2017, the CSB gave a recommendation to Arkema Inc. to update their emergency response training following a hurricane that caused a fire at one of their manufacturing sites. Many times, CSB-led investigations resulted in new regulations and standards that are enforced by the Occupational Safety and Health Administration (OSHA) or the Environmental Protection Agency (EPA). These critical recommendations positively impact not only the plant workers but also the surrounding community and the environment. While these safety measures enhance industrial safety culture, it is important that process safety also be integrated into university-level engineering curricula to promote safety culture while future engineers are still developing. Integrating process safety into the curriculum prepares students by familiarizing them with the difficult decisions they will be required to make in professional practice. ABET, the engineering program accreditation body, acknowledges the value of early, appropriate training within their program guidelines “Criteria for Chemical Engineering Curriculum” which states that recognition and assessment of the hazards associated with chemical processes must be included in the curriculum for program accreditation. Based on this requirement, many institutions have taken the approach to integrate process safety into their curriculum using video case studies, adding entire courses to cover hazard identification, and including safety lectures in design courses. A common theme missing from these methods is instruction on how to approach, recognize, and navigate decisions within a process safety context; a lack of this situational awareness was noted as a key element in industrial process safety incidents. Understanding how students approach process safety decisions is important for developing teaching methods and curriculum that will better prepare them for professional practice. As part of this study, we will measure how students rank criteria associated with process safety decisions, and how these prioritizations change after exposure to a process safety decision making intervention. Through this work, we hope to determine how process safety curriculum may be improved to help better prepare students for process safety decisions within industry.more » « less
-
We present a principal-agent model of a one-shot, shallow, systems engineering process. The process is "one-shot" in the sense that decisions are made during a one-time step and that they are final. The term "shallow" refers to a one-layer hierarchy of the process. Specifically, we assume that the systems engineer has already decomposed the problem in subsystems and that each subsystem is assigned to a different subsystem engineer. Each subsystem engineer works independently to maximize their own expected payoff. The goal of the systems engineer is to maximize the system-level payoff by incentivizing the subsystem engineers. We restrict our attention to requirements-based system-level payoffs, i.e., the systems engineer makes a profit only if all the design requirements are met. We illustrate the model using the design of an Earth-orbiting satellite system where the systems engineer determines the optimum incentive structures and requirements for two subsystems: the propulsion subsystem and the power subsystem. The model enables the analysis of a systems engineer's decisions about optimal passed-down requirements and incentives for sub-system engineers under different levels of task difficulty and associated costs. Sample results, for the case of risk-neutral systems and subsystems engineers, show that it is not always in the best interest of the systems engineer to pass down the true requirements. As expected, the model predicts that for small to moderate task uncertainties the optimal requirements are higher than the true ones, effectively eliminating the probability of failure for the systems engineer. In contrast, the model predicts that for large task uncertainties the optimal requirements should be smaller than the true ones in order to lure the subsystem engineers into participation.more » « less
-
As assurance cases have grown in popularity for safety-critical systems, so too has their complexity and thus the need for methods to systematically build them. Assurance cases can grow too large and too abstract for anyone but the original builders to understand, making reuse difficult. Reuse is important because different systems might have identical or similar components, and a good solution for one system should be applicable to similar systems. Prior research has shown engineers can alleviate some of the complexity issues through modularity and identifying common patterns which are more easily understood for reuse across different systems. However, we believe these patterns are too complicated for users who lack expertise in software engineering or assurance cases. This paper suggests the concept of lower-level patterns which we call recipes. We use the safety-critical field of synthetic biology, as an example discipline to demonstrate how a recipe can be built and applied.more » « less
