When dealing with safety–critical systems, various regulations, standards, and guidelines stipulate stringent requirements for certification and traceability of artifacts, but typically lack details with regards to the corresponding software engineering process. Given the industrial practice of only using semi-formal notations for describing engineering processes – with the lack of proper tool mapping – engineers and developers need to invest a significant amount of time and effort to ensure that all steps mandated by quality assurance are followed. The sheer size and complexity of systems and regulations make manual, timely feedback from Quality Assurance (QA) engineers infeasible. In order to address these issues, in this paper, we propose a novel framework for tracking, and “passively” executing processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to three case studies: a safety–critical open-source community system, a safety–critical system in the air-traffic control domain, and a non-safety–critical, web-based system. Results from our analysis confirm that trace links are often corrected or completed after the work step has been considered finished, and the engineer has already moved on to another step. Thus, support for timely and automated constraint checking has significant potential to reduce rework as the engineer receives continuous feedback already during their work step.
more »
« less
Supporting Quality Assurance with Automated Process-Centric Quality Constraints Checking
Regulations, standards, and guidelines for safety-critical systems stipulate stringent traceability but do not prescribe the corresponding, detailed software engineering process. Given the industrial practice of using only semi-formal notations to describe engineering processes, processes are rarely ``executable'' and developers have to spend significant manual effort in ensuring that they follow the steps mandated by quality assurance. The size and complexity of systems and regulations makes manual, timely feedback from Quality Assurance (QA) engineers infeasible. In this paper we propose a novel framework for tracking processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to two different case studies; one open source community system and a safety-critical system in the air-traffic control domain. Results from the analysis show that trace links are often corrected or completed after the fact and thus timely and automated constraint checking support has significant potential on reducing rework.
more »
« less
- Award ID(s):
- 1909007
- NSF-PAR ID:
- 10283462
- Date Published:
- Journal Name:
- 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)
- Volume:
- 2021
- Issue:
- 43
- Page Range / eLocation ID:
- 1298 to 1310
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
When dealing with safety-critical systems, various regulations, standards, and guidelines stipulate stringent requirements for certification and traceability of artifacts, but typically lack \rev{details} with regards to the corresponding software engineering process. Given the industrial practice of only using semi-formal notations for describing engineering processes with the lack of proper tool mapping engineers and developers need to invest a significant amount of time and effort to ensure that all steps mandated by quality assurance are followed. The sheer size and complexity of systems and regulations make manual, timely feedback from Quality Assurance (QA) engineers infeasible. In order to address these issues, in this paper, we propose a novel framework for tracking, and ``passively'' executing processes in the background, automatically checking QA constraints depending on process progress, and informing the developer of unfulfilled QA constraints. We evaluate our approach by applying it to three case studies: a safety-critical open-source community system, a safety-critical system in the air-traffic control domain, and a non-safety-critical, web-based system. Results from our analysis confirm that trace links are often corrected or completed after the work step has been considered finished, and the engineer has already moved on to another step. Thus, support for timely and automated constraint checking has significant potential to reduce rework as the engineer receives continuous feedback already during their work step.more » « less
-
null (Ed.)Purpose Due to the complexity of and variations in additive manufacturing (AM) processes, there is a level of uncertainty that creates critical issues in quality assurance (QA), which must be addressed by time-consuming and cost-intensive tasks. This deteriorates the process repeatability, reliability and part reproducibility. So far, many AM efforts have been performed in an isolated and scattered way over several decades. In this paper, a systematically integrated holistic view is proposed to achieve QA for AM. Design/methodology/approach A systematically integrated view is presented to ensure the predefined part properties before/during/after the AM process. It consists of four stages, namely, QA plan, prospective validation, concurrent validation and retrospective validation. As a foundation for QA planning, a functional workflow and the required information flows are proposed by using functional design models: Icam DEFinition for Function Modeling. Findings The functional design model of the QA plan provides the systematically integrated view that can be the basis for inspection of AM processes for the repeatability and qualification of AM parts for reproducibility. Research limitations/implications A powder bed fusion process was used to validate the feasibility of this QA plan. Feasibility was demonstrated under many assumptions; real validation is not included in this study. Social implications This study provides an innovative and transformative methodology that can lead to greater productivity and improved quality of AM parts across industries. Furthermore, the QA guidelines and functional design models provide the foundation for the development of a QA architecture and management system. Originality/value This systematically integrated view and the corresponding QA plan can pose fundamental questions to the AM community and initiate new research efforts in the in-situ digital inspection of AM processes and parts.more » « less
-
null ; null ; null (Ed.)One of the biggest challenges in code quality assurance is the amount of code that needs to be reviewed at an instance before the code is deployed on production. Reviewers need to check not only coding practices and formatting but also the meaning of the code and its compliance with requirements. Enterprise systems are notoriously known for the large codebase, challenging business logic, and advanced code constructs, which require significant resources for code review. However, enterprise systems use coding constructs that reveal aspects and constraints about the business logic, such as validation, database connection, and API. We extract these aspects and their relationships into a comprehensive metamodel. Next, we persist the metamodel into a graph database and conduct quality assurance checks via database queries. This method significantly reduces the amount of information that needs to be processed while maintaining key enterprise aspects. The method enables system administrators or project managers to discover defects and inconsistencies without reading the code.more » « less
-
Safety and security are the two most important properties of industrial control systems (ICS), and their integration is necessary to ensure that safety goals do not undermine security goals and vice versa. Sometimes, safety and security co-engineering leads to conflicting requirements or violations capable of impacting the normal behavior of the system. Identification, analysis, and resolution of conflicts arising from safety and security co-engineering is a major challenge, an under-researched area in safety-critical systems(ICS). This paper presents an STPA-SafeSec-CDCL approach that addresses the challenge. Our proposed methodology combines the STPA-SafeSec approach for safety and security analysis and the Conflict-Driven Clause Learning (CDCL) approach for the identification, analysis, and resolution of conflicts where conflicting constraints are encoded in satisfiability (SAT) problems. We apply our framework to the Tennessee Eastman Plant process model, a chemical process model developed specifically for the study of industrial control processes, to demonstrate how to use the proposed method. Our methodology goes beyond the requirement analysis phase and can be applied to the early stages of system design and development to increase system reliability, robustness, and resilience.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICSE43902.2021.00118
