An official website of the United States government
ABSTRACT Traditional software reliability growth models (SRGM) characterize defect discovery with the Non‐Homogeneous Poisson Process (NHPP) as a function of testing time or effort. More recently, covariate NHPP SRGM models have substantially improved tracking and prediction of the defect discovery process by explicitly incorporating discrete multivariate time series on the amount of each underlying testing activity performed in successive intervals. Both classes of NHPP models with and without covariates are parametric in nature, imposing assumptions on the defect discovery process, and, while neural networks have been applied to SRGM models without covariates, no such studies have been applied in the context of covariate SRGM models. Therefore, this paper assesses the effectiveness of neural networks in predicting the software defect discovery process, incorporating covariates. Three types of neural networks are considered, including (i) recurrent neural networks (RNNs), (ii) long short‐term memory (LSTM), and (iii) gated recurrent unit (GRU), which are then compared with covariate models to validate tracking and predictive accuracy. Our results suggest that GRU achieved better overall goodness‐of‐fit, such as approximately 3.22 and 1.10 times smaller predictive mean square error, and 5.33 and 1.22 times smaller predictive ratio risk in DS1G and DS2G data sets, respectively, compared to covariate models when of the data is used for training. Moreover, to provide an objective comparison, three different proportions for training data splits were employed to illustrate the advancements between the top‐performing covariate NHPP model and the neural network, in which GRU illustrated a better performance over most of the scenarios. Thus, the neural network model with gated recurrent units may be a suitable alternative to track and predict the number of defects based on covariates associated with the software testing process.
more »
« less
A heterogeneous single changepoint software reliability growth model framework
Summary Many non‐homogeneous Poisson process software reliability growth models (SRGM) are characterized by a single continuous curve. However, failures are driven by factors such as the testing strategy and environment, integration testing and resource allocation, which can introduce one or more changepoint into the fault detection process. Some researchers have proposed non‐homogeneous Poisson process SRGM, but only consider a common failure distribution before and after changepoints. This paper proposes a heterogeneous single changepoint framework for SRGM, which can exhibit different failure distributions before and after the changepoint. Combinations of two simple and distinct curves including an exponential and S‐shaped curve are employed to illustrate the concept. Ten data sets are used to compare these heterogeneous models against their homogeneous counterparts. Experimental results indicate that heterogeneous changepoint models achieve better goodness‐of‐fit measures on 60% and 80% of the data sets with respect to the Akaike information criterion and predictive sum of squares measures.
more »
« less
- Award ID(s):
- 1749635
- PAR ID:
- 10460239
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Software Testing, Verification and Reliability
- Volume:
- 29
- Issue:
- 8
- ISSN:
- 0960-0833
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The ability to accurately measure recovery rate of infrastructure systems and communities impacted by disasters is vital to ensure effective response and resource allocation before, during, and after a disruption. However, a challenge in quantifying such measures resides in the lack of data as community recovery information is seldom recorded. To provide accurate community recovery measures, a hierarchical Bayesian kernel model (HBKM) is developed to predict the recovery rate of communities experiencing power outages during storms. The performance of the proposed method is evaluated using cross‐validation and compared with two models, the hierarchical Bayesian regression model and the Poisson generalized linear model. A case study focusing on the recovery of communities in Shelby County, Tennessee after severe storms between 2007 and 2017 is presented to illustrate the proposed approach. The predictive accuracy of the models is evaluated using the log‐likelihood and root mean squared error. The HBKM yields on average the highest out‐of‐sample predictive accuracy. This approach can help assess the recoverability of a community when data are scarce and inform decision making in the aftermath of a disaster. An illustrative example is presented demonstrating how accurate measures of community resilience can help reduce the cost of infrastructure restoration.more » « less
-
Abstract To enable real-time control of next-generation active structures during shock events, there is a need to identify the start of a shock event within microseconds of its initiation. The delayed classification of a shock event may cause damage to the system that could have been prevented with assumed next-generation active control mechanisms. Addressing the challenge of ultra-low latency shock event classification requires utilizing prior information on normal behaviors (i.e., the system under vibrational loading) to identify abnormalities that can be classified as features of a shock event. The purpose of changepoint shock classification is to automatically recognize when a structure of interest behaves differently than expected in some measurable way. In this work, we analyze two different methods for shock classification using changepoint methodologies. We study the use of adaptive cumulative summation and expectation maximization algorithms in this work. Each method presents advantages and disadvantages for different scenarios. This study aims to derive features (streams of time series data) for the changepoint algorithms and revise the changepoint models to be used in real-time robust shock event detection. In this work, a printed circuit board under continuous vibrations before, during, and after a shock event is used to investigate the proposed methodologies. The printed circuit board is monitored with an accelerometer that is used to monitor both the vibrational and shock state of the system. The vibrational response of the system consists of accelerations up to 20 m/s2, while the shock event consists of loadings up to 2,000 m/s2. This work showed that the CUSUM algorithm is fairly effective at identifying the shock state in data but generates many false positives during normal behavior times, with no false positives post-shock, indicating accurate shock state detection despite early errors. In contrast, the Expectation Maximization (EM) algorithm shows improved performance by correctly predicting no shock in the initial phase and accurately identifying the onset of the shock state. It occasionally misclassifies shocked points as normal due to its change point identification process. Compared to CUSUM, EM has fewer false positives before the shock and similar performance during and after the shock event. Future research efforts will focus on developing online versions of these algorithms, which can identify system states with a minimum number of errors. The limitations of the system and its robustness to noise are discussed.more » « less
-
null (Ed.)Abstract Average lifetime, or mean time to failure (MTTF), of a product is an important metric to measure the product reliability. Current methods of evaluating MTTF are mainly statistics or data based. They need lifetime testing on a number of products to get the lifetime samples, which are then used to estimate MTTF. The lifetime testing, however, is expensive in terms of both time and cost. The efficiency is also low because it cannot be effectively incorporated in the early design stage where many physics-based models are available. We propose to predict MTTF in the design stage by means of physics-based models. The advantage is that the design can be continually improved by changing design variables until reliability measures, including MTTF, are satisfied. Since the physics-based models are usually computationally demanding, we face a problem with both big data (on the model input side) and small data (on the model output side). We develop an adaptive supervised training method based on Gaussian process regression, and the method can then quickly predict MTTF with minimized number of calling the physics-based models. The effectiveness of the method is demonstrated by two examples.more » « less
-
ABSTRACT We study the optimal sequencing of a batch of tasks on a machine subject to random disruptions driven by a non‐homogeneous Poisson process (NHPP), such that every disruption requires the interrupted task to be re‐processed from scratch, and partially completed work on a disrupted task is wasted. The NHPP models random disruptions whose frequency varies systematically with time. In general, the time taken to process a given batch of tasks depends on the order in which the tasks are processed. We find conditions under which the simplest possible sequencing rules—shortest processing time first (SPT) and longest processing time first (LPT)—suffice to minimize the completion time of a batch of tasks.more » « less
