The paper develops a methodology for the online built-in self-testing of deep neural network (DNN) accelerators to validate the correct operation with respect to their functional specifications. The DNN of interest is realized in the hardware to perform in-memory computing using non-volatile memory cells as computational units. Assuming a functional fault model, we develop methods to generate pseudorandom and structured test patterns to detect hardware faults. We also develop a test-sequencing strategy that combines these different classes of tests to achieve high fault coverage. The testing methodology is applied to a broad class of DNNs trained to classify images from the MNIST, Fashion-MNIST, and CIFAR-10 datasets. The goal is to expose hardware faults which may lead to the incorrect classification of images. We achieve an average fault coverage of 94% for these different architectures, some of which are large and complex.
more »
« less
To call, or not to call: contrasting direct and indirect branch coverage in test generation
While adequacy criteria offer an end-point for testing, they do not mandate how targets are covered. Branch Coverage may be attained through direct calls to methods, or through indirect calls between methods. Automated generation is biased towards the rapid gains offered by indirect coverage. Therefore, even with the same end-goal, humans and automation produce very different tests. Direct coverage may yield tests that are more understandable, and that detect faults missed by traditional approaches. However, the added burden for the generation framework may result in lower coverage and faults that emerge through method interactions may be missed. To compare the two approaches, we have generated test suites for both, judging efficacy against real faults. We have found that requiring direct coverage results in lower achieved coverage and likelihood of fault detection. However, both forms of Branch Coverage cover code and detect faults that the other does not. By isolating methods, Direct Branch Coverage is less constrained in the choice of input. However, traditional Branch Coverage is able to leverage method interactions to discover faults. Ultimately, both are situationally applicable within the context of a broader testing strategy.
more »
« less
- Award ID(s):
- 1657299
- PAR ID:
- 10082003
- Date Published:
- Journal Name:
- SBST '18 Proceedings of the 11th International Workshop on Search-Based Software Testing
- Page Range / eLocation ID:
- 43 to 50
- Subject(s) / Keyword(s):
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Coral reefs comprise some of the most biodiverse habitats on the planet. These ecosystems face a range of stressors, making quantifying community assemblages and potential changes vital to effective management. To understand short- and long-term changes in biodiversity and detect early warning signals of decline, new methods for quantifying biodiversity at scale are necessary. Acoustic monitoring techniques have proven useful in observing species activities and biodiversity on coral reefs through aggregate approaches (i.e. energy as a proxy). However, few studies have ground-truthed these acoustic analyses with human-based observations. In this study, we sought to expand these passive acoustic methods by investigating biological sounds and fish call rates on a healthy reef, providing a unique set of human-confirmed, labeled acoustic observations. We analyzed acoustic data from Tektite Reef, St. John, US Virgin Islands, over a 2 mo period. A subset of acoustic files was manually inspected to identify recurring biotic sounds and quantify reef activity throughout the day. We found a high variety of acoustic signals in this soundscape. General patterns of call rates across time conformed to expectations, with dusk and dawn showing important and significantly elevated peaks in soniferous fish activity. The data reflected high variability in call rates across days and lunar phases. Call rates did not correspond to sound pressure levels, suggesting that certain call types may drive crepuscular trends in sound levels while lower-level critical calls, likely key for estimating biodiversity and behavior, may be missed by gross sound level analyses.more » « less
-
After more than 30 years of development, combinatorial testing has become an essential method in the field of software testing, which has always been an active field. Through combined coverage, combinatorial testing can detect faults caused by various parameters and their interactions in a software under test. This paper aims to review the development of combinatorial testing and briefly introduce its basic applications. We classify today's combinatorial testing applications, including those of traditional industry and those of the IT industry. The research-and-application progress of combinatorial testing was investigated with respect to various fields of application, and potential application directions for the future were proposed to provide ideas its the extensive application.more » « less
-
Concolic testing is a scalable solution for automated generation of directed tests for validation of hardware designs. Unfortunately, concolic testing fails to cover complex corner cases such as hard-to-activate branches. In this article, we propose an incremental concolic testing technique to cover hard-to-activate branches in register-transfer level (RTL) models. We show that a complex branch condition can be viewed as a sequence of easy-to-activate events. We map the branch coverage problem to the coverage of a sequence of events. We propose an efficient algorithm to cover the sequence of events using concolic testing. Specifically, the test generated to activate the current event is used as the starting point to activate the next event in the sequence. Experimental results demonstrate that our approach can be used to generate directed tests to cover complex corner cases in RTL models while state-of-the-art methods fail to activate them.more » « less
-
Building precise call graphs for Javascript programs is a fundamental build- ing block for many important software engineering and security applications such as bug detection, program repair, and refactoring. However, resolving dynamic calls using static analysis is challenging because it requires enumerating all possible values of both the object and the field. As a result, static call graph construction algorithms for Javascript ignore such dynamic calls, resulting in missed edges and a high false negative rate. We present a new approach, CALLME, that combines Language Models (LMs) with a custom static analyzer to address this challenge. Our key insight is in using LMs to incorporate additional modalities such as variable names, natural language documentation, and calling contexts, which are often sufficient to resolve dynamic property calls, but are difficult to incorporate in traditional static analysis. We implement our approach in CALLME and evaluate it on a dataset of call edges that are dependent on dynamic property accesses. CALLME achieves 80% accuracy and .79 F1, outperforming the state-of-the- art static analyzer by 30% and .60, respectively. To study the effectiveness of CALLME on downstream analysis tasks, we evaluate it on our manually curated dataset with 25 known Javascript vulnerabilities. CALLME can detect 24 vulnerabilities with only 3 false positives, whereas static analysis tools based on current call graph construction algorithms miss all of them.more » « less
An official website of the United States government

