More Like this

1. Extracting Inline Tests from Unit Tests

   https://doi.org/10.1145/3597926.3598149  

   Liu, Yu; Nie, Pengyu; Guo, Anna; Gligoric, Milos; Legunsen, Owolabi (July 2023, ACM)

   We recently proposed inline tests for validating individual program statements; they allow developers to provide test inputs, expected outputs, and test oracles immediately after a target statement. But, existing code can have many target statements. So, automatic generation of inline tests is an important next step towards increasing their adoption. We propose ExLi, the first technique for automatically generating inline tests. ExLi extracts inline tests from unit tests; it first records all variable values at a target statement while executing unit tests. Then, ExLi uses those values as test inputs and test oracles in an initial set of generated inline tests. Target statements that are executed many times could have redundant initial inline tests. So, ExLi uses a novel coverage-then-mutants based reduction process to remove redundant inline tests. We implement ExLi for Java and use it to generate inline tests for 718 target statements in 31 open-source programs. ExLi reduces 17,273 initially generated inline tests to 905 inline tests. The final set of generated inline tests kills up to 25.1% more mutants on target statements than developer written and automatically generated unit tests. That is, ExLi generates inline tests that can improve the fault-detection capability of the test suites from which they are extracted. 

   more » « less
2. Extracting Inline Tests from Unit Tests

   Yu Liu; Pengyu Nie; Ana Guo; Milos Gligoric; Owolabi Legunsen (July 2023, 32nd International Symposium on Software Testing and Analysis)

   We recently proposed inline tests for validating individual program statements; they allow developers to provide test inputs, expected outputs, and test oracles immediately after a target statement. But, existing code can have many target statements. So, automatic generation of inline tests is an important next step towards increasing their adoption. We propose ExLi, the first technique for automatically generating inline tests. ExLi extracts inline tests from unit tests; it first records all variable values at a target statement while executing unit tests. Then, ExLi uses those values as test inputs and test oracles in an initial set of generated inline tests. Target statements that are executed many times could have redundant initial inline tests. So, ExLi uses a novel coverage-and-mutation based reduction process to remove redundant inline tests. We implement ExLi for Java and use it to generate inline tests for 718 target statements in 31 open-source programs. ExLi reduces 17,273 initially generated inline tests to 905 inline tests. The final set of generated inline tests kills up to 25.1% more mutants than developer written and automatically generated unit tests. That is, ExLi generates inline tests that can improve the fault-detection capability of the test suites from which they are extracted. 

   more » « less
3. Interconnect-Aware Tests to Complement Gate-Exhaustive Tests

   Pomeranz, I; Venkataraman, S (May 2018, European Test Symposium)

   Gate-exhaustive and cell-aware tests are generated based on input patterns of cells in a design. While the tests provide thorough testing of the cells, the interconnects between them are tested only as input and output lines of cells. This paper defines cell-based faults that allow the interconnects to be tested more thoroughly within a uniform framework that only targets input patterns of cells. In contrast to a real cell that is part of the design, a dummy cell is used for defining interconnect-aware faults. Using a gate-level description of the circuit, a dummy cell contains an interconnect, an output gate of the real cell that drives it, and an input gate of the real cell that it drives. Experimental results for benchmark circuits show that many of the interconnect-aware faults are not detected accidentally by gate-exhaustive tests, and that the quality of the test set is improved by targeting interconnect-aware faults. Here, quality is measured by the numbers of detections of single stuck-at faults in a gate-level representation of the circuit. 

   more » « less
4. Ranking Relevant Tests for Order-Dependent Flaky Tests

   https://doi.org/10.1109/ICSE55347.2025.00178  

   Rahman, Shanto; Chanumolu, Bala Naren; Rafi, Suzzana; Shi, August; Lam, Wing (April 2025, IEEE)
5. Inline Tests

   Yu Liu; Pengyu Nie; Owolabi Legunsen; Milos Gligoric (October 2022, IEEE/ACM International Conference on Automated Software Engineering)

   Unit tests are widely used to check source code quality, but they can be too coarse-grained or ill-suited for testing individual program statements. We introduce inline tests to make it easier to check for faults in statements. We motivate inline tests through several language features and a common testing scenario in which inline tests could be beneficial. For example, inline tests can allow a developer to test a regular expression in place. We also define language-agnostic requirements for inline testing frameworks. Lastly, we implement I-Test, the first inline testing framework. I-Test works for Python and Java, and it satisfies most of the requirements. We evaluate I-Test on open-source projects by using it to test 144 statements in 31 Python programs and 37 Java programs. We also perform a user study. All nine user study participants say that inline tests are easy to write and that inline testing is beneficial. The cost of running inline tests is negligible, at 0.007x -- 0.014x, and our inline tests helped find two faults that have been fixed by the developers. 

   more » « less