An official website of the United States government
Mutation testing is widely used in research as a metric for evaluating the quality of test suites. Mutation testing runs the test suite on generated mutants (variants of the code under test), where a test suite kills a mutant if any of the tests fail when run on the mutant. Mutation testing implicitly assumes that tests exhibit deterministic behavior, in terms of their coverage and the outcome of a test (not) killing a certain mutant. Such an assumption does not hold in the presence of flaky tests, whose outcomes can non-deterministically differ even when run on the same code under test. Without reliable test outcomes, mutation testing can result in unreliable results, e.g., in our experiments, mutation scores vary by four percentage points on average between repeated executions, and 9% of mutant-test pairs have an unknown status. Many modern software projects suffer from flaky tests. We propose techniques that manage flakiness throughout the mutation testing process, largely based on strategically re-running tests. We implement our techniques by modifying the open-source mutation testing tool, PIT. Our evaluation on 30 projects shows that our techniques reduce the number of "unknown" (flaky) mutants by 79.4%.
more »
« less
Optimal test methods for determining material parameters
Measuring the parameters that control the deformability and strength of soils through either laboratory experiments or in situ testing is critical for numerous applications in geotechnical engineering. While image- and wave-based techniques are increasingly prevalent, there is a perpetual need for techniques capable of sensing local, nonlinear properties, for which mechanical testing is the only viable option. Existing methods for inferring mechanical properties have evolved largely by trial and error, and there is no general, systematic approach for evaluating one possible approach against another. As a first step toward addressing these challenges, this paper describes a quantitative metric that can discriminate between different types of mechanical tests with respect to how well they are able to recover the true mechanical properties of the material. The metric is devised by (1) creating a min-max optimization of parameter sensitivities, considering the local and global topological properties of the forward model, and (2) evaluating the metric for fundamental material tests.
more »
« less
- Award ID(s):
- 1846817
- PAR ID:
- 10424498
- Editor(s):
- Rahman, M. M.; Jaksa, M.
- Date Published:
- Journal Name:
- 20th International Conference on Soil Mechanics and Geotechnical Engineering
- Page Range / eLocation ID:
- 389-394
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The rapid progression of the COVID‐19 pandemic revealed an inability to meet increased demand for N95 respirators. These respirators are designed to be used once and disposed, but throughout the pandemic, there was a need for their decontamination and reuse. This research investigates the effect of various decontamination methods on the chemical and mechanical properties of N95 mask straps made of natural rubber to explore how these straps change after decontamination and what materials characterization techniques are well‐suited to evaluate these changes. Using results from ozone decontamination, tensile testing of mask strap assemblies is identified as the most reliable way to quantify changes in strap properties with decontamination and reuse when compared to other analytical techniques. Additionally, visible strap degradation often precedes both strap failure and material property changes and can be a reasonable indicator to discontinue use. Aside from ozone, decontamination with other methods such as heat and UV light appears to be less damaging to the tested materials. Beyond the specific results presented, this study provides insight on testing strategies that can be employed to move forward with evaluating new materials and decontamination methods for use in future pandemics or in more resource‐limited regions.more » « less
-
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We show that two classes of Maximum Mean Discrepancy (MMD) based tests attain this optimality on Rd, while the quadratictime Kernel Stein Discrepancy (KSD) based tests achieve the maximum exponential decay rate under a relaxed level constraint. Under the same performance metric, we proceed to show that the quadratic-time MMD based two-sample tests are also optimal for general two-sample problems, provided that kernels are bounded continuous and characteristic. Key to our approach are Sanov’s theorem from large deviation theory and the weak metrizable properties of the MMD and KSD.more » « less
-
Abstract Bone fractures commonly repair by forming a bridging structure called callus, which begins as soft tissue and gradually ossifies to restore rigidity to the bone. Virtual mechanical testing is a promising technique for image-based assessment of structural bone healing in both preclinical and clinical settings, but its accuracy depends on the validity of the material model used to assign tissue mechanical properties. The goal of this study was to develop a constitutive model for callus that captures the heterogeneity and biomechanical duality of the callus, which contains both soft tissue and woven bone. To achieve this, a large-scale optimization analysis was performed on 2363 variations of 3D finite element models derived from computed tomography (CT) scans of 33 osteotomized sheep under normal and delayed healing conditions. A piecewise material model was identified that produced high absolute agreement between virtual and physical tests by differentiating between soft and hard callus based on radiodensity. The results showed that the structural integrity of a healing long bone is conferred by an internal architecture of mineralized hard callus that is supported by interstitial soft tissue. These findings suggest that with appropriate material modeling, virtual mechanical testing is a reliable surrogate for physical biomechanical testing.more » « less
-
The ability to manufacture complex design geometries via Additive Manufacturing (AM) has led to a rapid growth in advancing the design methods, fabrication, and application of Triply Periodic Minimal Surface (TPMS) lattices with minimal surface topologies. Due to its zero-mean curvature, TPMS lattices can be additively manufactured without any sacrificial support structures and offer both design and manufacturing engineers, unprecedented control over the local physical properties (surface area, relative density, etc.) and local mechanical properties (flexural strength, Young’s modulus, etc.). TPMS lattices are of high interest for a wide range of applications such as biomedical implants, energy absorption, and surface fluidic applications such as heat exchangers, and energy storage. Recent advancements in functionally graded TPMS lattice design by varying local lattice geometry has shown to result in different mechanical performance. However, there have been limited studies in understanding the functional grading of AM process conditions (e.g., Laser-Powder Bed Fusion in this study) and lattice sheet thickness to better map the design-processing conditions-properties. The goal of this study is to achieve similar mechanical properties in TPMS sheet lattices with two different TPMS sheet thicknesses by varying laser processing conditions (e.g., contour and hatch conditions in this study). Quasi-static tensile testing of solid samples with corresponding AM conditions and 3-point bending tests of TPMS lattices were performed in accordance with ASTM E8 and ASTM E290, respectively. It was observed that the flexural properties of the 0.75 mm and 0.25 mm TPMS lattices are similar and exhibit different properties with different scan strategies and speed variations under contour-only and hatch-only laser scanning strategies. Also, the 0.75 mm TPMS sheet lattices exhibited 79 % higher flexural stiffness than the 0.25 mm sheet lattices. It was also observed that this observed trend was reversed in the case of tensile properties. Findings from this study can provide new directions towards achieving gradient TPMS lattice designs with varying local mechanical performance by grading the laser scanning strategies to achieve desired mechanical properties and surface topologies.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
