An official website of the United States government
While the existing methods for testing XACML policies have varying levels of effectiveness, none of them can reveal the majority of policy faults. The undisclosed faults may lead to unauthorized access and denial of service. This paper presents an approach to strong mutation testing of XACML policies that automatically generates tests from the mutants of a given policy. Such mutants represent the targeted faults that may appear in the policy. In this approach, we first compose the strong mutation constraints that capture the semantic difference between each mutant and its original policy. Then, we use a constraint solver to derive an access request (i.e., test). The test suite generated from all the mutants of a policy can achieve a perfect mutation score, thus uncover all hypothesized faults or demonstrate their absence. Based on the mutation-based approach, this paper further explores optimal test suite that achieves a perfect mutation score without duplicate tests. To evaluate the proposed approach, our experiments have included all the subject policies in the relevant literature and used a number of new policies. The results demonstrate that: (1) it is scalable to generate a mutation-based test suite to achieve a perfect mutation score, (2) it can be impractical to generate the optimal test suite due to the expensive removal of duplicate tests, (3) different from the results of the existing study, the modified-condition/decision coverage-based method, currently the most effective one, has low mutation scores for several policies.
more »
« less
Mutation Analysis of NGAC Policies
The NGAC (Next Generation Access Control) standard for attribute-based access control (ABAC) allows for run-time changes of the permission and prohibition configurations through administrative obligations triggered by access events. It makes access control more fine-grained and dynamic. However, it raises challenges for assuring the correctness of NGAC policies. As policy testing is an important technique for quality assurance, this paper presents an approach to mutation analysis of NGAC policies. It can evaluate the effectiveness of a testing method and reveal potential faults in an inadequately tested policy. The mutation analysis covers various types of potential faults in the assignments, associations, prohibitions, and obligations of NGAC policies. This paper also proposes an incremental testing approach that first validates the initial configuration of a policy and then the policy as a whole. It helps determine whether faults appear in the configuration or the obligations. To evaluate the work, we have developed four working policies and their test suites based on the current NGAC reference implementation. The empirical studies show that the mutation analysis can shed light on the strengths and weaknesses of the test suites. They also demonstrate the need for developing more cost-effective testing methods.
more »
« less
- Award ID(s):
- 1954327
- PAR ID:
- 10376451
- Date Published:
- Journal Name:
- Proc. of the 26h ACM Symposium on Access Control Models and Technologies
- Page Range / eLocation ID:
- 71 to 82
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Policy testing is an important means for quality assurance of access control policies. Experimental studies on the testing methods of XACML policies have shown their varying levels of effectiveness. However, there is a lack of explanation for why they are unable to detect certain types of faults. It is unclear what is essential to the fault detection capability. To address this issue, we propose a theory on policy testing by formalizing the fault detection conditions with respect to a comprehensive fault model of XACML policies. The detection condition of a policy fault, composed of the reachability, necessity, and propagation constraints, is sufficient and necessary for revealing the fault. The formalized fault detection conditions can qualify the inherent strengths and limitations of testing methods. We have applied the formalization to the qualitative evaluations of five testing methods for the current version of the XACML standard. The results show that, for each method, there are certain types of faults that can always or never be revealed, while the detection of other faults may depend on the particular policy structure.more » « less
-
Greybox fuzzing and mutation testing are two popular but mostly independent fields of software testing research that have so far had limited overlap. Greybox fuzzing, generally geared towards searching for new bugs, predominantly uses code coverage for selecting inputs to save. Mutation testing is primarily used as a stronger alternative to code coverage in assessing the quality of regression tests; the idea is to evaluate tests for their ability to identify artificially injected faults in the target program. But what if we wanted to use greybox fuzzing to synthesize high-quality regression tests? In this paper, we develop and evaluate Mu2, a Java-based framework for incorporating mutation analysis in the greybox fuzzing loop, with the goal of producing a test-input corpus with a high mutation score. Mu2 makes use of a differential oracle for identifying inputs that exercise interesting program behavior without causing crashes. This paper describes several dynamic optimizations implemented in Mu2 to overcome the high cost of performing mutation analysis with every fuzzer-generated input. These optimizations introduce trade-offs in fuzzing throughput and mutation killing ability, which we evaluate empirically on five real-world Java benchmarks. Overall, variants of Mu2 are able to synthesize test-input corpora with a higher mutation score than state-of-the-art Java fuzzer Zest.more » « less
-
Code coverage is the most widely adopted criteria for measuring test effectiveness in software quality assurance. The performance of coverage criteria (in indicating test suites' effectiveness) has been widely studied in prior work. Most of the studies use randomly constructed pseudo test suites to facilitate data collection for correlation analysis, yet no previous work has systematically studied whether pseudo test suites would lead to inflated correlation results. This paper focuses on the potentially wide-spread threat with a study over 123 real-world Java projects. Following the typical experimental process of studying coverage criteria, we investigate the correlation between statement/assertion coverage and mutation score using both pseudo and original test suites. Except for direct correlation analysis, we control the number of assertions and the test suite size to conduct partial correlation analysis. The results reveal that 1) the correlation (between coverage criteria and mutation score) derived from pseudo test suites is much higher than from original test suites (from 0.21 to 0.39 higher in Kendall value); 2) contrary to previously reported, statement coverage has a stronger correlation with mutation score than assertion coverage.more » « less
-
Mining algorithms for relationship-based access control policies produce policies composed of relationship-based patterns that justify the input authorizations according to a given system graph. The correct functioning of a policy mining algorithm is typically tested based on experimental evaluations, in each of which the miner is presented with a set of authorizations and a system graph, and is expected to produce the corresponding ground truth policy. In this paper, we propose formal properties that must exist between the system graph and the ground truth policy in an evaluation test so that the miner is challenged to produce the exact ground truth policy. We show that failure to verify these properties in the experiment leads to inadequate evaluation, i.e., not truly testing whether the miner can handle the complexity of the ground truth policy. We also argue that following these properties would provide a computational advantage in the evaluations. We propose algorithms to identify and correct violations of these properties in system graphs. We also present our observations regarding these properties and their enforcement using a set of experimental studies.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3450569.3463563
