An official website of the United States government
Policy testing is an important means for quality assurance of access control policies. Experimental studies on the testing methods of XACML policies have shown their varying levels of effectiveness. However, there is a lack of explanation for why they are unable to detect certain types of faults. It is unclear what is essential to the fault detection capability. To address this issue, we propose a theory on policy testing by formalizing the fault detection conditions with respect to a comprehensive fault model of XACML policies. The detection condition of a policy fault, composed of the reachability, necessity, and propagation constraints, is sufficient and necessary for revealing the fault. The formalized fault detection conditions can qualify the inherent strengths and limitations of testing methods. We have applied the formalization to the qualitative evaluations of five testing methods for the current version of the XACML standard. The results show that, for each method, there are certain types of faults that can always or never be revealed, while the detection of other faults may depend on the particular policy structure.
more »
« less
Automated Coverage-Based Testing of XACML Policies
While the standard language XACML is very expressive for specifying fine-grained access control policies, defects can get into XACML policies for various reasons, such as misunderstanding of access control requirements, omissions, and coding errors. These defects may result in unauthorized accesses, escalation of privileges, and denial of service. Therefore, quality assurance of XACML policies for real-world information systems has become an important issue. To address this issue, this paper presents a family of coverage criteria for XACML policies, such as rule coverage, rule pair coverage, decision coverage, and Modified Condition/Decision Coverage (MC/DC). To demonstrate the assurance levels of these coverage criteria, we have developed methods for automatically generating tests, i.e., access requests, to satisfy the coverage criteria using a constraint solver. We have evaluated these methods through mutation analysis of various policies with different levels of complexity. The experiment results have shown that the rule coverage is far from adequate for revealing the majority of defects in XACML policies, and that both MC/DC and decision coverage tests have outperformed the existing methods for testing XACML policies. In particular, MC/DC tests achieve a very high level of quality assurance of XACML policies.
more »
« less
- Award ID(s):
- 1618229
- PAR ID:
- 10376453
- Date Published:
- Journal Name:
- Proc. of the 23rd ACM Symposium on Access Control Models and Technologies
- Page Range / eLocation ID:
- 3 to 14
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The NGAC (Next Generation Access Control) standard for attribute-based access control (ABAC) allows for run-time changes of the permission and prohibition configurations through administrative obligations triggered by access events. It makes access control more fine-grained and dynamic. However, it raises challenges for assuring the correctness of NGAC policies. As policy testing is an important technique for quality assurance, this paper presents an approach to mutation analysis of NGAC policies. It can evaluate the effectiveness of a testing method and reveal potential faults in an inadequately tested policy. The mutation analysis covers various types of potential faults in the assignments, associations, prohibitions, and obligations of NGAC policies. This paper also proposes an incremental testing approach that first validates the initial configuration of a policy and then the policy as a whole. It helps determine whether faults appear in the configuration or the obligations. To evaluate the work, we have developed four working policies and their test suites based on the current NGAC reference implementation. The empirical studies show that the mutation analysis can shed light on the strengths and weaknesses of the test suites. They also demonstrate the need for developing more cost-effective testing methods.more » « less
-
While the existing methods for testing XACML policies have varying levels of effectiveness, none of them can reveal the majority of policy faults. The undisclosed faults may lead to unauthorized access and denial of service. This paper presents an approach to strong mutation testing of XACML policies that automatically generates tests from the mutants of a given policy. Such mutants represent the targeted faults that may appear in the policy. In this approach, we first compose the strong mutation constraints that capture the semantic difference between each mutant and its original policy. Then, we use a constraint solver to derive an access request (i.e., test). The test suite generated from all the mutants of a policy can achieve a perfect mutation score, thus uncover all hypothesized faults or demonstrate their absence. Based on the mutation-based approach, this paper further explores optimal test suite that achieves a perfect mutation score without duplicate tests. To evaluate the proposed approach, our experiments have included all the subject policies in the relevant literature and used a number of new policies. The results demonstrate that: (1) it is scalable to generate a mutation-based test suite to achieve a perfect mutation score, (2) it can be impractical to generate the optimal test suite due to the expensive removal of duplicate tests, (3) different from the results of the existing study, the modified-condition/decision coverage-based method, currently the most effective one, has low mutation scores for several policies.more » « less
-
Code coverage is the most widely adopted criteria for measuring test effectiveness in software quality assurance. The performance of coverage criteria (in indicating test suites' effectiveness) has been widely studied in prior work. Most of the studies use randomly constructed pseudo test suites to facilitate data collection for correlation analysis, yet no previous work has systematically studied whether pseudo test suites would lead to inflated correlation results. This paper focuses on the potentially wide-spread threat with a study over 123 real-world Java projects. Following the typical experimental process of studying coverage criteria, we investigate the correlation between statement/assertion coverage and mutation score using both pseudo and original test suites. Except for direct correlation analysis, we control the number of assertions and the test suite size to conduct partial correlation analysis. The results reveal that 1) the correlation (between coverage criteria and mutation score) derived from pseudo test suites is much higher than from original test suites (from 0.21 to 0.39 higher in Kendall value); 2) contrary to previously reported, statement coverage has a stronger correlation with mutation score than assertion coverage.more » « less
-
As AI-mediated communication (AI-MC) becomes more prevalent in everyday interactions, it becomes increasingly important to develop a rigorous understanding of its effects on interpersonal relationships and on society at large. Controlled experimental studies offer a key means of developing such an understanding, but various complexities make it difficult for experimental AI-MC research to simultaneously achieve the criteria of experimental realism, experimental control, and scalability. After outlining these methodological challenges, this paper offers the concept of methodological middle spaces as a means to address these challenges. This concept suggests that the key to simultaneously achieving all three of these criteria is to abandon the perfect attainment of any single criterion. This concept's utility is demonstrated via its use to guide the design of a platform for conducting text-based AI-MC experiments. Through a series of three example studies, the paper illustrates how the concept of methodological middle spaces can inform the design of specific experimental methods. Doing so enabled these studies to examine research questions that would have been either difficult or impossible to investigate using existing approaches. The paper concludes by describing how future research could similarly apply the concept of methodological middle spaces to expand methodological possibilities for AI-MC research in ways that enable contributions not currently possible.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3205977.3205979
