An official website of the United States government
Detection of malicious behavior is a fundamental problem in security. One of the major challenges in using detection systems in practice is in dealing with an overwhelming number of alerts that are triggered by normal behavior (the so-called false positives), obscuring alerts resulting from actual malicious activity. While numerous methods for reducing the scope of this issue have been proposed, ultimately one must still decide how to prioritize which alerts to investigate, and most existing prioritization methods are heuristic, for example, based on suspiciousness or priority scores. We introduce a novel approach for computing a policy for prioritizing alerts using adversarial reinforcement learning. Our approach assumes that the attackers know the full state of the detection system and dynamically choose an optimal attack as a function of this state, as well as of the alert prioritization policy. The first step of our approach is to capture the interaction between the defender and attacker in a game theoretic model. To tackle the computational complexity of solving this game to obtain a dynamic stochastic alert prioritization policy, we propose an adversarial reinforcement learning framework. In this framework, we use neural reinforcement learning to compute best response policies for both the defender and the adversary to an arbitrary stochastic policy of the other. We then use these in a double-oracle framework to obtain an approximate equilibrium of the game, which in turn yields a robust stochastic policy for the defender. Extensive experiments using case studies in fraud and intrusion detection demonstrate that our approach is effective in creating robust alert prioritization policies.
more »
« less
Reachability-Based Covariance Control for Pursuit-Evasion in Stochastic Flow Fields
In this paper, pursuit-evasion scenarios in a stochastic flow field involving one pursuer and one evader are analyzed. Using a forward reachability set-based approach and the associated level set equations, nominal solutions of the players are generated. The dynamical system is linearized along the nominal solution to formulate a chance-constrained, linear-quadratic stochastic dynamic game. Assuming an affine disturbance feedback structure, the proposed game is solved using the standard Gauss-Seidel iterative scheme. Numerical simulations demonstrate the proposed approach for realistic flow fields.
more »
« less
- Award ID(s):
- 1662542
- PAR ID:
- 10318546
- Date Published:
- Journal Name:
- AIAA SciTech Forum, Guidance, Navigation, and Control
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This article poses the following problem: Does there exist a probability distribution over subsets of a finite partially ordered set (poset), such that a set of constraints involving marginal probabilities of the poset’s elements and maximal chains is satisfied? We present a combinatorial algorithm to positively resolve this question. The algorithm can be implemented in polynomial time in the special case where maximal chain probabilities are affine functions of their elements. This existence problem is relevant for the equilibrium characterization of a generic strategic interdiction game on a capacitated flow network. The game involves a routing entity that sends its flow through the network while facing path transportation costs and an interdictor who simultaneously interdicts one or more edges while facing edge interdiction costs. Using our existence result on posets and strict complementary slackness in linear programming, we show that the Nash equilibria of this game can be fully described using primal and dual solutions of a minimum-cost circulation problem. Our analysis provides a new characterization of the critical components in the interdiction game. It also leads to a polynomial-time approach for equilibrium computation.more » « less
-
We consider the stochastic generalized Nash equilibrium problem (SGNEP) where a set of self-interested players, subject to certain global constraints, aim to optimize their local objectives that depend on their own decisions and the decisions of others and are influenced by some random factors. A distributed stochastic generalized Nash equilibrium seeking algorithm is proposed based on the Douglas-Rachford operator splitting scheme, which only requires local communications among neighbors. The proposed scheme significantly relaxes assumptions on co-coercivity and contractiveness in the existing literature, where the projected stochastic subgradient method is applied to provide approximate solutions to the augmented best-response subproblems for each player. Finally, we illustrate the validity of the proposed algorithm through a Nash-Cournot production game.more » « less
-
We consider the stochastic generalized Nash equilibrium problem (SGNEP) where a set of self-interested players, subject to certain global constraints, aim to optimize their local objectives that depend on their own decisions and the decisions of others and are influenced by some random factors. A distributed stochastic generalized Nash equilibrium seeking algorithm is proposed based on the Douglas-Rachford operator splitting scheme, which only requires local communications among neighbors. The proposed scheme significantly relaxes assumptions on co-coercivity and contractiveness in the existing literature, where the projected stochastic subgradient method is applied to provide approximate solutions to the augmented best-response subproblems for each player. Finally, we illustrate the validity of the proposed algorithm through a Nash-Cournot production game.more » « less
-
We propose a novel methodology for fault detection and diagnosis in partially-observed Boolean dynamical systems (POBDS). These are stochastic, highly nonlinear, and derivative- less systems, rendering difficult the application of classical fault detection and diagnosis methods. The methodology comprises two main approaches. The first addresses the case when the normal mode of operation is known but not the fault modes. It applies an innovations filter (IF) to detect deviations from the nominal normal mode of operation. The second approach is applicable when the set of possible fault models is finite and known, in which case we employ a multiple model adaptive estimation (MMAE) approach based on a likelihood-ratio (LR) statistic. Unknown system parameters are estimated by an adaptive expectation- maximization (EM) algorithm. Particle filtering techniques are used to reduce the computational complexity in the case of systems with large state-spaces. The efficacy of the proposed methodology is demonstrated by numerical experiments with a large gene regulatory network (GRN) with stuck-at faults observed through a single noisy time series of RNA-seq gene expression measurements.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.2514/6.2022-1382
