An official website of the United States government
We develop a new dynamic continuous-time model of optimal consumption and investment to include independent stochastic labor income. We reduce the problem of solving the Bellman equation to a problem of solving an integral equation. We then explicitly characterize the optimal consumption and investment strategy as a function of income-to-wealth ratio. We provide some analytical comparative statics associated with the value function and optimal strategies. We also develop a quite general numerical algorithm for control iteration and solve the Bellman equation as a sequence of solutions to ordinary differential equations. This numerical algorithm can be readily applied to many other optimal consumption and investment problems especially with extra nondiversifiable Brownian risks, resulting in nonlinear Bellman equations. Finally, our numerical analysis illustrates how the presence of stochastic labor income affects the optimal consumption and investment strategy. Funding: A. Bensoussan was supported by the National Science Foundation under grant [DMS-2204795]. S. Park was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea, South Korea [NRF-2022S1A3A2A02089950].
more »
« less
Detection of Man-in-the-Middle Attacks in Model-Free Reinforcement Learning
This paper proposes a Bellman Deviation algorithm for the detection of man-in-the-middle (MITM) attacks occurring when an agent controls a Markov Decision Process (MDP) system using model-free reinforcement learning. This algorithm is derived by constructing a "Bellman Deviation sequence" and finding stochastic bounds on its running sequence average. We show that an intuitive, necessary and sufficient "informational advantage" condition must be met for the proposed algorithm to guarantee the detection of attacks with high probability, while also avoiding false alarms.
more »
« less
- Award ID(s):
- 2127946
- PAR ID:
- 10466156
- Editor(s):
- Nikolai Matni, Manfred Morari
- Date Published:
- Journal Name:
- Proceedings of Machine Learning Research
- Volume:
- 211
- ISSN:
- 2640-3498
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In centralized multi-robot systems, a central entity (CE) checks that robots follow their assigned motion plans by comparing their expected location to the location they self-report. We show that this self-reporting monitoring mechanism is vulnerable to plan- deviation attacks where compromised robots don’t follow their assigned plans while trying to conceal their movement by misreporting their location. We propose a two-pronged mitigation for plan-deviation attacks: (1) an attack detection technique leveraging both the robots’ local sensing capabilities to report observations of other robots and co-observation schedules generated by the CE, and (2) a prevention technique where the CE issues horizon-limiting announcements to the robots, reducing their instantaneous knowledge of forward lookahead steps in the global motion plan. On a large-scale automated warehouse benchmark, we show that our solution enables attack prevention guarantees from a stealthy attacker that has compromised multiple robots.more » « less
-
While many research efforts on Cyber-Physical System (CPS) security are devoted to attack detection, how to respond to the detected attacks receives little attention. Attack response is essential since serious consequences can be caused if CPS continues to act on the compromised data by the attacks. In this work, we aim at the response to sensor attacks and adapt machine learning techniques to recover CPSs from such attacks. There are, however, several major challenges. i) Cumulative error. Recovery needs to estimate the current state of a physical system (e.g., the speed of a vehicle) in order to know if the system has been driven to a certain state. However, the estimation error accumulates over time in presence of compromised sensors. ii) Timely response. A fast response is needed since slow recovery not only comes with large estimation errors but also may be too late to avoid irreparable consequences. To address these challenges, we propose a novel learning-based solution, named sequence-predictive recovery (or SeqRec). To reduce the estimation error, SeqRec designs the first sequence-to-sequence (Seq2Seq) model to uncover the temporal and spatial dependencies among sensors and control demands, and then uses the model to estimate system states using the trustworthy data logged in history. To achieve an adequate and fast recovery, SeqRec designs the second Seq2Seq model that considers both the current time step using the remaining intact sensors and the future time steps based on a given target state, and embeds the model into a novel recovery control algorithm to drive a physical system back to that state. Experimental results demonstrate that SeqRec can effectively and efficiently recover CPSs from sensor attacks.more » « less
-
Recently, adversarial examples against object detection have been widely studied. However, it is difficult for these attacks to have an impact on visual perception in autonomous driving because the complete visual pipeline of real-world autonomous driving systems includes not only object detection but also object tracking. In this paper, we present a novel tracker hijacking attack against the multi-target tracking algorithm employed by real-world autonomous driving systems, which controls the bounding box of object detection to spoof the multiple object tracking process. Our approach exploits the detection box generation process of the anchor-based object detection algorithm and designs new optimization methods to generate adversarial patches that can successfully perform tracker hijacking attacks, causing security risks. The evaluation results show that our approach has 85% attack success rate on two detection models employed by real-world autonomous driving systems. We discuss our potential next step for this work.more » « less
-
null (Ed.)Spurious power consumption data reported from compromised meters controlled by organized adversaries in the Advanced Metering Infrastructure (AMI) may have drastic consequences on a smart grid’s operations. While existing research on data falsification in smart grids mostly defends against isolated electricity theft, we introduce a taxonomy of various data falsification attack types, when smart meters are compromised by organized or strategic rivals. To counter these attacks, we first propose a coarse-grained and a fine-grained anomaly-based security event detection technique that uses indicators such as deviation and directional change in the time series of the proposed anomaly detection metrics to indicate: (i) occurrence, (ii) type of attack, and (iii) attack strategy used, collectively known as attack context . Leveraging the attack context information, we propose three attack response metrics to the inferred attack context: (a) an unbiased mean indicating a robust location parameter; (b) a median absolute deviation indicating a robust scale parameter; and (c) an attack probability time ratio metric indicating the active time horizon of attacks. Subsequently, we propose a trust scoring model based on Kullback-Leibler (KL) divergence, that embeds the appropriate unbiased mean, the median absolute deviation, and the attack probability ratio metric at runtime to produce trust scores for each smart meter. These trust scores help classify compromised smart meters from the non-compromised ones. The embedding of the attack context, into the trust scoring model, facilitates accurate and rapid classification of compromised meters, even under large fractions of compromised meters, generalize across various attack strategies and margins of false data. Using real datasets collected from two different AMIs, experimental results show that our proposed framework has a high true positive detection rate, while the average false alarm and missed detection rates are much lesser than 10% for most attack combinations for two different real AMI micro-grid datasets. Finally, we also establish fundamental theoretical limits of the proposed method, which will help assess the applicability of our method to other domains.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
