Search for: All records

This paper proposes a Bellman Deviation algorithm for the detection of man-in-the-middle (MITM) attacks occurring when an agent controls a Markov Decision Process (MDP) system using model-free reinforcement learning. This algorithm is derived by constructing a "Bellman Deviation sequence" and finding stochastic bounds on its running sequence average. We show that an intuitive, necessary and sufficient "informational advantage" condition must be met for the proposed algorithm to guarantee the detection of attacks with high probability, while also avoiding false alarms. 

Networked control systems, where feedback loops are closed over communication networks, arise in several domains, including smart energy grids, autonomous driving, unmanned aerial vehicles, and many industrial and robotic systems active in service, production, agriculture, and smart homes and cities. In these settings, the two main layers of the system, control and communication, strongly affect each other's performance, and they also reveal the interaction between a cyber-system component, represented by information-based computing and communication technologies, and a physical-system component, represented by the environment that needs to be controlled. The information access and distribution constraints required to achieve reliable state estimation and stabilization in networked control systems have been intensively studied over the course of roughly two decades. This article reviews some of the cornerstone results in this area, draws a map for what we have learned over these years, and describes the new challenges that we will face in the future. Rather than simply listing different results, we present them in a coherent fashion using a uniform notation, and we also put them in context, highlighting both their theoreticalinsights and their practical significance. Particular attention is given to recent developments related to decentralized estimation in distributed sensing and communication systems and the information-theoretic value of event timing in the context of networked control.