We consider damped stochastic systems in a controlled (time-varying) potential and study their transition between specified Gibbs-equilibria states in finite time. By the second law of thermody- namics, the minimum amount of work needed to transition from one equilibrium state to another is the difference between the Helmholtz free energy of the two states and can only be achieved by a reversible (infinitely slow) process. The minimal gap between the work needed in a finite-time transition and the work during a reversible one, turns out to equal the square of the optimal mass transport (Wasserstein- 2) distance between the two end-point distributions times the inverse of the duration needed for the transition. This result, in fact, relates non-equilibrium optimal control strategies (protocols) to gradient flows of entropy functionals via the Jordan-Kinderlehrer-Otto scheme. The purpose of this paper is to introduce ideas and results from the emerging field of stochastic thermodynamics in the setting of classical regulator theory, and to draw connections and derive such fundamental relations from a control perspective in a multivariable setting.
more »
« less
Driving rapidly while remaining in control: classical shortcuts from Hamiltonian to stochastic dynamics
Abstract Stochastic thermodynamics lays down a broad framework to revisit the venerable concepts of heat, work and entropy production for individual stochastic trajectories of mesoscopic systems. Remarkably, this approach, relying on stochastic equations of motion, introduces time into the description of thermodynamic processes—which opens the way to fine control them. As a result, the field of finite-time thermodynamics of mesoscopic systems has blossomed. In this article, after introducing a few concepts of control for isolated mechanical systems evolving according to deterministic equations of motion, we review the different strategies that have been developed to realize finite-time state-to-state transformations in both over and underdamped regimes, by the proper design of time-dependent control parameters/driving. The systems under study are stochastic, epitomized by a Brownian object immersed in a fluid; they are thus strongly coupled to their environment playing the role of a reservoir. Interestingly, a few of those methods (inverse engineering, counterdiabatic driving, fast-forward) are directly inspired by their counterpart in quantum control. The review also analyzes the control through reservoir engineering. Besides the reachability of a given target state from a known initial state, the question of the optimal path is discussed. Optimality is here defined with respect to a cost function, a subject intimately related to the field of information thermodynamics and the question of speed limit. Another natural extension discussed deals with the connection between arbitrary states or non-equilibrium steady states. This field of control in stochastic thermodynamics enjoys a wealth of applications, ranging from optimal mesoscopic heat engines to population control in biological systems.
more »
« less
- Award ID(s):
- 2127900
- NSF-PAR ID:
- 10404872
- Date Published:
- Journal Name:
- Reports on Progress in Physics
- Volume:
- 86
- Issue:
- 3
- ISSN:
- 0034-4885
- Page Range / eLocation ID:
- 035902
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Chen, Ho-Lin ; Evans, Constantine G. (Ed.)Polynomial time dynamic programming algorithms play a crucial role in the design, analysis and engineering of nucleic acid systems including DNA computers and DNA/RNA nanostructures. However, in complex multistranded or pseudoknotted systems, computing the minimum free energy (MFE), and partition function of nucleic acid systems is NP-hard. Despite this, multistranded and/or pseudoknotted systems represent some of the most utilised and successful systems in the field. This leaves open the tempting possibility that many of the kinds of multistranded and/or pseudoknotted systems we wish to engineer actually fall into restricted classes, that do in fact have polynomial time algorithms, but we've just not found them yet. Here, we give polynomial time algorithms for MFE and partition function calculation for a restricted kind of multistranded system called the 1D scaffolded DNA computer. This model of computation thermodynamically favours correct outputs over erroneous states, simulates finite state machines in 1D and Boolean circuits in 2D, and is amenable to DNA storage applications. In an effort to begin to ask the question of whether we can naturally compare the expressivity of nucleic acid systems based on the computational complexity of prediction of their preferred energetic states, we show our MFE problem is in logspace (the complexity class L), making it perhaps one of the simplest known, natural, nucleic acid MFE problems. Finally, we provide a stochastic kinetic simulator for the 1D scaffolded DNA computer and evaluate strategies for efficiently speeding up this thermodynamically favourable system in a constant-temperature kinetic regime.more » « less
-
This paper studies the adaptive optimal control problem for a class of linear time-delay systems described by delay differential equations (DDEs). A crucial strategy is to take advantage of recent developments in reinforcement learning (RL) and adaptive dynamic programming (ADP) and develop novel methods to learn adaptive optimal controllers from finite samples of input and state data. In this paper, the data-driven policy iteration (PI) is proposed to solve the infinite-dimensional algebraic Riccati equation (ARE) iteratively in the absence of exact model knowledge. Interestingly, the proposed recursive PI algorithm is new in the present context of continuous-time time-delay systems, even when the model knowledge is assumed known. The efficacy of the proposed learning-based control methods is validated by means of practical applications arising from metal cutting and autonomous driving.more » « less
-
null (Ed.)We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chains by means of semimartingales and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control at the same time, but also extends the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game as well as how the equilibrium of a mean field game consists of an approximative Nash equilibrium for the game with a finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states.more » « less
-
We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chain by means of semimartingale and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control in the same time, but also extend the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game, as well as how the equilibrium of mean field game consists of an approximative Nash equilibrium for the game with finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1088/1361-6633/acacad
