skip to main content

Title: Stochastic control and non-equilibrium thermodynamics: fundamental limits
We consider damped stochastic systems in a controlled (time-varying) potential and study their transition between specified Gibbs-equilibria states in finite time. By the second law of thermody- namics, the minimum amount of work needed to transition from one equilibrium state to another is the difference between the Helmholtz free energy of the two states and can only be achieved by a reversible (infinitely slow) process. The minimal gap between the work needed in a finite-time transition and the work during a reversible one, turns out to equal the square of the optimal mass transport (Wasserstein- 2) distance between the two end-point distributions times the inverse of the duration needed for the transition. This result, in fact, relates non-equilibrium optimal control strategies (protocols) to gradient flows of entropy functionals via the Jordan-Kinderlehrer-Otto scheme. The purpose of this paper is to introduce ideas and results from the emerging field of stochastic thermodynamics in the setting of classical regulator theory, and to draw connections and derive such fundamental relations from a control perspective in a multivariable setting.
Authors:
; ;
Award ID(s):
1665031 1901599 1839441 1807664
Publication Date:
NSF-PAR ID:
10114183
Journal Name:
IEEE Transactions on Automatic Control
Page Range or eLocation-ID:
1 to 1
ISSN:
0018-9286
Sponsoring Org:
National Science Foundation
More Like this
  1. We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chains by means of semimartingales and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control at the same time, but also extends the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game as well as how the equilibrium of a mean field game consists of an approximative Nash equilibrium for the game with a finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states.
  2. We develop a probabilistic approach to continuous-time finite state mean field games. Based on an alternative description of continuous-time Markov chain by means of semimartingale and the weak formulation of stochastic optimal control, our approach not only allows us to tackle the mean field of states and the mean field of control in the same time, but also extend the strategy set of players from Markov strategies to closed-loop strategies. We show the existence and uniqueness of Nash equilibrium for the mean field game, as well as how the equilibrium of mean field game consists of an approximative Nash equilibrium for the game with finite number of players under different assumptions of structure and regularity on the cost functions and transition rate between states.
  3. The theory of mean field games is a tool to understand noncooperative dynamic stochastic games with a large number of players. Much of the theory has evolved under conditions ensuring uniqueness of the mean field game Nash equilibrium. However, in some situations, typically involving symmetry breaking, non-uniqueness of solutions is an essential feature. To investigate the nature of non-unique solutions, this paper focuses on the technically simple setting where players have one of two states, with continuous time dynamics, and the game is symmetric in the players, and players are restricted to using Markov strategies. All the mean field game Nash equilibria are identified for a symmetric follow the crowd game. Such equilibria correspond to symmetric $\epsilon$-Nash Markov equilibria for $N$ players with $\epsilon$ converging to zero as $N$ goes to infinity. In contrast to the mean field game, there is a unique Nash equilibrium for finite $N.$ It is shown that fluid limits arising from the Nash equilibria for finite $N$ as $N$ goes to infinity are mean field game Nash equilibria, and evidence is given supporting the conjecture that such limits, among all mean field game Nash equilibria, are the ones that are stable fixed points of themore »mean field best response mapping.« less
  4. Service systems are typically limited resource environments where scarce capacity is reserved for the most urgent customers. However, there has been a growing interest in the use of proactive service when a less urgent customer may become urgent while waiting. On one hand, providing service for customers when they are less urgent could mean that fewer resources are needed to fulfill their service requirement. On the other hand, using limited capacity for customers who may never need the service in the future takes the capacity away from other more urgent customers who need it now. To understand this tension, we propose a multiserver queueing model with two customer classes: moderate and urgent. We allow customers to transition classes while waiting. In this setting, we characterize how moderate and urgent customers should be prioritized for service when proactive service for moderate customers is an option. We identify an index, the modified [Formula: see text]-index, which plays an important role in determining the optimal scheduling policy. This index lends itself to an intuitive interpretation of how to balance holding costs, service times, abandonments, and transitions between customer classes. This paper was accepted by David Simchi-Levi, stochastic models and simulation.
  5. The promise of self-assembly to enable the bottom-up formation of materials with prescribed architectures and functions has driven intensive efforts to uncover rational design principles for maximizing the yield of a target structure. Yet, despite many successful examples of self-assembly, ensuring kinetic accessibility of the target structure remains an unsolved problem in many systems. In particular, long-lived kinetic traps can result in assembly times that vastly exceed experimentally accessible timescales. One proposed solution is to design non-equilibrium assembly protocols in which system parameters change over time to avoid such kinetic traps. Here, we develop a framework to combine Markov state model (MSM) analysis with optimal control theory to compute a time-dependent protocol that maximizes the yield of the target structure at a finite time. We present an adjoint-based gradient descent method that, in conjunction with MSMs for a system as a function of its control parameters, enables efficiently optimizing the assembly protocol. We also describe an interpolation approach to significantly reduce the number of simulations required to construct the MSMs. We demonstrate our approach with two examples; a simple semi-analytic model for the folding of a polymer of colloidal particles, and a more complex model for capsid assembly. Our resultsmore »show that optimizing time-dependent protocols can achieve significant improvements in the yields of selected structures, including equilibrium free energy minima, long-lived metastable structures, and transient states.

    « less