More Like this

1. A Distributed Douglas-Rachford Based Algorithm for Stochastic GNE Seeking with Partial Information

   Huang, Yuanhanqing and ( June 2022 , 2022 American Control Conference)

   We consider the stochastic generalized Nash equilibrium problem (SGNEP) where a set of self-interested players, subject to certain global constraints, aim to optimize their local objectives that depend on their own decisions and the decisions of others and are influenced by some random factors. A distributed stochastic generalized Nash equilibrium seeking algorithm is proposed based on the Douglas-Rachford operator splitting scheme, which only requires local communications among neighbors. The proposed scheme significantly relaxes assumptions on co-coercivity and contractiveness in the existing literature, where the projected stochastic subgradient method is applied to provide approximate solutions to the augmented best-response subproblems for each player. Finally, we illustrate the validity of the proposed algorithm through a Nash-Cournot production game. 

   more » « less
2. Distributed Solution of GNEP over Networks via the Douglas-Rachford Splitting Method

   https://doi.org/10.1109/CDC45484.2021.9682862  

   Huang, Yuanhanqing ; Hu, Jianghai ( December 2021 , 60th IEEE Conference on Decision and Control (CDC))

   The aim of this paper is to find the distributed solution of the generalized Nash equilibrium problem (GNEP) for a group of players that can communicate with each other over a connected communication network. Each player tries to minimize a local objective function of its own that may depend on the other players’ decisions, and collectively all the players’ decisions are subject to some globally shared resource constraints. After reformulating the local optimization problems, we introduce the notion of network Lagrangian and recast the GNEP as the zero finding problem of a properly defined operator. Utilizing the Douglas-Rachford operator splitting method, a distributed algorithm is proposed that requires only local information exchanges between neighboring players in each iteration. The convergence of the proposed algorithm to an exact variational generalized Nash equilibrium is established under two different sets of assumptions. The effectiveness of the proposed algorithm is demonstrated using the example of a Nash-Cournot production game. 

   more » « less
3. Learning Nash Equilibria in Zero-Sum Stochastic Games via Entropy-Regularized Policy Approximation

   Q. Zhang, Y. Guan ( January 2021 , 30th International Joint Conference on Artificial Intelligence)

   We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to approximate the Nash policy during the Q-function updates. We prove that under certain conditions, by updating the regularized Q-function, the algorithm converges to a Nash equilibrium. We also demonstrate the proposed algorithm’s ability to transfer previous training experiences, enabling the agents to adapt quickly to new environments. We provide a dynamic hyper-parameter scheduling scheme to further expedite convergence. Empirical results applied to a number of stochastic games verify that the proposed algorithm converges to the Nash equilibrium while exhibiting a major speed-up over existing algorithms. 

   more » « less
4. Equilibrium analysis of electricity markets with day-ahead market power mitigation and real-time intercept bidding

   https://doi.org/10.1145/3538637.3538839  

   Bansal, Rajni Kant ; Chen, Yue ; You, Pengcheng ; Mallada, Enrique ( June 2022 , Proceedings of the Thirteenth ACM International Conference on Future Energy Systems (e-Energy))

   Electricity markets are cleared by a two-stage, sequential process consisting of a forward (day-ahead) market and a spot (real-time) market. While their design goal is to achieve efficiency, the lack of sufficient competition introduces many opportunities for price manipulation. To discourage this phenomenon, some Independent System Operators (ISOs) mandate generators to submit (approximately) truthful bids in the day-ahead market. However, without fully accounting for all participants' incentives (generators and loads), the application of such a mandate may lead to unintended consequences. In this paper, we model and study the interactions of generators and inelastic loads in a two-stage settlement where generators are required to bid truthfully in the day-ahead market. We show that such mandate, when accounting for generator and load incentives, leads to a {generalized} Stackelberg-Nash game where load decisions (leaders) are performed in day-ahead market and generator decisions (followers) are relegated to the real-time market. Furthermore, the use of conventional supply function bidding for generators in real-time, does not guarantee the existence of a Nash equilibrium. This motivates the use of intercept bidding, as an alternative bidding mechanism for generators in the real-time market. An equilibrium analysis in this setting, leads to a closed-form solution that unveils several insights. Particularly, it shows that, unlike standard two-stage markets, loads are the winners of the competition in the sense that their aggregate payments are less than that of the competitive equilibrium. Moreover, heterogeneity in generators cost has the unintended effect of mitigating loads market power. Numerical studies validate and further illustrate these insights. 

   more » « less
5. Convergence of deep fictitious play for stochastic differential games

   https://doi.org/10.3934/fmf.2021011  

   Han, Jiequn ; Hu, Ruimeng ; Long, Jihao ( January 2022 , Frontiers of Mathematical Finance)

   Stochastic differential games have been used extensively to model agents' competitions in finance, for instance, in P2P lending platforms from the Fintech industry, the banking system for systemic risk, and insurance markets. The recently proposed machine learning algorithm, deep fictitious play, provides a novel and efficient tool for finding Markovian Nash equilibrium of large \begin{document}$ N $\end{document}-player asymmetric stochastic differential games [J. Han and R. Hu, Mathematical and Scientific Machine Learning Conference, pages 221-245, PMLR, 2020]. By incorporating the idea of fictitious play, the algorithm decouples the game into \begin{document}$ N $\end{document} sub-optimization problems, and identifies each player's optimal strategy with the deep backward stochastic differential equation (BSDE) method parallelly and repeatedly. In this paper, we prove the convergence of deep fictitious play (DFP) to the true Nash equilibrium. We can also show that the strategy based on DFP forms an \begin{document}$ \epsilon $\end{document}-Nash equilibrium. We generalize the algorithm by proposing a new approach to decouple the games, and present numerical results of large population games showing the empirical convergence of the algorithm beyond the technical assumptions in the theorems.

    

   more » « less