More Like this

1. Selector-Actor-Critic and Tuner-Actor-Critic Algorithms for Reinforcement Learning

   https://doi.org/10.1109/WCSP.2019.8928124  

   Masadeh, Ala'eddin; Wang, Zhengdao; Kamal, Ahmed E. (October 2019, International Conference on Wireless Communications and Signal Processing)
2. Average-Reward Soft Actor-Critic

   Adamczyk, Jacob; Makarenko, Volodymyr; Tiomkin, Stas; Kulkarni, Rahul V (May 2025, Open Review)

   The average-reward formulation of reinforcement learning (RL) has drawn increased interest in recent years for its ability to solve temporally-extended problems without relying on discounting. Meanwhile, in the discounted setting, algorithms with entropy regularization have been developed, leading to improvements over deterministic methods. Despite the distinct benefits of these approaches, deep RL algorithms for the entropy-regularized average-reward objective have not been developed. While policy-gradient based approaches have recently been presented for the average-reward literature, the corresponding actor-critic framework remains less explored. In this paper, we introduce an average-reward soft actor-critic algorithm to address these gaps in the field. We validate our method by comparing with existing average-reward algorithms on standard RL benchmarks, achieving superior performance for the average-reward criterion. 

   more » « less
3. Actor-Critic PAC Robust Policy Search

   Sheckells, Matthew; Garimella, Gowtham; Michra, Subhransu; Kobilarov, Marin (May 2019, ICRA 2019)

   This work studies an approach for computing provably robust control laws for robotic systems operating in uncertain environments. We develop an actor-critic style policy search algorithm based on the idea of minimizing an upper confidence bound on the negative expected advantage of a control policy at each policy update iteration. This new algorithm is a reformulation of Probably-Approximately-Correct Robust Policy Search (PROPS) and, unlike PROPS, allows for both step-based evaluation and step-based sampling strategies in policy parameter space, enabled by the use of Generalized Advantage Estimation and Generalized Exploration. As a result, the new algorithm is more data efficient and is expected to compute higher quality policies faster. We empirically evaluate the algorithm in simulation on a challenging robot navigation task using a high-fidelity deep stochastic model of an agile ground vehicle and compare its performance to the original trajectory-based PROPS 

   more » « less
4. Critic-over-Actor-Critic Modeling: Finding Optimal Strategy in ICU Environments

   https://doi.org/10.1109/BigData55660.2022.10021125  

   Ryan, Riazat; Shao, Ming (December 2022, 2022 IEEE International Conference on Big Data (Big Data))

   Reinforcement learning (RL) is mechanized to learn from experience. It solves the problem in sequential decisions by optimizing reward-punishment through experimentation of the distinct actions in an environment. Unlike supervised learning models, RL lacks static input-output mappings and the objective of minimization of a vector error. However, to find out an optimal strategy, it is crucial to learn both continuous feedback from training data and the offline rules of the experiences with no explicit dependence on online samples. In this paper, we present a study of a multi-agent RL framework which involves a Critic in semi-offline mode criticizing over an online Actor-Critic network, namely, Critic-over-Actor-Critic (CoAC) model, in finding optimal treatment plan of ICU patients as well as optimal strategy in a combative battle game. For further validation, we also examine the model in the adversarial assignment. 

   more » « less
5. MARS: Malleable Actor-Critic Reinforcement Learning Scheduler

   https://doi.org/10.1109/IPCCC55026.2022.9894315  

   Baheri, Betis; Tronge, Jacob; Fang, Bo; Li, Ang; Chaudhary, Vipin; Guan, Qiang (November 2022, 2022 IEEE International Performance, Computing, and Communications Conference (IPCCC))