Search for: All records

This work focuses on online reinforcement learning (RL) in the presence of en- vironmental constraints. Specifically, we consider applications involving robot agents exploring in an environment where obstacles and unsafe zones are present, and the agents must maximize cumulative rewards and at the same time meet the environmental constraints. To address this challenge, we formulate the prob- lem using the constrained Markov Decision Process (CMDP) and incorporate the environmental constraint costs into the policy updates in the proposed Aug- mented Proximal Policy Optimization (APPO) algorithm. At each state and for each possible action, we apply a Variational Auto-Encoder (VAE) [1] to obtain a probabilistic estimate of the discounted cumulative future environmental con- straint costs and integrate them as a regularization term to the reward function. This augmented reward function updates the action-value functions within the APPO algorithm, which is trained by an efficient optimization scheme. Ex- perimental results demonstrate that our methodology enables robot agents to navigate within the safety-constrained regions effectively.