More Like this

1. Flexible control as surrogate reward or dynamic reward maximization

   https://doi.org/10.1016/j.cognition.2022.105262  

   Liljeholm, Mimi (December 2022, Cognition)
2. Self-Supervised Online Reward Shaping in Sparse-Reward Environments

   Memarian, F; Goo, W; Lioutikov, R; Niekum, S; Topcu, U (September 2021, Proceedings of the IEEERSJ International Conference on Intelligent Robots and Systems)

   null (Ed.)
3. Causal Confusion and Reward Misidentification in Preference-Based Reward Learning

   J. Tien, D. Brown (January 2023, International Conference on Learning Representations)
4. Omega-Regular Reward Machines

   Hahn, E M; Perez, M; Schewe, S; Somenzi, F; Trivedi, A; Wojtczak, D (October 2023, ECAI 2023 (Open Access by IOS Press))

   Reinforcement learning (RL) is a powerful approach for training agents to perform tasks, but designing an appropriate re- ward mechanism is critical to its success. However, in many cases, the complexity of the learning objectives goes beyond the capabili- ties of the Markovian assumption, necessitating a more sophisticated reward mechanism. Reward machines and ω-regular languages are two formalisms used to express non-Markovian rewards for quantita- tive and qualitative objectives, respectively. This paper introduces ω- regular reward machines, which integrate reward machines with ω- regular languages to enable an expressive and effective reward mech- anism for RL. We present a model-free RL algorithm to compute ε-optimal strategies against ω-regular reward machines and evaluate the effectiveness of the proposed algorithm through experiments. 

   more » « less
5. Assisted Robust Reward Design

   He, Jerry Zhi-Yang; Dragan, Anca D. (November 2021, Conference on Robot Learning)

   Real-world robotic tasks require complex reward functions. When we define the problem the robot needs to solve, we pretend that a designer specifies this complex reward exactly, and it is set in stone from then on. In practice, however, reward design is an iterative process: the designer chooses a reward, eventually encounters an "edge-case" environment where the reward incentivizes the wrong behavior, revises the reward, and repeats. What would it mean to rethink robotics problems to formally account for this iterative nature of reward design? We propose that the robot not take the specified reward for granted, but rather have uncertainty about it, and account for the future design iterations as future evidence. We contribute an Assisted Reward Design method that speeds up the design process by anticipating and influencing this future evidence: rather than letting the designer eventually encounter failure cases and revise the reward then, the method actively exposes the designer to such environments during the development phase. We test this method in a simplified autonomous driving task and find that it more quickly improves the car's behavior in held-out environments by proposing environments that are "edge cases" for the current reward. 

   more » « less