More Like this

1. Statistically Efficient Off-Policy Policy Gradients

   Kallus, Nathan; Uehara, Masatoshi (January 2020, International Conference on Machine Learning)
2. Modular Policy Composition with Policy Centroids

   Adhikaray, Sandesh; Boots, Byron (April 2022, The Multi-disciplinary Conference on Reinforcement Learning and Decision Making)
3. Representation Balancing MDPsfor Off-Policy Policy Evaluation

   Liu, Y; Gottesman, O; Raghu, A; Komorowski, M; Faisal, A; Doshi-Velez, F; Brunskill, E (January 2018, Advances in neural information processing systems)
4. Data-Efficient Policy Evaluation Through Behavior Policy Search

   Hanna, J; Thomas, P; Stone, P; Niekum, S (August 2017, International Conference on Machine Learning)
5. Data-Efficient Policy Evaluation Through Behavior Policy Search

   Hanna, Josiah P; Chandak, Yash; Thomas, Philip S; White, Martha; Stone, Peter; Niekum, Scott (October 2024, Journal of machine learning research)

   Ravikumar, Pradeep (Ed.)

   We consider the task of evaluating a policy for a Markov decision process (MDP). The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance. We show that the data collected from deploying a different policy, commonly called the behavior policy, can be used to produce unbiased estimates with lower mean squared error than this standard technique. We derive an analytic expression for a minimal variance behavior policy -- a behavior policy that minimizes the mean squared error of the resulting estimates. Because this expression depends on terms that are unknown in practice, we propose a novel policy evaluation sub-problem, behavior policy search: searching for a behavior policy that reduces mean squared error. We present two behavior policy search algorithms and empirically demonstrate their effectiveness in lowering the mean squared error of policy performance estimates. 

   more » « less