More Like this

1. Data-Efficient Policy Evaluation Through Behavior Policy Search

   Hanna, Josiah P; Chandak, Yash; Thomas, Philip S; White, Martha; Stone, Peter; Niekum, Scott (October 2024, Journal of machine learning research)

   Ravikumar, Pradeep (Ed.)

   We consider the task of evaluating a policy for a Markov decision process (MDP). The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance. We show that the data collected from deploying a different policy, commonly called the behavior policy, can be used to produce unbiased estimates with lower mean squared error than this standard technique. We derive an analytic expression for a minimal variance behavior policy -- a behavior policy that minimizes the mean squared error of the resulting estimates. Because this expression depends on terms that are unknown in practice, we propose a novel policy evaluation sub-problem, behavior policy search: searching for a behavior policy that reduces mean squared error. We present two behavior policy search algorithms and empirically demonstrate their effectiveness in lowering the mean squared error of policy performance estimates. 

   more » « less
2. Model-Based Offline Meta-Reinforcement Learning with Regularization

   Lin, Sen; Wan, Jialin; Xu, Tengyu; Liang, Yingbin; Zhang. Junshan (April 2022, The Tenth International Conference on Learning Representations)

   Existing offline reinforcement learning (RL) methods face a few major challenges, particularly the distributional shift between the learned policy and the behavior policy. Offline Meta-RL is emerging as a promising approach to address these challenges, aiming to learn an informative meta-policy from a collection of tasks. Nevertheless, as shown in our empirical studies, offline Meta-RL could be outperformed by offline single-task RL methods on tasks with good quality of datasets, indicating that a right balance has to be delicately calibrated between "exploring" the out-of-distribution state-actions by following the meta-policy and "exploiting" the offline dataset by staying close to the behavior policy. Motivated by such empirical analysis, we propose model-based offline ta-RL with regularized policy optimization (MerPO), which learns a meta-model for efficient task structure inference and an informative meta-policy for safe exploration of out-of-distribution state-actions. In particular, we devise a new meta-Regularized model-based Actor-Critic (RAC) method for within-task policy optimization, as a key building block of MerPO, using both conservative policy evaluation and regularized policy improvement; and the intrinsic tradeoff therein is achieved via striking the right balance between two regularizers, one based on the behavior policy and the other on the meta-policy. We theoretically show that the learnt policy offers guaranteed improvement over both the behavior policy and the meta-policy, thus ensuring the performance improvement on new tasks via offline Meta-RL. Our experiments corroborate the superior performance of MerPO over existing offline Meta-RL methods. 

   more » « less
3. Emotions and policy information predicting water-quality policy support

   https://doi.org/10.1016/j.jenvp.2024.102385  

   Swim, Janet K; Guerriero, Joseph G; Gasper, Karen; DeCoster, Jamie; Lengieza, Micheal L (September 2024, Journal of Environmental Psychology)

   The public acknowledges the importance of water quality, and threats to water quality can provoke strong emotional responses. Despite this, the public often resists policies protecting water quality. Research with 349 US residents demonstrated that (1) emotions about specific water policies were more predictive of policy support than emotions about water quality and (2) hope about water policies was a particularly strong predictor of water policy support. In both between-person and within-person analyses, water-policy hope was a stronger predictor of water-policy support than water-policy anxiety, anger, and neutral affect–although these other emotions were related to water-policy support. These findings among water-policy emotions replicated results from a Pilot study with 148 US undergraduate students. The main study also demonstrated that water-policy support increased when policy descriptions explained how policies would improve water quality via hydro systems, and it did so by increasing feelings of water-policy hope. This research suggests that a full range of affective reactions to water policy and water quality should be considered when motivating support for policies protecting water quality. 

   more » « less
4. The Future of HCI-Policy Collaboration

   https://doi.org/10.1145/3613904.3642771  

   Yang, Qian; Wong, Richmond Y; Jackson, Steven; Junginger, Sabine; Hagan, Margaret D; Gilbert, Thomas; Zimmerman, John (May 2024, ACM)

   Policies significantly shape computation’s societal impact, a crucial HCI concern. However, challenges persist when HCI professionals attempt to integrate policy into their work or affect policy outcomes. Prior research considered these challenges at the “border” of HCI and policy. This paper asks: What if HCI considers policy integral to its intellectual concerns, placing system-people-policy interaction not at the border but nearer the center of HCI research, practice, and education? What if HCI fosters a mosaic of methods and knowledge contributions that blend system, human, and policy expertise in various ways, just like HCI has done with blending system and human expertise? We present this re-imagined HCI-policy relationship as a provocation and highlight its usefulness: It spotlights previously overlooked system-people-policy interaction work in HCI. It unveils new opportunities for HCI’s futuring, empirical, and design projects. It allows HCI to coordinate its diverse policy engagements, enhancing its collective impact on policy outcomes. 

   more » « less
5. Measuring interest group agendas in regulatory proposals: a method and the case of US education policy

   https://doi.org/10.1057/s41309-021-00129-w  

   Workman, Samuel; Carlson, Deven; Bark, Tracey; Bell, Elizabeth (January 2021, Interest Groups & Advocacy)

   null (Ed.)

   We introduce a new way to measure interest group agendas and demonstrate an approach to extending the CAP topic coding scheme to policy domains at lower levels of analysis. We use public comments on regulatory proposals in US education policy to examine the topics contained in policy arguments. We map the education policy space using a data set of 493 comments and 5315 hand-coded comment paragraphs. A unique measurement model accounts for group and topic diversity and allows us to validate our approach. The findings have implications for measuring topic agendas in lower-level policy domains and understanding group coalitions and competition in education policy. We contribute to text-as-data approaches tracing policy change in the study of public policy. The findings suggest the relationship between issue attention observed by scholars and larger policy reform movements. 

   more » « less