More Like this

1. Pareto Policy Adaptation

   Panagiotis Kyriakis, Jyotirmoy Deshmukh ( May 2022 , International Conference on Learning Representations)

   We present a policy gradient method for Multi-Objective Reinforcement Learning under unknown, linear preferences. By enforcing Pareto stationarity, a first-order condition for Pareto optimality, we are able to design a simple policy gradient algorithm that approximates the Pareto front and infers the unknown preferences. Our method relies on a projected gradient descent solver that identifies common ascent directions for all objectives. Leveraging the solution of that solver, we introduce Pareto Policy Adaptation (PPA), a loss function that adapts the policy to be optimal with respect to any distribution over preferences. PPA uses implicit differentiation to back-propagate the loss gradient bypassing the operations of the projected gradient descent solver. Our approach is straightforward, easy to implement and can be used with all existing policy gradient and actor-critic methods. We evaluate our method in a series of reinforcement learning tasks 

   more » « less
2. Pareto Policy Adaptation

   Panagiotis Kyriakis, Jyotirmoy Deshmukh ( June 2022 , International Conference on Learning Representations)

   We present a policy gradient method for Multi-Objective Reinforcement Learning under unknown, linear preferences. By enforcing Pareto stationarity, a first-order condition for Pareto optimality, we are able to design a simple policy gradient al- gorithm that approximates the Pareto front and infers the unknown preferences. Our method relies on a projected gradient descent solver that identifies common ascent directions for all objectives. Leveraging the solution of that solver, we in- troduce Pareto Policy Adaptation (PPA), a loss function that adapts the policy to be optimal with respect to any distribution over preferences. PPA uses implicit differentiation to back-propagate the loss gradient bypassing the operations of the projected gradient descent solver. Our approach is straightforward, easy to imple- ment and can be used with all existing policy gradient and actor-critic methods. We evaluate our method in a series of reinforcement learning tasks. 

   more » « less
3. Higher-Level Meta-Adaptation Mitigates Visual Distortions Produced by Lower-Level Adaptation

   https://doi.org/10.1177/0956797620907090  

   Liu, Xinyu ; Engel, Stephen_A ( April 2020 , Psychological Science)

   The visual system adapts to the environment, changing neural responses to aid efficiency and improve perception. However, these changes sometimes lead to negative consequences: If neurons at later processing stages fail to account for adaptation at earlier stages, perceptual errors result, including common visual illusions. These negative effects of adaptation have been termed the coding catastrophe. How does the visual system resolve them? We hypothesized that higher-level adaptation can correct errors arising from the coding catastrophe by changing what appears normal, a common form of adaptation across domains. Observers ( N = 15) viewed flickering checkerboards that caused a normal face to appear distorted. We tested whether the visual system can adapt to this adaptation-distorted face through repeated viewing. Results from two experiments show that such meta-adaptation does occur and that it makes the distorted face gradually appear more normal. Meta-adaptation may be a general strategy to correct negative consequences of low-level adaptation.

    

   more » « less
4. The treacheries of adaptation

   https://doi.org/10.1126/science.aaz5189  

   Miller, Craig R. ( October 2019 , Science)
5. Distributionally Robust Domain Adaptation

   https://doi.org/10.1109/MLSP55844.2023.10286010  

   Awad, Akram S. ; Atia, George K. ( September 2023 , IEEE 33rd International Workshop on Machine Learning for Signal Processing (MLSP))