More Like this

1. Bandits with Stochastic Experts: Constant Regret, Empirical Experts and Episodes

   https://doi.org/10.1145/3680279  

   Sharma, Nihal; Sen, Rajat; Basu, Soumya; Shanmugam, Karthikeyan; Shakkottai, Sanjay (September 2024, ACM Transactions on Modeling and Performance Evaluation of Computing Systems)

   We study a variant of the contextual bandit problem where an agent can intervene through a set of stochastic expert policies. Given a fixed context, each expert samples actions from a fixed conditional distribution. The agent seeks to remain competitive with the “best” among the given set of experts. We propose the Divergence-based Upper Confidence Bound (D-UCB) algorithm that uses importance sampling to share information across experts and provide horizon-independent constant regret bounds that only scale linearly in the number of experts. We also provide the Empirical D-UCB (ED-UCB) algorithm that can function with only approximate knowledge of expert distributions. Further, we investigate the episodic setting where the agent interacts with an environment that changes over episodes. Each episode can have different context and reward distributions resulting in the best expert changing across episodes. We show that by bootstrapping from\(\mathcal {O}(N\log (NT^2\sqrt {E}))\)samples, ED-UCB guarantees a regret that scales as\(\mathcal {O}(E(N+1) + \frac{N\sqrt {E}}{T^2})\)forNexperts overEepisodes, each of lengthT. We finally empirically validate our findings through simulations. 

   more » « less
2. DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts

   https://doi.org/10.18653/v1/2021.acl-long.522  

   Liu, Alisa; Sap, Maarten; Lu, Ximing; Swayamdipta, Swabha; Bhagavatula, Chandra; Smith, Noah A.; Choi, Yejin (January 2021, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers))

   Despite recent advances in natural language generation, it remains challenging to control attributes of generated text. We propose DExperts: Decoding-time Experts, a decoding-time method for controlled text generation that combines a pretrained language model with “expert” LMs and/or “anti-expert” LMs in a product of experts. Intuitively, under the ensemble, tokens only get high probability if they are considered likely by the experts, and unlikely by the anti-experts. We apply DExperts to language detoxification and sentiment-controlled generation, where we outperform existing controllable generation methods on both automatic and human evaluations. Moreover, because DExperts operates only on the output of the pretrained LM, it is effective with (anti-)experts of smaller size, including when operating on GPT-3. Our work highlights the promise of tuning small LMs on text with (un)desirable attributes for efficient decoding-time steering. 

   more » « less
3. Interpretable Mixture of Experts

   Ismail, Aya; Arik, Sercan; Yoon, Jinsung; Taly, Ankur; Feizi, Soheil; Pfister, Tomas (January 2023, TMLR 2023)
4. Interpretable Mixture of Experts

   Ismail, Aya; Arik, Sercan; Yoon, Jinsung; Taly, Ankur; Feizi, Soheil; Pfister, Tomas (January 2023, TMLR 2023)
5. Bandits and Experts in Metric Spaces

   https://doi.org/10.1145/3299873  

   Kleinberg, Robert; Slivkins, Aleksandrs; Upfal, Eli (August 2019, Journal of the ACM)