More Like this

1. Computably Continuous Reinforcement-Learning Objectives are PAC-learnable

   Cambridge Yang; Michael Littman; Michael Carbin (January 2023, arXivorg)

   In reinforcement learning, the classic objectives of maximizing discounted and finite-horizon cumulative rewards are PAC-learnable: There are algorithms that learn a near-optimal policy with high probability using a finite amount of samples and computation. In recent years, researchers have introduced objectives and corresponding reinforcement-learning algorithms beyond the classic cumulative rewards, such as objectives specified as linear temporal logic formulas. However, questions about the PAC-learnability of these new objectives have remained open. This work demonstrates the PAC-learnability of general reinforcement-learning objectives through sufficient conditions for PAC-learnability in two analysis settings. In particular, for the analysis that considers only sample complexity, we prove that if an objective given as an oracle is uniformly continuous, then it is PAC-learnable. Further, for the analysis that considers computational complexity, we prove that if an objective is computable, then it is PAC-learnable. In other words, if a procedure computes successive approximations of the objective's value, then the objective is PAC-learnable. We give three applications of our condition on objectives from the literature with previously unknown PAC-learnability and prove that these objectives are PAC-learnable. Overall, our result helps verify existing objectives' PAC-learnability. Also, as some studied objectives that are not uniformly continuous have been shown to be not PAC-learnable, our results could guide the design of new PAC-learnable objectives. 

   more » « less
2. An Equivalence Between Private Classification and Online Prediction

   Bun, Mark; Livni, Roi; Moran, Shay (January 2020, 61st Annual IEEE Symposium on Foundations of Computer Science)

   null (Ed.)

   We prove that every concept class with finite Littlestone dimension can be learned by an (approximate) differentially-private algorithm. This answers an open question of Alon et al. (STOC 2019) who proved the converse statement (this question was also asked by Neel et al. (FOCS 2019)). Together these two results yield an equivalence between online learnability and private PAC learnability. We introduce a new notion of algorithmic stability called “global stability” which is essential to our proof and may be of independent interest. We also discuss an application of our results to boosting the privacy and accuracy parameters of differentially-private learners. 

   more » « less
3. Online Learning and Disambiguations of Partial Concept Classes

   https://doi.org/10.4230/LIPIcs.ICALP.2023.42  

   Cheung, Tsun-Ming; Hatami, Hamed; Hatami, Pooya; Hosseini, Kaave (January 2023, 50th International Colloquium on Automata, Languages, and Programming (ICALP 2023))

   Etessami, Kousha; Feige, Uriel; Puppis, Gabriele (Ed.)

   In a recent article, Alon, Hanneke, Holzman, and Moran (FOCS '21) introduced a unifying framework to study the learnability of classes of partial concepts. One of the central questions studied in their work is whether the learnability of a partial concept class is always inherited from the learnability of some "extension" of it to a total concept class. They showed this is not the case for PAC learning but left the problem open for the stronger notion of online learnability. We resolve this problem by constructing a class of partial concepts that is online learnable, but no extension of it to a class of total concepts is online learnable (or even PAC learnable). 

   more » « less
4. MODEL THEORY AND MACHINE LEARNING

   https://doi.org/10.1017/bsl.2018.71  

   CHASE, HUNTER; FREITAG, JAMES (September 2019, The Bulletin of Symbolic Logic)

   Abstract About 25 years ago, it came to light that a single combinatorial property determines both an important dividing line in model theory (NIP) and machine learning (PAC-learnability). The following years saw a fruitful exchange of ideas between PAC-learning and the model theory of NIP structures. In this article, we point out a new and similar connection between model theory and machine learning, this time developing a correspondence between stability and learnability in various settings of online learning. In particular, this gives many new examples of mathematically interesting classes which are learnable in the online setting. 

   more » « less
5. Crowdsourced PAC Learning under Classification Noise

   Heinecke, Shelby; Reyzin, Lev (October 2019, Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing)

   Law, Edith; Vaughan, Jennifer W (Ed.)

   In this paper, we analyze PAC learnability from labels produced by crowdsourcing. In our setting, unlabeled examples are drawn from a distribution and labels are crowdsourced from workers who operate under classification noise, each with their own noise parameter. We develop an end-to-end crowdsourced PAC learning algorithm that takes unlabeled data points as input and outputs a trained classifier. Our threestep algorithm incorporates majority voting, pure-exploration bandits, and noisy-PAC learning. We prove several guarantees on the number of tasks labeled by workers for PAC learning in this setting and show that our algorithm improves upon the baseline by reducing the total number of tasks given to workers. We demonstrate the robustness of our algorithm by exploring its application to additional realistic crowdsourcing settings. 

   more » « less