More Like this

1. Learning a Policy for Opportunistic Active Learning

   Padmakumar, A.; Stone, P.; Mooney, R.J. (October 2018, Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP))

   Active learning identifies data points to label that are expected to be the most useful in improving a supervised model. Opportunistic active learning incorporates active learning into interactive tasks that constrain possible queries during interactions. Prior work has shown that opportunistic active learning can be used to improve grounding of natural language descriptions in an interactive object retrieval task. In this work, we use reinforcement learning for such an object retrieval task, to learn a policy that effectively trades off task completion with model improvement that would benefit future tasks. 

   more » « less
2. Transfer Learning in Deep Reinforcement Learning: A Survey

   https://doi.org/10.1109/TPAMI.2023.3292075  

   Zhu, Zhuangdi; Lin, Kaixiang; Jain, Anil K; Zhou, Jiayu (November 2023, IEEE Transactions on Pattern Analysis and Machine Intelligence)
3. Learning from a Learning User for Optimal Recommendations

   Yao, Fan; Li, Chuanhao; Nekipelov, Denis; Wang, Hongning; Xu, Haifeng (July 2022, Proceedings of the 39th International Conference on Machine Learning)

   Chaudhuri, Kamalika; Jegelka, Stefanie; Song, Le; Szepesvari, Csaba; Niu, Gang; Sabato, Sivan (Ed.)

   In real-world recommendation problems, especially those with a formidably large item space, users have to gradually learn to estimate the utility of any fresh recommendations from their experience about previously consumed items. This in turn affects their interaction dynamics with the system and can invalidate previous algorithms built on the omniscient user assumption. In this paper, we formalize a model to capture such ”learning users” and design an efficient system-side learning solution, coined Noise-Robust Active Ellipsoid Search (RAES), to confront the challenges brought by the non-stationary feedback from such a learning user. Interestingly, we prove that the regret of RAES deteriorates gracefully as the convergence rate of user learning becomes worse, until reaching linear regret when the user’s learning fails to converge. Experiments on synthetic datasets demonstrate the strength of RAES for such a contemporaneous system-user learning problem. Our study provides a novel perspective on modeling the feedback loop in recommendation problems. 

   more » « less
4. Desynchronous learning in a physics-driven learning network

   https://doi.org/10.1063/5.0084631  

   Wycoff, J. F.; Dillavou, S.; Stern, M.; Liu, A. J.; Durian, D. J. (April 2022, The Journal of Chemical Physics)

   In a neuron network, synapses update individually using local information, allowing for entirely decentralized learning. In contrast, elements in an artificial neural network are typically updated simultaneously using a central processor. Here, we investigate the feasibility and effect of desynchronous learning in a recently introduced decentralized, physics-driven learning network. We show that desynchronizing the learning process does not degrade the performance for a variety of tasks in an idealized simulation. In experiment, desynchronization actually improves the performance by allowing the system to better explore the discretized state space of solutions. We draw an analogy between desynchronization and mini-batching in stochastic gradient descent and show that they have similar effects on the learning process. Desynchronizing the learning process establishes physics-driven learning networks as truly fully distributed learning machines, promoting better performance and scalability in deployment. 

   more » « less
5. Predicting Learning Interactions in Social Learning Networks: A Deep Learning Enabled Approach

   https://doi.org/10.1109/TNET.2023.3237978  

   Sahay, Rajeev; Nicoll, Serena; Zhang, Minjun; Yang, Tsung-Yen; Joe-Wong, Carlee; Douglas, Kerrie A.; Brinton, Christopher G. (January 2023, IEEE/ACM Transactions on Networking)