More Like this

1. Two Routes to Scalable Credit Assignment without Weight Symmetry, International Conference on Machine

   Kunin, D. ; Nayebi, A ; Javier, S. ; Ganguli, S ; Bloom, J. ; Yamins, D. ( April 2020 , Proceedings of Machine Learning Research)

   null (Ed.)

   The neural plausibility of backpropagation has long been disputed, primarily for its use of non-local weight transport — the biologically dubious requirement that one neuron instantaneously measure the synaptic weights of another. Until recently, attempts to create local learning rules that avoid weight transport have typically failed in the large-scale learning scenarios where backpropagation shines, e.g. ImageNet categorization with deep convolutional networks. Here, we investigate a recently proposed local learning rule that yields competitive performance with backpropagation and find that it is highly sensitive to metaparameter choices, requiring laborious tuning that does not transfer across network architecture. Our analysis indicates the underlying mathematical reason for this instability, allowing us to identify a more robust local learning rule that better transfers without metaparameter tuning. Nonetheless, we find a performance and stability gap between this local rule and backpropagation that widens with increasing model depth. We then investigate several non-local learning rules that relax the need for instantaneous weight transport into a more biologically-plausible "weight estimation" process, showing that these rules match state-of-the-art performance on deep networks and operate effectively in the presence of noisy updates. Taken together, our results suggest two routes towards the discovery of neural implementations for credit assignment without weight symmetry: further improvement of local rules so that they perform consistently across architectures and the identification of biological implementations for non-local learning mechanisms. 

   more » « less
2. Improving policy optimization with generalist-specialist learning. In International Conference on Machine Learning

   Jia, Z. ; Li, X. ; Ling, Z. ; Liu, S. ; Wu, Y. ; and Su, H. ( June 2022 , Proceedings of Machine Learning Research)
3. 2021, April. Understanding health and behavioral trends of successful students through machine learning models. In International Conference on Human Interaction and Emerging Technologies (pp. 516-525). Springer, Cham.

   https://doi.org/10.1007/978-3-030-74009  

   Kim, A. ; Nikseresht, F. ; Dutcher, J.M. ; Tumminia, M. ; Villalba, D. ; Cohen, S. ; Creswell, K. ; Creswell, D. ; Dey, A.K. ; Mankoff, J. ; et al ( April 2021 , International Conference on Human Interaction and Emerging Technologies)

   This study analyzes patterns of physical, mental, lifestyle, and personality factors in college students in different periods over the course of a semester and models their relationships with students’ academic performance. The data analyzed was collected through smartphones and Fitbit. The use of machine learning models derived from the gathered data was employed to observe the extent of students’ behavior associated with their GPA, lifestyle, physical health, mental health, and personality attributes. A mutual agreement method was used in which rather than looking at the accuracy of results, the model parameters and weights of features were used to find common behavioral trends. From the results of the model creation, it was determined that the most significant indicator of academic success defined as a higher GPA, was the places a student spent their time. Lifestyle and personality factors were deemed more significant than mental and physical factors. This study will provide insight into the impact of different factors and the timing of those factors on students’ academic performance . 

   more » « less
4. International Conference on Machine Learning

   Li, Wenjie ; Wang, Chi-hua ; Cheng, Guang ; Song, Qifan ( January 2023 , Transactions on machine learning research)
5. International Conference on Machine Learning and Applications

   Cheng, Long ; Wilson, Christin ; Liao, Song ; Young, Jeffrey ; Dong, Daniel ; Hu, Hongxin ( January 2020 , Conference on Computer and Communications Security)