More Like this

1. Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning

   https://doi.org/10.18653/v1/2023.findings-emnlp.286  

   Zhou, Jianing; Zeng, Ziheng; Gong, Hongyu; Bhat, Suma (January 2023, Findings of the Association for Computational Linguistics: EMNLP 2023)

   Non-compositional expressions, by virtue of their non-compositionality, are a classic ‘pain in the neck’ for NLP systems. Different from the general language modeling and generation tasks that are primarily compositional, generating non-compositional expressions is more challenging for current neural models, including large pre-trained language models. The main reasons are 1) their non-compositionality, and 2) the limited data resources. Therefore, to make the best use of available data for modeling non-compositionality, we propose a dynamic curriculum learning framework, which learns training examples from easy ones to harder ones thus optimizing the learning step by step but suffers from the forgetting problem. To alleviate the forgetting problem brought by the arrangement of training examples, we also apply a continual learning method into our curriculum learning framework. Our proposed method combined curriculum and continual learning, to gradually improve the model’s performance on the task of non-compositional expression generation. Experiments on idiomatic expression generation and metaphor generation affirm the effectiveness of our proposed curriculum learning framework and the application of continual learning. Our codes are available at https://github.com/zhjjn/CL2Gen.git. 

   more » « less
2. Intrinsic and extrinsic deep learning on manifolds

   https://doi.org/10.1214/24-EJS2227  

   Fang, Yihao; Ohn, Ilsang; Gupta, Vijay; Lin, Lizhen (January 2024, Electronic Journal of Statistics)

   We propose extrinsic and intrinsic deep neural network architectures as general frameworks for deep learning on manifolds. Specifically, extrinsic deep neural networks (eDNNs) preserve geometric features on manifolds by utilizing an equivariant embedding from the manifold to its image in the Euclidean space. Moreover, intrinsic deep neural networks (iDNNs) incorporate the underlying intrinsic geometry of manifolds via exponential and log maps with respect to a Riemannian structure. Consequently, we prove that the empirical risk of the empirical risk minimizers (ERM) of eDNNs and iDNNs converge in optimal rates. Overall, The eDNNs framework is simple and easy to compute, while the iDNNs framework is accurate and fast converging. To demonstrate the utilities of our framework, various simulation studies, and real data analyses are presented with eDNNs and iDNNs. 

   more » « less
3. International Conference on Machine Learning and Applications

   Cheng, Long; Wilson, Christin; Liao, Song; Young, Jeffrey; Dong, Daniel; Hu, Hongxin (January 2020, Conference on Computer and Communications Security)
4. On Learning Necessary and Sufficient Causal Graphs

   Cai, Hengrui; Wang, Yixin; Jordan, Michael; Song, Rui (December 2023, NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems)
5. Learning on manifolds without manifold learning

   https://doi.org/10.1016/j.neunet.2024.106759  

   Mhaskar, HN; O’Dowd, Ryan (October 2024, Neural Networks)