More Like this

1. Strengthening Low-resource Neural Machine Translation through Joint Learning: The Case of Farsi-Spanish [Strengthening Low-resource Neural Machine Translation through Joint Learning: The Case of Farsi-Spanish]

   https://doi.org/10.5220/0010362604750481  

   Ahmadnia, Benyamin ; Aranovich, Raul ; Dorr, Bonnie ( January 2021 , Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: NLPinAI)

   null (Ed.)

   This paper describes a systematic study of an approach to Farsi-Spanish low-resource Neural Machine Translation (NMT) that leverages monolingual data for joint learning of forward and backward translation models. As is standard for NMT systems, the training process begins using two pre-trained translation models that are iteratively updated by decreasing translation costs. In each iteration, either translation model is used to translate monolingual texts from one language to another, to generate synthetic datasets for the other translation model. Two new translation models are then learned from bilingual data along with the synthetic texts. The key distinguishing feature between our approach and standard NMT is an iterative learning process that improves the performance of both translation models, simultaneously producing a higher-quality synthetic training dataset upon each iteration. Our empirical results demonstrate that this approach outperforms baselines. 

   more » « less
2. Syntax-driven Data Augmentation for Named Entity Recognition

   Sutiono, Arie ; Hahn-Powell, Gus ( October 2022 , Proceedings of the First Workshop on Pattern-based Approaches to NLP in the Age of Deep Learning)

   In low resource settings, data augmentation strategies are commonly leveraged to improve performance. Numerous approaches have attempted document-level augmentation (e.g., text classification), but few studies have explored token-level augmentation. Performed naively, data augmentation can produce semantically incongruent and ungrammatical examples. In this work, we compare simple masked language model replacement and an augmentation method using constituency tree mutations to improve the performance of named entity recognition in low-resource settings with the aim of preserving linguistic cohesion of the augmented sentences. 

   more » « less
3. Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank

   https://doi.org/10.18653/v1/2020.findings-emnlp.118  

   Chau, Ethan C. ; Lin, Lucy H. ; Smith, Noah A. ( January 2020 , : Findings of the Association for Computational Linguistics: EMNLP 2020)

   Pretrained multilingual contextual representations have shown great success, but due to the limits of their pretraining data, their benefits do not apply equally to all language varieties. This presents a challenge for language varieties unfamiliar to these models, whose labeled and unlabeled data is too limited to train a monolingual model effectively. We propose the use of additional language-specific pretraining and vocabulary augmentation to adapt multilingual models to low-resource settings. Using dependency parsing of four diverse low-resource language varieties as a case study, we show that these methods significantly improve performance over baselines, especially in the lowest-resource cases, and demonstrate the importance of the relationship between such models’ pretraining data and target language varieties. 

   more » « less
4. Elaborative Simplification as Implicit Questions Under Discussion

   https://doi.org/10.18653/v1/2023.emnlp-main.336  

   Wu, Yating ; Sheffield, William ; Mahowald, Kyle ; Li, Junyi Jessy ( December 2023 , Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing)

   Automated text simplification, a technique useful for making text more accessible to people such as children and emergent bilinguals, is often thought of as a monolingual translation task from complex sentences to simplified sentences using encoder-decoder models. This view fails to account for elaborative simplification, where new information is added into the simplified text. This paper proposes to view elaborative simplification through the lens of the Question Under Discussion (QUD) framework, providing a robust way to investigate what writers elaborate upon, how they elaborate, and how elaborations fit into the discourse context by viewing elaborations as explicit answers to implicit questions. We introduce ELABQUD, consisting of 1.3K elaborations accompanied with implicit QUDs, to study these phenomena. We show that explicitly modeling QUD (via question generation) not only provides essential understanding of elaborative simplification and how the elaborations connect with the rest of the discourse, but also substantially improves the quality of elaboration generation. 

   more » « less
5. TAMS: Translation-Assisted Morphological Segmentation

   Rice, Enora ; Marashian, Ali ; Gessler, Luke ; Palmer, Alexis ; von_der_Wense, Katharina ( August 2024 , Association for Computational Linguistics)

   Canonical morphological segmentation is the process of analyzing words into the standard (aka underlying) forms of their constituent morphemes.This is a core task in endangered language documentation, and NLP systems have the potential to dramatically speed up this process. In typical language documentation settings, training data for canonical morpheme segmentation is scarce, making it difficult to train high quality models. However, translation data is often much more abundant, and, in this work, we present a method that attempts to leverage translation data in the canonical segmentation task. We propose a character-level sequence-to-sequence model that incorporates representations of translations obtained from pretrained high-resource monolingual language models as an additional signal. Our model outperforms the baseline in a super-low resource setting but yields mixed results on training splits with more data. Additionally, we find that we can achieve strong performance even without needing difficult-to-obtain word level alignments. While further work is needed to make translations useful in higher-resource settings, our model shows promise in severely resource-constrained settings. 

   more » « less