More Like this

1. Neural Machine Translation of Text from Non-Native Speakers

   Anastasopoulos, Antonios; Lui, Alison; Nguyen, Toan Q.; Chiang, David (June 2019, Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics)

   Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data, especially when the system is trained only on clean data. In this paper, we show that augmenting training data with sentences containing artificially-introduced grammatical errors can make the system more robust to such errors. In combination with an automatic grammar error correction system, we can recover 1.0 BLEU out of 2.4 BLEU lost due to grammatical errors. We also present a set of Spanish translations of the JFLEG grammar error correction corpus, which allows for testing NMT robustness to real grammatical errors. 

   more » « less
2. BLiMP: The Benchmark of Linguistic Minimal Pairs for English

   https://doi.org/10.1162/tacl_a_00321  

   Warstadt, Alex; Parrish, Alicia; Liu, Haokun; Mohananey, Anhad; Peng, Wei; Wang, Sheng-Fu; Bowman, Samuel R. (December 2020, Transactions of the Association for Computational Linguistics)

   null (Ed.)

   We introduce The Benchmark of Linguistic Minimal Pairs (BLiMP), 1 a challenge set for evaluating the linguistic knowledge of language models (LMs) on major grammatical phenomena in English. BLiMP consists of 67 individual datasets, each containing 1,000 minimal pairs—that is, pairs of minimally different sentences that contrast in grammatical acceptability and isolate specific phenomenon in syntax, morphology, or semantics. We generate the data according to linguist-crafted grammar templates, and human aggregate agreement with the labels is 96.4%. We evaluate n-gram, LSTM, and Transformer (GPT-2 and Transformer-XL) LMs by observing whether they assign a higher probability to the acceptable sentence in each minimal pair. We find that state-of-the-art models identify morphological contrasts related to agreement reliably, but they struggle with some subtle semantic and syntactic phenomena, such as negative polarity items and extraction islands. 

   more » « less
3. Augmented Spanish-Persian Neural Machine Translation [Augmented Spanish-Persian Neural Machine Translation]

   https://doi.org/10.5220/0010369804820488  

   Ahmadnia, Benyamin; Aranovich, Raul (January 2021, Proceedings of the 13th International Conference on Agents and Artificial Intelligence (ICAART 2021))

   null (Ed.)

   Neural Machine Translation (NMT) performs training of a neural network employing an encoder-decoder architecture. However, the quality of the neural-based translations predominantly depends on the availability of a large amount of bilingual training dataset. In this paper, we explore the performance of translations predicted by attention-based NMT systems for Spanish to Persian low-resource language pairs. We analyze the errors of NMT systems that occur in the Persian language and provide an in-depth comparison of the performance of the system based on variations in sentence length and size of the training dataset. We evaluate our translation results using BLEU and human evaluation measures based on the adequacy, fluency, and overall rating. 

   more » « less
4. Can Synthetic Translations Improve Bitext Quality?

   https://doi.org/10.18653/v1/2022.acl-long.326  

   Briakou, Eleftheria; Carpuat, Marine (May 2022, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers))

   Synthetic translations have been used for a wide range of NLP tasks primarily as a means of data augmentation. This work explores, instead, how synthetic translations can be used to revise potentially imperfect reference translations in mined bitext. We find that synthetic samples can improve bitext quality without any additional bilingual supervision when they replace the originals based on a semantic equivalence classifier that helps mitigate NMT noise. The improved quality of the revised bitext is confirmed intrinsically via human evaluation and extrinsically through bilingual induction and MT tasks. 

   more » « less
5. From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translation

   Marashian, Ali; Rice, Enora; Gessler, Luke; Palmer, Alexis; von_der_Wense, Katharina (January 2025, Association for Computational Linguistics)

   Many of the world’s languages have insufficient data to train high-performing general neural machine translation (NMT) models, let alone domain-specific models, and often the only available parallel data are small amounts of religious texts. Hence, domain adaptation (DA) is a crucial issue faced by contemporary NMT and has, so far, been underexplored for low-resource languages. In this paper, we evaluate a set of methods from both low-resource NMT and DA in a realistic setting, in which we aim to translate between a high-resource and a low-resource language with access to only: a) parallel Bible data, b) a bilingual dictionary, and c) a monolingual target-domain corpus in the high-resource language. Our results show that the effectiveness of the tested methods varies, with the simplest one, DALI, being most effective. We follow up with a small human evaluation of DALI, which shows that there is still a need for more careful investigation of how to accomplish DA for low-resource NMT. 

   more » « less