More Like this

1. Strengthening Low-resource Neural Machine Translation through Joint Learning: The Case of Farsi-Spanish [Strengthening Low-resource Neural Machine Translation through Joint Learning: The Case of Farsi-Spanish]

   https://doi.org/10.5220/0010362604750481  

   Ahmadnia, Benyamin ; Aranovich, Raul ; Dorr, Bonnie ( January 2021 , Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: NLPinAI)

   null (Ed.)

   This paper describes a systematic study of an approach to Farsi-Spanish low-resource Neural Machine Translation (NMT) that leverages monolingual data for joint learning of forward and backward translation models. As is standard for NMT systems, the training process begins using two pre-trained translation models that are iteratively updated by decreasing translation costs. In each iteration, either translation model is used to translate monolingual texts from one language to another, to generate synthetic datasets for the other translation model. Two new translation models are then learned from bilingual data along with the synthetic texts. The key distinguishing feature between our approach and standard NMT is an iterative learning process that improves the performance of both translation models, simultaneously producing a higher-quality synthetic training dataset upon each iteration. Our empirical results demonstrate that this approach outperforms baselines. 

   more » « less
2. An Effective Optimization Method for Neural Machine Translation: The Case of English-Persian Bilingually Low-Resource Scenario

   Benyamin Ahmadnia, Raul Aranovich ( December 2020 , Proceedings of the 7th Workshop on Asian Translation)

   null (Ed.)

   In this paper, we propose a useful optimization method for low-resource Neural Machine Translation (NMT) by investigating the effectiveness of multiple neural network optimization algorithms. Our results confirm that applying the proposed optimization method on English-Persian translation can exceed translation quality compared to the English-Persian Statistical Machine Translation (SMT) paradigm. 

   more » « less
3. Neural Machine Translation of Text from Non-Native Speakers

   Anastasopoulos, Antonios ; Lui, Alison ; Nguyen, Toan Q. ; Chiang, David ( June 2019 , Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics)

   Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data, especially when the system is trained only on clean data. In this paper, we show that augmenting training data with sentences containing artificially-introduced grammatical errors can make the system more robust to such errors. In combination with an automatic grammar error correction system, we can recover 1.0 BLEU out of 2.4 BLEU lost due to grammatical errors. We also present a set of Spanish translations of the JFLEG grammar error correction corpus, which allows for testing NMT robustness to real grammatical errors. 

   more » « less
4. Amortized Noisy Channel Neural Machine Translation

   Pang, Richard Yuanzhe ; He, He ; Cho, Kyunghyun ( July 2022 , INLG 2022)

   Noisy channel models have been especially effective in neural machine translation (NMT). However, recent approaches like "beam search and rerank" (BSR) incur significant computation overhead during inference, making real-world application infeasible. We aim to study if it is possible to build an amortized noisy channel NMT model such that when we do greedy decoding during inference, the translation accuracy matches that of BSR in terms of reward (based on the source-to-target log probability and the target-to-source log probability) and quality (based on BLEU and BLEURT). We attempt three approaches to train the new model: knowledge distillation, one-step-deviation imitation learning, and Q learning. The first approach obtains the noisy channel signal from a pseudo-corpus, and the latter two approaches aim to optimize toward a noisy-channel MT reward directly. For all three approaches, the generated translations fail to achieve rewards comparable to BSR, but the translation quality approximated by BLEU and BLEURT is similar to the quality of BSR-produced translations. Additionally, all three approaches speed up inference by 1-2 orders of magnitude. 

   more » « less
5. Backpropagation-Based Decoding for Multimodal Machine Translation

   https://doi.org/10.3389/frai.2021.736722  

   Yang, Ziyan ; Pinto-Alva, Leticia ; Dernoncourt, Franck ; Ordonez, Vicente ( January 2022 , Frontiers in Artificial Intelligence)

   People are able to describe images using thousands of languages, but languages share only one visual world. The aim of this work is to use the learned intermediate visual representations from a deep convolutional neural network to transfer information across languages for which paired data is not available in any form. Our work proposes using backpropagation-based decoding coupled with transformer-based multilingual-multimodal language models in order to obtain translations between any languages used during training. We particularly show the capabilities of this approach in the translation of German-Japanese and Japanese-German sentence pairs, given a training data of images freely associated with text in English, German, and Japanese but for which no single image contains annotations in both Japanese and German. Moreover, we demonstrate that our approach is also generally useful in the multilingual image captioning task when sentences in a second language are available at test time. The results of our method also compare favorably in the Multi30k dataset against recently proposed methods that are also aiming to leverage images as an intermediate source of translations. 

   more » « less