More Like this

1. Neural Machine Translation of Text from Non-Native Speakers

   Anastasopoulos, Antonios ; Lui, Alison ; Nguyen, Toan Q. ; Chiang, David ( June 2019 , Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics)

   Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data, especially when the system is trained only on clean data. In this paper, we show that augmenting training data with sentences containing artificially-introduced grammatical errors can make the system more robust to such errors. In combination with an automatic grammar error correction system, we can recover 1.0 BLEU out of 2.4 BLEU lost due to grammatical errors. We also present a set of Spanish translations of the JFLEG grammar error correction corpus, which allows for testing NMT robustness to real grammatical errors. 

   more » « less
2. BLiMP: The Benchmark of Linguistic Minimal Pairs for English

   https://doi.org/10.1162/tacl_a_00321  

   Warstadt, Alex ; Parrish, Alicia ; Liu, Haokun ; Mohananey, Anhad ; Peng, Wei ; Wang, Sheng-Fu ; Bowman, Samuel R. ( December 2020 , Transactions of the Association for Computational Linguistics)

   null (Ed.)

   We introduce The Benchmark of Linguistic Minimal Pairs (BLiMP), 1 a challenge set for evaluating the linguistic knowledge of language models (LMs) on major grammatical phenomena in English. BLiMP consists of 67 individual datasets, each containing 1,000 minimal pairs—that is, pairs of minimally different sentences that contrast in grammatical acceptability and isolate specific phenomenon in syntax, morphology, or semantics. We generate the data according to linguist-crafted grammar templates, and human aggregate agreement with the labels is 96.4%. We evaluate n-gram, LSTM, and Transformer (GPT-2 and Transformer-XL) LMs by observing whether they assign a higher probability to the acceptable sentence in each minimal pair. We find that state-of-the-art models identify morphological contrasts related to agreement reliably, but they struggle with some subtle semantic and syntactic phenomena, such as negative polarity items and extraction islands. 

   more » « less
3. Can Synthetic Translations Improve Bitext Quality?

   https://doi.org/10.18653/v1/2022.acl-long.326  

   Briakou, Eleftheria ; Carpuat, Marine ( May 2022 , Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers))

   Synthetic translations have been used for a wide range of NLP tasks primarily as a means of data augmentation. This work explores, instead, how synthetic translations can be used to revise potentially imperfect reference translations in mined bitext. We find that synthetic samples can improve bitext quality without any additional bilingual supervision when they replace the originals based on a semantic equivalence classifier that helps mitigate NMT noise. The improved quality of the revised bitext is confirmed intrinsically via human evaluation and extrinsically through bilingual induction and MT tasks. 

   more » « less
4. Does grammatical number support the acquisition of number words?

   Shusterman, A. ; Seevak, A. ; Mišmaš, P. ; Marušič, F. L. ; Žaucer, R. ( January 2023 , Proceedings of the 2022 Boston University Conference on Language Development)

   Cascadilla Press (Ed.)

   The morphosyntactic information in grammatical number marking may be a useful cue for children in the process of acquiring number words. A language with dual marking, like Slovenian, may help children to bootstrap the meaning of the word “two” by drawing their attention to sets of two as a referent of language. If the dual marker indeed facilitates number learning, then we hypothesized that “two” should be acquired earlier in populations exposed to the dual marker; the dual should be learned before “two”; and knowledge of the dual form should be correlated with knowledge of “two”. We tested these hypotheses by having Slovenian and English-speaking children complete the Give-a-Number and Give-Morphology tasks. We analyzed the Give-Morphology in a new way, using stricter criteria to determine that children “know” the morphological markers than simple percent correct. In this sample, Slovenian children exposed to the dual marker did not show evidence of knowing “two” (i.e., being 2-knowers) at very young ages or earlier than English-speaking children. Knowledge of the dual marker did not precede nor correlate with the acquisition of “two”; indeed, the dual form was only acquired after the singular and plural. These analyses were conducted using an open data set with more Slovenian 2-knowers, yielding similar results. These findings present challenges for the view that grammatical number plays a role in number acquisition. This theory requires articulation about how a dual-marked language can facilitate number acquisition if children do not notice or learn the dual form. The information in grammatical number marking may be a useful cue for children in the process of acquiring number words. A language with dual marking, like Slovenian, may help children to bootstrap the meaning of the word “two” by drawing their attention to sets of two as a referent of language. If the dual marker indeed facilitates number learning, we hypothesized that “two” should be acquired earlier in populations exposed to the dual marker; the dual should be learned before “two”; and knowledge of the dual form should be correlated with knowledge of “two”. We tested these hypotheses by having Slovenian and English-speaking children complete the Give-a-Number and Give-Morphology tasks. We analyzed the Give-Morphology in a new way, using stricter criteria to determine that children “know” the morphological markers than simple percent correct. In this sample, Slovenian children exposed to the dual marker did not show evidence of knowing “two” (i.e., being 2-knowers) at very young ages or earlier than English-speaking children. Knowledge of the dual marker did not precede nor correlate with the acquisition of “two”. Indeed, the dual form was acquired only after the singular and plural. Parallel analyses were also conducted using an open data set with more Slovenian 2-knowers, yielding similar results. These findings present challenges for the claim that grammatical number plays a role in number acquisition. Specifically, this theory requires better articulation about how a dual-marked language can facilitate number acquisition if children do not notice or learn the dual form. 

   more » « less
5. Correcting motion induced fluorescence artifacts in two-channel neural imaging

   https://doi.org/10.1371/journal.pcbi.1010421  

   Creamer, Matthew S. ; Chen, Kevin S. ; Leifer, Andrew M. ; Pillow, Jonathan W. ( September 2022 , PLOS Computational Biology)

   Louis, Matthieu (Ed.)

   Imaging neural activity in a behaving animal presents unique challenges in part because motion from an animal’s movement creates artifacts in fluorescence intensity time-series that are difficult to distinguish from neural signals of interest. One approach to mitigating these artifacts is to image two channels simultaneously: one that captures an activity-dependent fluorophore, such as GCaMP, and another that captures an activity-independent fluorophore such as RFP. Because the activity-independent channel contains the same motion artifacts as the activity-dependent channel, but no neural signals, the two together can be used to identify and remove the artifacts. However, existing approaches for this correction, such as taking the ratio of the two channels, do not account for channel-independent noise in the measured fluorescence. Here, we present Two-channel Motion Artifact Correction (TMAC), a method which seeks to remove artifacts by specifying a generative model of the two channel fluorescence that incorporates motion artifact, neural activity, and noise. We use Bayesian inference to infer latent neural activity under this model, thus reducing the motion artifact present in the measured fluorescence traces. We further present a novel method for evaluating ground-truth performance of motion correction algorithms by comparing the decodability of behavior from two types of neural recordings; a recording that had both an activity-dependent fluorophore and an activity-independent fluorophore (GCaMP and RFP) and a recording where both fluorophores were activity-independent (GFP and RFP). A successful motion correction method should decode behavior from the first type of recording, but not the second. We use this metric to systematically compare five models for removing motion artifacts from fluorescent time traces. We decode locomotion from a GCaMP expressing animal 20x more accurately on average than from control when using TMAC inferred activity and outperforms all other methods of motion correction tested, the best of which were ~8x more accurate than control. 

   more » « less