Natural language inference (NLI) is an increasingly important task for natural language understanding, which requires one to infer whether a sentence entails another. However, the ability of NLI models to make pragmatic inferences remains understudied. We create an IMPlicature and PRESupposition diagnostic dataset (IMPPRES), consisting of 32K semi-automatically generated sentence pairs illustrating well-studied pragmatic inference types. We use IMPPRES to evaluate whether BERT, InferSent, and BOW NLI models trained on MultiNLI (Williams et al., 2018) learn to make pragmatic inferences. Although MultiNLI appears to contain very few pairs illustrating these inference types, we find that BERT learns to draw pragmatic inferences. It reliably treats scalar implicatures triggered by “some” as entailments. For some presupposition triggers like “only”, BERT reliably recognizes the presupposition as an entailment, even when the trigger is embedded under an entailment canceling operator like negation. BOW and InferSent show weaker evidence of pragmatic reasoning. We conclude that NLI training encourages models to learn some, but not all, pragmatic inferences.
Four-year-olds incorporate speaker knowledge into pragmatic inference
Human communication relies on the ability to take into account the speaker's mental state to infer the intended meaning of an utterance in context. For example, a sentence such as 'Some of the animals are safe to pet' can be interpreted as giving rise to the inference 'Some and not all animals are safe to pet' when uttered by an expert. The same inference, known as a scalar implicature, does not arise when the sentence is spoken by someone with partial knowledge. Adults have been shown to derive scalar implicatures in accordance with the speaker's knowledge state, but in young children this ability is debated. Here, we revisit this question using a simple visual world paradigm. We find that both 4- and 5-year-olds successfully incorporate speaker knowledge into the derivation of scalar inferences. However, this ability does not generalize immediately to non-linguistic communicative contexts. These findings have important implications for the development of pragmatic abilities.
- Award ID(s):
- Publication Date:
- NSF-PAR ID:
- Journal Name:
- Developmental science
- Sponsoring Org:
- National Science Foundation
More Like this
Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question AnsweringIn this paper, we analyze several neural network designs (and their variations) for sentence pair modeling and compare their performance extensively across eight datasets, including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks. Although most of these models have claimed state-of-the-art performance, the original papers often reported on only one or two selected datasets. We provide a systematic study and show that (i) encoding contextual information by LSTM and inter-sentence interactions are critical, (ii) Tree-LSTM does not help as much as previously claimed but surprisingly improves performance on Twitter datasets, (iii) the Enhanced Sequential Inference Model is the best so far for larger datasets, while the Pairwise Word Interaction Model achieves the best performance when less data is available. We release our implementations as an open-source toolkit.
Jbabdi, Saad (Ed.)Whether the brain operates at a critical “tipping” point is a long standing scientific question, with evidence from both cellular and systems-scale studies suggesting that the brain does sit in, or near, a critical regime. Neuroimaging studies of humans in altered states of consciousness have prompted the suggestion that maintenance of critical dynamics is necessary for the emergence of consciousness and complex cognition, and that reduced or disorganized consciousness may be associated with deviations from criticality. Unfortunately, many of the cellular-level studies reporting signs of criticality were performed in non-conscious systems (in vitro neuronal cultures) or unconscious animals (e.g. anaesthetized rats). Here we attempted to address this knowledge gap by exploring critical brain dynamics in invasive ECoG recordings from multiple sessions with a single macaque as the animal transitioned from consciousness to unconsciousness under different anaesthetics (ketamine and propofol). We use a previously-validated test of criticality: avalanche dynamics to assess the differences in brain dynamics between normal consciousness and both drug-states. Propofol and ketamine were selected due to their differential effects on consciousness (ketamine, but not propofol, is known to induce an unusual state known as “dissociative anaesthesia”). Our analyses indicate that propofol dramatically restricted the size and duration ofmore »
Modeling tree radial growth in a warming climate: where, when, and how much do potential evapotranspiration models matter?Abstract Process-based models of tree-ring width are used both for reconstructing past climates and for projecting changes in growth due to climate change. Since soil moisture observations are unavailable at appropriate spatial and temporal scales, these models generally rely on simple water budgets driven in part by temperature-based potential evapotranspiration (PET) estimates, but the choice of PET model could have large effects on simulated soil moisture, moisture stress, and radial growth. Here, I use four different PET models to drive the VS-Lite model and evaluate the extent to which they differ in both their ability to replicate observed growth variability and their simulated responses to projected 21st century warming. Across more than 1200 tree-ring width chronologies in the conterminous United States, there were no significant differences among the four PET models in their ability to replicate observed radial growth, but the models differed in their responses to 21st century warming. The temperature-driven empirical PET models (Thornthwaite and Hargreaves) simulated much larger warming-induced increases in PET and decreases in soil moisture than the more physically realistic PET models (Priestley–Taylor and Penman–Monteith). In cooler and more mesic regions with relatively minimal moisture constraints to growth, the models simulated similarly small reductions inmore »
Abstract Locomotion on the narrow and compliant supports of the arboreal environment is inherently precarious. Previous studies have identified a host of morphological and behavioral specializations in arboreal animals broadly thought to promote stability when on precarious substrates. Less well-studied is the role of the tail in maintaining balance. However, prior anatomical studies have found that arboreal taxa frequently have longer tails for their body size than their terrestrial counterparts, and prior laboratory studies of tail kinematics and the effects of tail reduction in focal taxa have broadly supported the hypothesis that the tail is functionally important for maintaining balance on narrow and mobile substrates. In this set of studies, we extend this work in two ways. First, we used a laboratory dataset on three-dimensional segmental kinematics and tail inertial properties in squirrel monkeys (Saimiri boliviensis) to investigate how tail angular momentum is modulated during steady-state locomotion on narrow supports. In the second study, we used a quantitative dataset on quadrupedal locomotion in wild platyrrhine monkeys to investigate how free-ranging arboreal animals adjust tail movements in response to substrate variation, focusing on kinematic measures validated in prior laboratory studies of tail mechanics (including the laboratory data presented). Our laboratory resultsmore »