More Like this

1. Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition

   https://doi.org/10.18653/v1/2020.acl-main.768  

   Jeretic, Paloma; Warstadt, Alex; Bhooshan, Suvrat; Williams, Adina (January 2020, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics)

   null (Ed.)

   Natural language inference (NLI) is an increasingly important task for natural language understanding, which requires one to infer whether a sentence entails another. However, the ability of NLI models to make pragmatic inferences remains understudied. We create an IMPlicature and PRESupposition diagnostic dataset (IMPPRES), consisting of 32K semi-automatically generated sentence pairs illustrating well-studied pragmatic inference types. We use IMPPRES to evaluate whether BERT, InferSent, and BOW NLI models trained on MultiNLI (Williams et al., 2018) learn to make pragmatic inferences. Although MultiNLI appears to contain very few pairs illustrating these inference types, we find that BERT learns to draw pragmatic inferences. It reliably treats scalar implicatures triggered by “some” as entailments. For some presupposition triggers like “only”, BERT reliably recognizes the presupposition as an entailment, even when the trigger is embedded under an entailment canceling operator like negation. BOW and InferSent show weaker evidence of pragmatic reasoning. We conclude that NLI training encourages models to learn some, but not all, pragmatic inferences. 

   more » « less
2. Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering

   Lan, Wuwei; Xu, Wei. (August 2018, Proceedings of the 27th International Conference on Computational Linguistics)

   In this paper, we analyze several neural network designs (and their variations) for sentence pair modeling and compare their performance extensively across eight datasets, including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks. Although most of these models have claimed state-of-the-art performance, the original papers often reported on only one or two selected datasets. We provide a systematic study and show that (i) encoding contextual information by LSTM and inter-sentence interactions are critical, (ii) Tree-LSTM does not help as much as previously claimed but surprisingly improves performance on Twitter datasets, (iii) the Enhanced Sequential Inference Model is the best so far for larger datasets, while the Pairwise Word Interaction Model achieves the best performance when less data is available. We release our implementations as an open-source toolkit. 

   more » « less
3. Geometry of Radial Basis Neural Networks for Safety Biased Approximation of Unsafe Regions

   https://doi.org/10.23919/ACC55779.2023.10156278  

   Abuaish, Ahmad; Srinivasan, Mohit; Vela, Patricio A. (May 2023, American Control Conference)

   Barrier function-based inequality constraints are a means to enforce safety specifications for control systems. When used in conjunction with a convex optimization program, they provide a computationally efficient method to enforce safety for the general class of control-affine systems. One of the main assumptions when taking this approach is the a priori knowledge of the barrier function itself, i.e., knowledge of the safe set. In the context of navigation through unknown environments where the locally safe set evolves with time, such knowledge does not exist. This manuscript focuses on the synthesis of a zeroing barrier function characterizing the safe set based on safe and unsafe sample measurements, e.g., from perception data in navigation applications. Prior work formulated a supervised machine learning algorithm whose solution guaranteed the construction of a zeroing barrier function with specific level-set properties. However, it did not explore the geometry of the neural network design used for the synthesis process. This manuscript describes the specific geometry of the neural network used for zeroing barrier function synthesis, and shows how the network provides the necessary representation for splitting the state space into safe and unsafe regions. 

   more » « less
4. Automatic Dialect Density Estimation for African American English

   https://doi.org/10.21437/Interspeech.2022-796  

   Johnson, Alexander; Everson, Kevin; Ravi, Vijay; Gladney, Anissa; Ostendorf, Mari; Alwan, Abeer (September 2022, INTERSPEECH 2023)

   ISCA (Ed.)

   In this paper, we explore automatic prediction of dialect density of the African American English (AAE) dialect, where dialect density is defined as the percentage of words in an utterance that contain characteristics of the non-standard dialect. We investigate several acoustic and language modeling features, including the commonly used X-vector representation and ComParE feature set, in addition to information extracted from ASR transcripts of the audio files and prosodic information. To address issues of limited labeled data, we use a weakly supervised model to project prosodic and X-vector features into low-dimensional task-relevant representations. An XGBoost model is then used to predict the speaker's dialect density from these features and show which are most significant during inference. We evaluate the utility of these features both alone and in combination for the given task. This work, which does not rely on hand-labeled transcripts, is performed on audio segments from the CORAAL database. We show a significant correlation between our predicted and ground truth dialect density measures for AAE speech in this database and propose this work as a tool for explaining and mitigating bias in speech technology. 

   more » « less
5. Toward an improved understanding of causation in the ecological sciences

   https://doi.org/10.1002/fee.2530  

   Addicott, Ethan_T; Fenichel, Eli_P; Bradford, Mark_A; Pinsky, Malin_L; Wood, Stephen_A (June 2022, Frontiers in Ecology and the Environment)

   Abstract Society increasingly demands accurate predictions of complex ecosystem processes under novel conditions to address environmental challenges. However, obtaining the process‐level knowledge required to do so does not necessarily align with the burgeoning use in ecology of correlative model selection criteria, such as Akaike information criterion. These criteria select models based on their ability to reproduce outcomes, not on their ability to accurately represent causal effects. Causal understanding does not require matching outcomes, but rather involves identifying model forms and parameter values that accurately describe processes. We contend that researchers can arrive at incorrect conclusions about cause‐and‐effect relationships by relying on information criteria. We illustrate via a specific example that inference extending beyond prediction into causality can be seriously misled by information‐theoretic evidence. Finally, we identify a solution space to bridge the gap between the correlative inference provided by model selection criteria and a process‐based understanding of ecological systems. 

   more » « less