More Like this

1. COMET: Commonsense Transformers for Knowledge Graph Construction

   Bosselut, Antoine; Rashkin, Hannah; Sap, Maarten; Malaviya, Chaitanya; Celikyilmaz, Asli; Choi, Yejin (August 2019, Association for Computational Linguistics (ACL))

   We present the first comprehensive study on automatic knowledge base construction for two prevalent commonsense knowledge graphs: ATOMIC (Sap et al., 2019) and ConceptNet (Speer et al., 2017). Contrary to many conventional KBs that store knowledge with canonical templates, commonsense KBs only store loosely structured open-text descriptions of knowledge. We posit that an important step toward automatic commonsense completion is the development of generative models of commonsense knowledge, and propose COMmonsEnse Transformers (COMET) that learn to generate rich and diverse commonsense descriptions in natural language. Despite the challenges of commonsense modeling, our investigation reveals promising results when implicit knowledge from deep pre-trained language models is transferred to generate explicit knowledge in commonsense knowledge graphs. Empirical results demonstrate that COMET is able to generate novel knowledge that humans rate as high quality, with up to 77.5% (ATOMIC) and 91.7% (ConceptNet) precision at top 1, which approaches human performance for these resources. Our findings suggest that using generative commonsense models for automatic commonsense KB completion could soon be a plausible alternative to extractive methods. 

   more » « less
2. Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts

   https://doi.org/10.18653/v1/2022.findings-acl.149  

   Yu, Wenhao; Zhu, Chenguang; Qin, Lianhui; Zhang, Zhihan; Zhao, Tong; Jiang, Meng (January 2022, Findings of the Association for Computational Linguistics: ACL 2022)

   Generative commonsense reasoning (GCR) in natural language is to reason about the commonsense while generating coherent text. Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks. Nevertheless, these approaches have seldom investigated diversity in the GCR tasks, which aims to generate alternative explanations for a real-world situation or predict all possible outcomes. Diversifying GCR is challenging as it expects to generate multiple outputs that are not only semantically different but also grounded in commonsense knowledge. In this paper, we propose MoKGE, a novel method that diversifies the generative reasoning by a mixture of expert (MoE) strategy on commonsense knowledge graphs (KG). A set of knowledge experts seek diverse reasoning on KG to encourage various generation outputs. Empirical experiments demonstrated that MoKGE can significantly improve the diversity while achieving on par performance on accuracy on two GCR benchmarks, based on both automatic and human evaluations. 

   more » « less
3. Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering

   Bosselut, Antoine; Le Bras, Ronan; Choi, Yejin (November 2020, The Thirty-Fifth AAAI Conference on Artificial Intelligence)

   Understanding narratives requires reasoning about implicit world knowledge related to the causes, effects, and states of situations described in text. At the core of this challenge is how to access contextually relevant knowledge on demand and reason over it. In this paper, we present initial studies toward zero-shot commonsense question answering by formulating the task as inference over dynamically generated commonsense knowledge graphs. In contrast to previous studies for knowledge integration that rely on retrieval of existing knowledge from static knowledge graphs, our study requires commonsense knowledge integration where contextually relevant knowledge is often not present in existing knowledge bases. Therefore, we present a novel approach that generates contextually-relevant symbolic knowledge structures on demand using generative neural commonsense knowledge models. Empirical results on two datasets demonstrate the efficacy of our neuro-symbolic approach for dynamically constructing knowledge graphs for reasoning. Our approach achieves significant performance boosts over pretrained language models and vanilla knowledge models, all while providing interpretable reasoning paths for its predictions. 

   more » « less
4. Scoring Integrated Meaningful Play in Story-Driven Games with Q-Learning

   Bettis, Alan; Newberry, Danielle; Munoz-Avila, Hector (January 2020, CEUR workshop proceedings)

   Osborn, Joseph C. (Ed.)

   Integrated meaningful play is the idea that player’s choices should have a long-term effect on the game. In this paper we present I-score (for integrated), a scoring function for scoring integrative game play as a function of the game’s storylines. The I-scores are in the range [0,1]. In games with I-scores close to one, player’s early choices determine the game’s ending; choices made later in the game do not change the final ending of the game. In contrast, games with I-scores close to zero, players’s choices can change the ending until the very end. Games with scores closer to 0.5 provide a more balanced player choice whereby the game’s ending still can be changed despite early decisions, but not so much that the ending could be changed at any point. 

   more » « less
5. ISD-QA: Iterative Distillation of Commonsense Knowledge from General Language Models for Unsupervised Question Answering

   Ramamurthy, Priyadharsini; Aakur, Sathyanarayanan (October 2022, International Conference on Pattern Recognition)

   Commonsense question answering has primarily been tackled through supervised transfer learning, where a language model pre-trained on large amounts of data is used as the starting point. While successful, the approach requires large amounts of labeled question-answer pairs, with increasingly larger amounts of data required as the complexity of scenarios or tasks such as commonsense QA increases. In this paper, we hypothesize that large-scale pre-training of language models encodes the necessary commonsense knowledge to answer common questions in context without labeled data. We propose a novel framework called Iterative Self Distillation for QA (ISD-QA), which extracts the “dark knowledge” encoded during largescale pre-training of language models to provide supervision for commonsense question answering. We show that the approach can be used to train common neural QA models for commonsense question answering by distilling knowledge from language models in an unsupervised manner. With no bells and whistles, we achieve an average of 68% of the performance of fully supervised QA models while requiring no labeled training data. Extensive experiments on three public benchmarks (OpenBookQA, HellaSWAG, and CommonsenseQA) show the effectiveness of the proposed approach. 

   more » « less