More Like this

1. A spoken dialogue system for spatial question answering in a physical blocks world

   Platonov, Georgiy; Kane, Benjamin; Gindi, Aaron; Schubert, Lenhart K. (June 2020, Proceedings of SigDial 2020, ISBN 978-1-952148-02-6)

   A physical blocks world, despite its relative simplicity, requires (in fully interactive form) a rich set of functional capabilities, ranging from vision to natural language understanding. In this work we tackle spatial question answering in a holistic way, using a vision system, speech input and output mediated by an animated avatar, a dialogue system that robustly interprets spatial queries, and a constraint solver that derives answers based on 3-D spatial modeling. The contributions of this work include a semantic parser that maps spatial questions into logical forms consistent with a general approach to meaning representation, a dialogue manager based on a schema representation, and a constraint solver for spatial questions that provides answers in agreement with human perception. These and other components are integrated into a multi-modal human-computer interaction pipeline. 

   more » « less
2. Open-World Narrative Generation to Answer Players’ Questions

   https://doi.org/10.1609/aiide.v18i1.21981  

   Siler, Cory (October 2022, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment)

   Planning-based narrative generation is effective at producing stories with a logically-sound flow of events, but it can be limiting due to the rigidity of its constraints and the high burden on the domain author to define story-world objects, initial states, and author and character goals. Giving the system the freedom to add objects and events to the story-world history arbitrarily can improve variety and reduce authorial burden, but risks leading to stories that seem jarringly contrived to the audience. I propose to use question-answering as the antidote to contrivance in a highly-generative interactive narrative system: By modeling the player's beliefs about the story world, inferring the implicit questions the player may be asking through their interactions, and answering those questions in a way consistent with the player's prior knowledge, a system could focus on creating cohesion in the ways that matter most to the player while accepting a degree of contrivance in the details that the player is likely to overlook. 

   more » « less
3. Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections

   Ko, Wei-Jen; Dalton, Cutter; Simmons, Mark; Fisher, Eliza; Durrett, Greg; Li, Junyi Jessy (December 2022, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing)

   While there has been substantial progress in text comprehension through simple factoid question answering, more holistic comprehension of a discourse still presents a major challenge (Dunietz et al., 2020). Someone critically reflecting on a text as they read it will pose curiosity-driven, often open-ended questions, which reflect deep understanding of the content and require complex reasoning to answer (Ko et al., 2020; Westera et al., 2020). A key challenge in building and evaluating models for this type of discourse comprehension is the lack of annotated data, especially since collecting answers to such questions requires high cognitive load for annotators. This paper presents a novel paradigm that enables scalable data collection targeting the comprehension of news documents, viewing these questions through the lens of discourse. The resulting corpus, DCQA (Discourse Comprehension by Question Answering), captures both discourse and semantic links between sentences in the form of free-form, open-ended questions. On an evaluation set that we annotated on questions from Ko et al. (2020), we show that DCQA provides valuable supervision for answering open-ended questions. We additionally design pre-training methods utilizing existing question-answering resources, and use synthetic data to accommodate unanswerable questions. 

   more » « less
4. Modeling semantics and pragmatics of spatial prepositions via hierarchical common-sense primitives

   Platonov, GEorgiy; Kane, Benjamin; Schubert, Lenhart K. (October 2021, Proc. of the Reasoning and Interaction Conference (ReInAct 2021))

   Howes, Christine; Dobnik, Simon; Breitholtz, Ellen; Chatzikyriakidis, Stergios (Ed.)

   As AI reaches wider adoption, designing systems that are explainable and interpretable be- comes a critical necessity. In particular, when it comes to dialogue systems, their reasoning must be transparent and must comply with human intuitions in order for them to be inte- grated seamlessly into day-to-day collaborative human-machine activities. Here, we de- scribe our ongoing work on a (general purpose) dialogue system equipped with a spatial specialist with explanatory capabilities. We applied this system to a particular task of char- acterizing spatial configurations of blocks in a simple physical Blocks World (BW) domain using natural locative expressions, as well as generating justifications for the proposed spa- tial descriptions by indicating the factors that the system used to arrive at a particular conclu- sion. 

   more » « less
5. Annotating low-confidence questions improves classifier performance

   Hernandez, Stephanie; Artstein, Ron (September 2021, Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue - Poster Abstracts)

   This paper compares methods to select data for annotation in order to improve a classifier used in a question-answering dialogue system. With a classifier trained on 1,500 questions, adding 300 training questions on which the classifier is least confident results in consistently improved performance, whereas adding 300 arbitrarily selected training questions does not yield consistent improvement, and sometimes even degrades performance. The paper uses a new method for comparative evaluation of classifiers for dialogue, which scores each classifier based on the number of appropriate responses retrieved. 

   more » « less