An official website of the United States government
Patients seeking medical information often ask questions that embed incorrect assumptions or misconceptions. In such cases, safe medical communication requires not only answering the question, but identifying and correcting the underlying false belief. These interactions naturally unfold over multiple turns, a pattern now mirrored in interactions with LLMs. Yet current evaluation frameworks do not capture model behavior in these settings, where misconceptions can emerge, persist, or evolve over the course of a conversation. Whether LLMs can reliably correct such misconceptions over time remains largely unexamined. To study this, we introduce ThReadMed-QA, a multi-turn medical dialogue dataset of 2,437 patient–physician conversation threads comprising 8,204 question–answer pairs, derived from real patient interactions on r/AskDocs. This dataset enables systematic evaluation of whether models can detect and correct misconceptions under a multi-turn context. We evaluate five state-of-the-art LLMs using a rubric-based LLM-as-a-Judge framework that scores responses based on their ability to identify and correct misconceptions. Our experiments reveal a consistent pattern: even frontier models that can address misconceptions in a single interaction degrade substantially over subsequent turns. GPT-5 and Claude-Haiku correct these false presuppositions around 85% on initial questions but drop to approximately 50% within two follow-ups. Additionally, GPT-4o exhibits a sharper decline, falling from 65% to 21%, indicating a failure to sustain safe reasoning across dialogue. An oracle analysis replacing prior model outputs with physician responses shows that much of the degradation is driven by error propagation, while performance remains imperfect even under correct context. These findings reveal a critical reliability gap in LLMs. Even when models tend to correct misconceptions initially, their performance degrades substantially over subsequent turns, leading to inconsistent and potentially unsafe guidance in patient-facing settings and highlighting the need for evaluation frameworks that capture multi-turn behavior.
more »
« less
(QA)^2: Question Answering with Questionable Assumptions
Naturally-occurring information-seeking questions often contain questionable assumptions -- assumptions that are false or unverifiable. Questions containing questionable assumptions are challenging because they require a distinct answer strategy that deviates from typical answers to information-seeking questions. For instance, the question "When did Marie Curie discover Uranium?" cannot be answered as a typical when question without addressing the false assumption "Marie Curie discovered Uranium". In this work, we propose (QA)2 (Question Answering with Questionable Assumptions), an open-domain evaluation dataset consisting of naturally-occurring search engine queries that may or may not contain questionable assumptions. To be successful on (QA)2, systems must be able to detect questionable assumptions and also be able to produce adequate responses for both typical information-seeking questions and ones with questionable assumptions. We find that current models do struggle with handling questionable assumptions -- the best performing model achieves 59% human rater acceptability on abstractive QA with (QA)2 questions, leaving substantial headroom for progress.
more »
« less
- Award ID(s):
- 2046556
- PAR ID:
- 10441675
- Date Published:
- Journal Name:
- Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics
- Volume:
- 1
- Page Range / eLocation ID:
- 8466-8487
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
During the preschool years, children’s question-explanation exchanges with teachers serve as a powerful mechanism for their early STEM knowledge acquisition. Utilizing naturalistic longitudinal classroom data, we examined how such conversations in an inquiry-based preschool classroom change during an extended scientific inquiry unit. We were particularly interested in information-seeking questions (causal, e.g. “How will you construct a pathway?”; fact-based, e.g., “Where’s the marble?”). Videos (n = 18; 14 hours) were collected during a three-week inquiry unit on forces and motion and transcribed in CLAN-CHILDES software at the utterance level. Utterances were coded for delivery (question vs. statement) and content (e.g., fact-based, causal). Although teachers ask more questions than children, we found a significant increase in information-seeking questions during Weeks 2 and 3. We explored the content of information-seeking questions and found that the majority of these questions were asked by teachers, and focused on facts. However, the timing of fact-based and causal questions varied. Whereas more causal questions occurred in earlier weeks, more fact-based questions were asked towards the end of the inquiry. These findings provide insight into how children’s and teacher’s questions develop during an inquiry, informing our understanding of early science learning. Even in an inquiry-learning environment, teachers guide interactions, asking questions to support children’s learning. Children’s information-seeking questions increase during certain weeks, suggesting that providing opportunities to ask questions may allow children to be more active in constructing knowledge. Such findings are important for considering how science questions are naturally embedded in an inquiry-based learning classroom.more » « less
-
To build robust question answering systems, we need the ability to verify whether answers to questions are truly correct, not just “good enough” in the context of imperfect QA datasets. We explore the use of natural language inference (NLI) as a way to achieve this goal, as NLI inherently requires the premise (document context) to contain all necessary information to support the hypothesis (proposed answer to the question). We leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules, which can reformulate QA instances as premise-hypothesis pairs with very high reliability. Then, by combining standard NLI datasets with NLI examples automatically derived from QA training data, we can train NLI models to evaluate QA models’ proposed answers. We show that our approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. Careful manual analysis over the predictions of our NLI model shows that it can further identify cases where the QA model produces the right answer for the wrong reason, i.e., when the answer sentence cannot address all aspects of the question.more » « less
-
Understanding and characterizing how people interact in information-seeking conversations will be a crucial component in developing effective conversational search systems. In this paper, we introduce a new dataset designed for this purpose and use it to analyze information-seeking conversations by user intent distribution, co-occurrence, and flow patterns. The MSDialog dataset is a labeled conversation dataset of question answering (QA) interactions between information seekers and providers from an online forum on Microsoft products. The dataset contains more than 2,000 multi-turn QA dialogs with 10,000 utterances that are annotated with user intents on the utterance level. Annotations were done using crowdsourcing. With MSDialog, we find some highly recurring patterns in user intent during an information-seeking process. They could be useful for designing conversational search systems. We will make our dataset freely available to encourage exploration of information-seeking conversation models.more » « less
-
Clinical question answering (QA) aims to automatically answer questions from medical professionals based on clinical texts. Studies show that neural QA models trained on one corpus may not generalize well to new clinical texts from a different institute or a different patient group, where large-scale QA pairs are not readily available for model retraining. To address this challenge, we propose a simple yet effective framework, CliniQG4QA, which leverages question generation (QG) to synthesize QA pairs on new clinical contexts and boosts QA models without requiring manual annotations. In order to generate diverse types of questions that are essential for training QA models, we further introduce a seq2seq-based question phrase prediction (QPP) module that can be used together with most existing QG models to diversify the generation. Our comprehensive experiment results show that the QA corpus generated by our framework can improve QA models on the new contexts (up to 8% absolute gain in terms of Exact Match), and that the QPP module plays a crucial role in achieving the gain.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- The DOI is not currently available.
