The Physics Inventory of Quantitative Reasoning: Assessing Student Reasoning About Sign

Olsho, Alexis; White Brahmia, Suzanne; Smith, Trevor I.; Boudreaux, Andrew

Spatial reasoning over text is challenging as the models not only need to extract the direct spatial information from the text but also reason over those and infer implicit spatial relations. Recent studies highlight the struggles even large language models encounter when it comes to performing spatial reasoning over text. In this paper, we explore the potential benefits of disentangling the processes of information extraction and reasoning in models to address this challenge. To explore this, we design various models that disentangle extraction and reasoning(either symbolic or neural) and compare them with state-of-the-art(SOTA) baselines with no explicit design for these parts. Our experimental results consistently demonstrate the efficacy of disentangling, showcasing its ability to enhance models{'} generalizability within realistic data domains.

More Like this