skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Learning algebraic representation for systematic generalization in abstract reasoning
Is intelligence realized by connectionist or classicist? While connectionist approaches have achieved superhuman performance, there has been growing evidence that such task-specific superiority is particularly fragile in systematic generalization. This observation lies in the central debate between connectionist and classicist, wherein the latter continually advocates an algebraic treatment in cognitive architectures. In this work, we follow the classicist’s call and propose a hybrid approach to improve systematic generalization in reasoning. Specifically, we showcase a prototype with algebraic representation for the abstract spatial-temporal reasoning task of Raven’s Progressive Matrices (RPM) and present the ALgebra-Aware Neuro-Semi-Symbolic (ALANS) learner. The ALANS learner is motivated by abstract algebra and the representation theory. It consists of a neural visual perception frontend and an algebraic abstract reasoning backend: the frontend summarizes the visual information from object-based representation, while the backend transforms it into an algebraic structure and induces the hidden operator on the fly. The induced operator is later executed to predict the answer’s representation, and the choice most similar to the prediction is selected as the solution. Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization. We further show the generative nature of the learned algebraic representation; it can be decoded by isomorphism to generate an answer.  more » « less
Award ID(s):
2015577
PAR ID:
10351390
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
European Conference on Computer Vision (ECCV 2022)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Proceedings of the 15th International Conference of the Learning Sciences - ICLS 2021)
    de Vries, E.; Hod, Y.; Ahn, J. (Ed.)
    Research has shown that tape diagrams are beneficial for algebra learning. However, it is unclear whether certain visual features of tape diagrams have implications for learning. We investigated, with undergraduate students and math teachers, whether tape diagrams with different visual features (color, presence of outer lines, and position of the constant) differentially support reasoning about equations and whether people have preferences for certain visual features. Variations in visual features did not affect students’ or teachers’ reasoning accuracy; but each group displayed systematic preferences for most visual features considered. Future research should examine the effects of these visual features on performance while solving equations. 
    more » « less
  2. ; ; ; ; ; (, Proceedings of the 15th International Conference of the Learning Sciences - ICLS 2021)
    de Vries, E.; Hod, Y.; Ahn, J. (Ed.)
    Research has shown that tape diagrams are beneficial for algebra learning. However, it is unclear whether certain visual features of tape diagrams have implications for learning. We investigated, with undergraduate students and math teachers, whether tape diagrams with different visual features (color, presence of outer lines, and position of the constant) differentially support reasoning about equations and whether people have preferences for certain visual features. Variations in visual features did not affect students’ or teachers’ reasoning accuracy; but each group displayed systematic preferences for most visual features considered. Future research should examine the effects of these visual features on performance while solving equations. 
    more » « less
  3. ; ; ; ; ; (, Proceedings of the 15th International Conference of the Learning Sciences - ICLS 2021)
    de Vries, E.; Hod, Y.; Ahn, J. (Ed.)
    Research has shown that tape diagrams are beneficial for algebra learning. However, it is unclear whether certain visual features of tape diagrams have implications for learning. We investigated, with undergraduate students and math teachers, whether tape diagrams with different visual features (color, presence of outer lines, and position of the constant) differentially support reasoning about equations and whether people have preferences for certain visual features. Variations in visual features did not affect students’ or teachers’ reasoning accuracy; but each group displayed systematic preferences for most visual features considered. Future research should examine the effects of these visual features on performance while solving equations. 
    more » « less
  4. ; ; ; ; (, Advanced Intelligent Systems)
    The adoption of large language models (LLMs) in healthcare has garnered significant research interest, yet their performance remains limited due to a lack of domain‐specific knowledge, medical reasoning skills, and their unimodal nature, which restricts them to text‐only inputs. To address these limitations, we propose MultiMedRes, a multimodal medical collaborative reasoning framework that simulates human physicians’ communication by incorporating a learner agent to proactively acquire information from domain‐specific expert models. MultiMedRes addresses medical multimodal reasoning problems through three steps i) Inquire: The learner agent decomposes complex medical reasoning problems into multiple domain‐specific sub‐problems; ii) Interact: The agent engages in iterative “ask‐answer” interactions with expert models to obtain domain‐specific knowledge; and iii) Integrate: The agent integrates all the acquired domain‐specific knowledge to address the medical reasoning problems (e.g., identifying the difference of disease levels and abnormality sizes between medical images). We validate the effectiveness of our method on the task of difference visual question answering for X‐ray images. The experiments show that our zero‐shot prediction achieves state‐of‐the‐art performance, surpassing fully supervised methods, which demonstrates that MultiMedRes could offer trustworthy and interpretable assistance to physicians in monitoring the treatment progression of patients, paving the way for effective human–AI interaction and collaboration. 
    more » « less
  5. ; ; ; ; (, Proceedings of the National Academy of Sciences)
    We develop an algebraic framework for sequential data assimilation of partially observed dynamical systems. In this framework, Bayesian data assimilation is embedded in a nonabelian operator algebra, which provides a representation of observables by multiplication operators and probability densities by density operators (quantum states). In the algebraic approach, the forecast step of data assimilation is represented by a quantum operation induced by the Koopman operator of the dynamical system. Moreover, the analysis step is described by a quantum effect, which generalizes the Bayesian observational update rule. Projecting this formulation to finite-dimensional matrix algebras leads to computational schemes that are i) automatically positivity-preserving and ii) amenable to consistent data-driven approximation using kernel methods for machine learning. Moreover, these methods are natural candidates for implementation on quantum computers. Applications to the Lorenz 96 multiscale system and the El Niño Southern Oscillation in a climate model show promising results in terms of forecast skill and uncertainty quantification. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email