skip to main content

Title: Visual analogy: Deep learning versus compositional models
Is analogical reasoning a task that must be learned to solve from scratch by applying deep learning models to massive numbers of reasoning problems? Or are analogies solved by computing similarities between structured representations of analogs? We address this question by comparing human performance on visual analogies created using images of familiar three-dimensional objects (cars and their subregions) with the performance of alternative computational models. Human reasoners achieved above-chance accuracy for all problem types, but made more errors in several conditions (e.g., when relevant subregions were occluded). We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) directly trained to solve these analogy problems, as well as to that of a compositional model that assesses relational similarity between part-based representations. The compositional model based on part representations, but not the deep learning models, generated qualitative performance similar to that of human reasoners.
; ; ; ; ;
Fitch, T.; Lamm, C.; Leder, H.; Teßmar-Raible, K.
Award ID(s):
Publication Date:
Journal Name:
Proceedings of the 43rd Annual Meeting of the Cognitive Science Society
Sponsoring Org:
National Science Foundation
More Like this
  1. By middle childhood, humans are able to learn abstract semantic relations (e.g., antonym, synonym, category membership) and use them to reason by analogy. A deep theoretical challenge is to show how such abstract relations can arise from nonrelational inputs, thereby providing key elements of a protosymbolic representation system. We have developed a computational model that exploits the potential synergy between deep learning from “big data” (to create semantic features for individual words) and supervised learning from “small data” (to create representations of semantic relations between words). Given as inputs labeled pairs of lexical representations extracted by deep learning, the modelmore »creates augmented representations by remapping features according to the rank of differences between values for the two words in each pair. These augmented representations aid in coping with the feature alignment problem (e.g., matching those features that make “love-hate” an antonym with the different features that make “rich-poor” an antonym). The model extracts weight distributions that are used to estimate the probabilities that new word pairs instantiate each relation, capturing the pattern of human typicality judgments for a broad range of abstract semantic relations. A measure of relational similarity can be derived and used to solve simple verbal analogies with human-level accuracy. Because each acquired relation has a modular representation, basic symbolic operations are enabled (notably, the converse of any learned relation can be formed without additional training). Abstract semantic relations can be induced by bootstrapping from nonrelational inputs, thereby enabling relational generalization and analogical reasoning.

    « less
  2. Many computational models of reasoning rely on explicit relation representations to account for human cognitive capacities such as analogical reasoning. Relational luring, a phenomenon observed in recognition memory, has been interpreted as evidence that explicit relation representations also impact episodic memory; however, this assumption has not been rigorously assessed by computational modeling. We implemented an established model of recognition memory, the Generalized Context Model (GCM), as a framework for simulating human performance on an old/new recognition task that elicits relational luring. Within this basic theoretical framework, we compared representations based on explicit relations, lexical semantics (i.e., individual word meanings), andmore »a combination of the two. We compared the same alternative representations as predictors of accuracy in solving explicit verbal analogies. In accord with previous work, we found that explicit relation representations are necessary for modeling analogical reasoning. In contrast, preliminary simulations incorporating model parameters optimized to fit human data reproduce relational luring using any of the alternative representations, including one based on non-relational lexical semantics. Further work on model comparisons is needed to examine the contributions of lexical semantics and relations on the luring effect in recognition memory.« less
  3. Analogy problems involving multiple ordered relations of the same type create mapping ambiguity, requiring some mechanism for relational integration to achieve mapping accuracy. We address the question of whether the integration of ordered relations depends on their logical form alone, or on semantic representations that differ across relation types. We developed a triplet mapping task that provides a basic paradigm to investigate analogical reasoning with simple relational structures. Experimental results showed that mapping performance differed across orderings based on category, linear order, and causal relations, providing evidence that each transitive relation has its own semantic representation. Hence, human analogical mappingmore »of ordered relations does not depend solely on their formal property of transitivity. Instead, human ability to solve mapping problems by integrating relations relies on the semantics of relation representations. We also compared human performance to the performance of several vector-based computational models of analogy. These models performed above chance but fell short of human performance for some relations, highlighting the need for further model development.« less
  4. We report a first effort to model the solution of meaningful four-term visual analogies, by combining a machine-vision model (ResNet50-A) that can classify pixel-level images into object categories, with a cognitive model (BART) that takes semantic representations of words as input and identifies semantic relations instantiated by a word pair. Each model achieves above-chance performance in selecting the best analogical option from a set of four. However, combining the visual and the semantic models increases analogical performance above the level achieved by either model alone. The contribution of vision to reasoning thus may extend beyond simply generating verbal representations frommore »images. These findings provide a proof of concept that a comprehensive model can solve semantically-rich analogies from pixel-level inputs.« less
  5. There is a large gap between the ability of experts and students in grasping spatial concepts and representations. Engineering and the geosciences require the highest expertise in spatial thinking, and weak spatial skills are a significant barrier to success for many students [1]. Spatial skills are also highly malleable [2]; therefore, a current challenge is to identify how to promote students’ spatial thinking. Interdisciplinary research on how students think about spatially-demanding problems in the geosciences has identified several major barriers for students and interventions to help scaffold learning at a variety of levels from high school through upper level undergraduatemore »majors. The Geoscience Education Transdisciplinary Spatial Learning Network (GET-Spatial; is an NSF-funded collaboration between geoscientists, cognitive psychologists, and education researchers. Our goal is to help students overcome initial hurdles in reasoning about spatial problems in an effort to diversify the geoscience workforce. Examples of spatial problems in the fields of geochemistry include scaling, both in size and time; penetrative thinking to make inferences about internal structures from surface properties; and graph-reading, especially ternary diagrams. Understanding scales outside of direct human experience, both very large (e.g. cosmochemistry, deep time) and very small (e.g. mineralogy, nanoparticles) can be acutely difficult for students. However, interventions have successfully resulted in improvements to scale estimations and improve exam performance [3]. We will discuss best practices for developing effective interdisciplinary teams, and how to overcome challenges of working across disciplines and across grade levels. We will provide examples of spatial interventions in scaling and penetrative thinking. [1] Hegarty et al. (2010) in Spatial Cognition VII 6222, 85- 94. [2] Uttal et al. (2012) Psychology of Learning and Motivation 57, 147-181. [3] Resnick et al. (2016) Educational Psychology Review, 1-15.« less