skip to main content


Title: Verbal analogy problem sets: An inventory of testing materials
Analogical reasoning is an active topic of investigation across education, artificial intelligence (AI), cognitive psychology, and related fields. In all fields of inquiry, explicit analogy problems provide useful tools for investigating the mechanisms underlying analogical reasoning. Such sets have been developed by researchers working in the fields of educational testing, AI, and cognitive psychology. However, these analogy tests have not been systematically made accessible across all the relevant fields. The present paper aims to remedy this situation by presenting a working inventory of verbal analogy problem sets, intended to capture and organize sets from diverse sources.  more » « less
Award ID(s):
1827374
NSF-PAR ID:
10148744
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Behavior research methods
ISSN:
1554-351X
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    We examined the relationship between metaphor comprehension and verbal analogical reasoning in young adults who were either typically developing (TD) or diagnosed with Autism Spectrum Disorder (ASD). The ASD sample was highly educated and high in verbal ability, and closely matched to a subset of TD participants on age, gender, educational background, and verbal ability. Additional TD participants with a broader range of abilities were also tested. Each participant solved sets of verbal analogies and metaphors in verification formats, allowing measurement of both accuracy and reaction times. Measures of individual differences in vocabulary, verbal working memory, and autistic traits were also obtained. Accuracy for both the verbal analogy and the metaphor task was very similar across the ASD and matched TD groups. However, reaction times on both tasks were longer for the ASD group. Additionally, stronger correlations between verbal analogical reasoning and working memory capacity in the ASD group indicated that processing verbal analogies was more effortful for them. In the case of both groups, accuracy on the metaphor and analogy tasks was correlated. A mediation analysis revealed that after controlling for working memory capacity, the inter‐task correlation could be accounted for by the mediating variable of vocabulary knowledge, suggesting that the primary common mechanisms linking the two tasks involve language skills.

     
    more » « less
  2. Analogy—the ability to find and apply deep structural patterns across domains—has been fundamental to human innovation in science and technology. Today there is a growing opportunity to accelerate innovation by moving analogy out of a single person’s mind and distributing it across many information processors, both human and machine. Doing so has the potential to overcome cognitive fixation, scale to large idea repositories, and support complex problems with multiple constraints. Here we lay out a perspective on the future of scalable analogical innovation and first steps using crowds and artificial intelligence (AI) to augment creativity that quantitatively demonstrate the promise of the approach, as well as core challenges critical to realizing this vision.

     
    more » « less
  3. There is a large gap between the ability of experts and students in grasping spatial concepts and representations. Engineering and the geosciences require the highest expertise in spatial thinking, and weak spatial skills are a significant barrier to success for many students [1]. Spatial skills are also highly malleable [2]; therefore, a current challenge is to identify how to promote students’ spatial thinking. Interdisciplinary research on how students think about spatially-demanding problems in the geosciences has identified several major barriers for students and interventions to help scaffold learning at a variety of levels from high school through upper level undergraduate majors. The Geoscience Education Transdisciplinary Spatial Learning Network (GET-Spatial; http://serc.carleton.edu/getspatial/) is an NSF-funded collaboration between geoscientists, cognitive psychologists, and education researchers. Our goal is to help students overcome initial hurdles in reasoning about spatial problems in an effort to diversify the geoscience workforce. Examples of spatial problems in the fields of geochemistry include scaling, both in size and time; penetrative thinking to make inferences about internal structures from surface properties; and graph-reading, especially ternary diagrams. Understanding scales outside of direct human experience, both very large (e.g. cosmochemistry, deep time) and very small (e.g. mineralogy, nanoparticles) can be acutely difficult for students. However, interventions have successfully resulted in improvements to scale estimations and improve exam performance [3]. We will discuss best practices for developing effective interdisciplinary teams, and how to overcome challenges of working across disciplines and across grade levels. We will provide examples of spatial interventions in scaling and penetrative thinking. [1] Hegarty et al. (2010) in Spatial Cognition VII 6222, 85- 94. [2] Uttal et al. (2012) Psychology of Learning and Motivation 57, 147-181. [3] Resnick et al. (2016) Educational Psychology Review, 1-15. 
    more » « less
  4. Analogical reasoning is considered to be a critical cognitive skill in programming. However, it has been rarely studied in a block-based programming context, especially involving both virtual and physical objects. In this multi-case study, we examined how novice programming learners majoring in early childhood education used analogical reasoning while debugging block code to make a robot perform properly. Screen recordings, scaffolding entries, reflections, and block code were analyzed. The cross-case analysis suggested multimodal objects enabled the novice programming learners to identify and use structural relations. The use of a robot eased the verification process by enabling them to test their analogies immediately after the analogy application. Noticing similar functional analogies led to noticing similarities in the relation between block code as well as between block code and the robot, guiding to locate bugs. Implications and directions for future educational computing research are discussed.

     
    more » « less
  5. Abstract

    Advances in artificial intelligence have raised a basic question about human intelligence: Is human reasoning best emulated by applying task‐specific knowledge acquired from a wealth of prior experience, or is it based on the domain‐general manipulation and comparison of mental representations? We address this question for the case of visual analogical reasoning. Using realistic images of familiar three‐dimensional objects (cars and their parts), we systematically manipulated viewpoints, part relations, and entity properties in visual analogy problems. We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) that were directly trained to solve these problems and to apply their task‐specific knowledge to analogical reasoning. We also developed a new model using part‐based comparison (PCM) by applying a domain‐general mapping procedure to learned representations of cars and their component parts. Across four‐term analogies (Experiment 1) and open‐ended analogies (Experiment 2), the domain‐general PCM model, but not the task‐specific deep learning models, generated performance similar in key aspects to that of human reasoners. These findings provide evidence that human‐like analogical reasoning is unlikely to be achieved by applying deep learning with big data to a specific type of analogy problem. Rather, humans do (and machines might) achieve analogical reasoning by learning representations that encode structural information useful for multiple tasks, coupled with efficient computation of relational similarity.

     
    more » « less