More Like this

1. Not quite any way you slice it: How different analogical constructions affect Raven's Matrices performance

   Yang, Y.; McGreggor, K.; Kunda, M. (January 2020, Proceedings of the Eighth Annual Conference on Advances in Cognitive Systems (ACS))

   null (Ed.)

   Analogical reasoning fundamentally involves exploiting redundancy in a given task, but there are many different ways an intelligent agent can choose to define and exploit redundancy, often resulting in very different levels of task performance. We explore such variations in analogical reasoning within the domain of geometric matrix reasoning tasks, namely on the Raven’s Standard Progressive Matrices intelligence test. We show how different analogical constructions used by the same basic visual-imagery-based computational model—varying only in how they “slice” a matrix problem into parts and do search and optimization within/across these parts—achieve very different levels of test performance, ranging from 13/60 correct all the way up to 57/60 correct. Our findings suggest that the ability to select or build effective high-level analogical constructions can be as important as an agent’s competencies in low-level reasoning skills, which raises interesting open questions about the extent to which building the “right” analogies might contribute to individual differences in human matrix reasoning performance, and how intelligent agents might learn to build or select from among different analogical constructions in the first place. 

   more » « less
2. AI, visual imagery, and a case study on the challenges posed by human intelligence tests

   https://doi.org/10.1073/pnas.1912335117  

   Kunda, Maithilee (November 2020, Proceedings of the National Academy of Sciences)

   Observations abound about the power of visual imagery in human intelligence, from how Nobel prize-winning physicists make their discoveries to how children understand bedtime stories. These observations raise an important question for cognitive science, which is, what are the computations taking place in someone’s mind when they use visual imagery? Answering this question is not easy and will require much continued research across the multiple disciplines of cognitive science. Here, we focus on a related and more circumscribed question from the perspective of artificial intelligence (AI): If you have an intelligent agent that uses visual imagery-based knowledge representations and reasoning operations, then what kinds of problem solving might be possible, and how would such problem solving work? We highlight recent progress in AI toward answering these questions in the domain of visuospatial reasoning, looking at a case study of how imagery-based artificial agents can solve visuospatial intelligence tests. In particular, we first examine several variations of imagery-based knowledge representations and problem-solving strategies that are sufficient for solving problems from the Raven’s Progressive Matrices intelligence test. We then look at how artificial agents, instead of being designed manually by AI researchers, might learn portions of their own knowledge and reasoning procedures from experience, including learning visuospatial domain knowledge, learning and generalizing problem-solving strategies, and learning the actual definition of the task in the first place. 

   more » « less
3. Learning algebraic representation for systematic generalization in abstract reasoning

   Zhang, C.; Xie, S.; Jia B.; Wu, Y. N.; Zhu, S.-C.; Zhu, Y. (January 2022, European Conference on Computer Vision (ECCV 2022))

   Is intelligence realized by connectionist or classicist? While connectionist approaches have achieved superhuman performance, there has been growing evidence that such task-specific superiority is particularly fragile in systematic generalization. This observation lies in the central debate between connectionist and classicist, wherein the latter continually advocates an algebraic treatment in cognitive architectures. In this work, we follow the classicist’s call and propose a hybrid approach to improve systematic generalization in reasoning. Specifically, we showcase a prototype with algebraic representation for the abstract spatial-temporal reasoning task of Raven’s Progressive Matrices (RPM) and present the ALgebra-Aware Neuro-Semi-Symbolic (ALANS) learner. The ALANS learner is motivated by abstract algebra and the representation theory. It consists of a neural visual perception frontend and an algebraic abstract reasoning backend: the frontend summarizes the visual information from object-based representation, while the backend transforms it into an algebraic structure and induces the hidden operator on the fly. The induced operator is later executed to predict the answer’s representation, and the choice most similar to the prediction is selected as the solution. Extensive experiments show that by incorporating an algebraic treatment, the ALANS learner outperforms various pure connectionist models in domains requiring systematic generalization. We further show the generative nature of the learned algebraic representation; it can be decoded by isomorphism to generate an answer. 

   more » « less
4. Swarm Intelligence Amplifies the IQ of Collaborating Teams

   https://doi.org/10.1109/AI4I.2019.00036  

   Rosenberg, Louis; Willcox, Gregg (September 2019, ai4i 2019)

   In the natural world, Swarm Intelligence (SI) is a well-known phenomenon that enables groups of organisms to make collective decisions with significantly greater accuracy than the individuals could do on their own. In recent years, a new AI technology called Artificial Swarm Intelligence (ASI) has been developed that enables similar benefits for human teams. It works by connecting networked teams into real-time systems modeled on natural swarms. Referred to commonly as “human swarms” or “hive minds,” these closed-loop systems have been shown to amplify group performance across a wide range of tasks, from financial forecasting to strategic decision-making. The current study explores the ability of ASI technology to amplify the IQ of small teams. Five small teams answered a series of questions from a commonly used intelligence test known as the Raven’s Standard Progressive Matrices (RSPM) test. Participants took the test first as individuals, and then as groups moderated by swarming algorithms (i.e. “swarms”). The average individual achieved 53.7% correct, while the average swarm achieved 76.7% correct, corresponding to an estimated IQ increase of 14 points. When the individual responses were aggregated by majority vote, the groups scored 56.7% correct, still 12 IQ points less than the real-time swarming method. 

   more » « less
5. Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videos

   Chang, Matthew; Prakash, Aditya; Gupta, Saurabh (December 2023, Neural Information Processing Systems)

   The analysis and use of egocentric videos for robotic tasks is made challenging by occlusion due to the hand and the visual mismatch between the human hand and a robot end-effector. In this sense, the human hand presents a nuisance. However, often hands also provide a valuable signal, e.g. the hand pose may suggest what kind of object is being held. In this work, we propose to extract a factored representation of the scene that separates the agent (human hand) and the environment. This alleviates both occlusion and mismatch while preserving the signal, thereby easing the design of models for downstream robotics tasks. At the heart of this factorization is our proposed Video Inpainting via Diffusion Model (VIDM) that leverages both a prior on real-world images (through a large-scale pre-trained diffusion model) and the appearance of the object in earlier frames of the video (through attention). Our experiments demonstrate the effectiveness of VIDM at improving inpainting quality on egocentric videos and the power of our factored representation for numerous tasks: object detection, 3D reconstruction of manipulated objects, and learning of reward functions, policies, and affordances from videos. 

   more » « less