More Like this

1. Quantifying Individuals’ Theory-Based Knowledge Using Probabilistic Causal Graphs: A Bayesian Hierarchical Approach

   https://doi.org/10.1115/DETC2020-22613  

   Hans, Atharva ; Chaudhari, Ashish M. ; Bilionis, Ilias ; Panchal, Jitesh H. ( August 2020 , ASME 2020 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference)

   null (Ed.)

   Extracting an individual’s knowledge structure is a challenging task as it requires formalization of many concepts and their interrelationships. While there has been significant research on how to represent knowledge to support computational design tasks, there is limited understanding of the knowledge structures of human designers. This understanding is necessary for comprehension of cognitive tasks such as decision making and reasoning, and for improving educational programs. In this paper, we focus on quantifying theory-based causal knowledge, which is a specific type of knowledge held by human designers. We develop a probabilistic graph-based model for representing individuals’ concept-specific causal knowledge for a given theory. We propose a methodology based on probabilistic directed acyclic graphs (DAGs) that uses logistic likelihood function for calculating the probability of a correct response. The approach involves a set of questions for gathering responses from 205 engineering students, and a hierarchical Bayesian approach for inferring individuals’ DAGs from the observed responses. We compare the proposed model to a baseline three-parameter logistic (3PL) model from the item response theory. The results suggest that the graph-based logistic model can estimate individual students’ knowledge graphs. Comparisons with the 3PL model indicate that knowledge assessment is more accurate when quantifying knowledge at the level of causal relations than quantifying it using a scalar ability parameter. The proposed model allows identification of parts of the curriculum that a student struggles with and parts they have already mastered which is essential for remediation.

    

   more » « less
2. A Bayesian Hierarchical Model for Extracting Individuals' Theory-based Causal Knowledge

   https://doi.org/10.1115/1.4055596  

   Hans, Atharva ; Chaudhari, Ashish M. ; Bilionis, Ilias ; Panchal, Jitesh H. ( September 2022 , Journal of Computing and Information Science in Engineering)

   Abstract Extracting an individual's scientific knowledge is essential for improving educational assessment and understanding cognitive tasks in engineering activities such as reasoning and decision making. However, knowledge extraction is an almost impossible endeavor if the domain of knowledge and the available observational data are unrestricted. The objective of this paper is to quantify individuals' theory-based causal knowledge from their responses to given questions. Our approach uses directed acyclic graphs (DAGs) to represent causal knowledge for a given theory and a graph-based logistic model that maps individuals' question-specific subgraphs to question responses. We follow a hierarchical Bayesian approach to estimate individuals' DAGs from observations.The method is illustrated using 205 engineering students' responses to questions on fatigue analysis in mechanical parts. In our results, we demonstrate how the developed methodology provides estimates of population-level DAG and DAGs for individual students. This dual representation is essential for remediation since it allows us to identify parts of a theory that a population or individual struggles with and parts they have already mastered. An addendum of the method is that it enables predictions about individuals' responses to new questions based on the inferred individual-specific DAGs. The latter has implications for the descriptive modeling of human problem-solving, a critical ingredient in sociotechnical systems modeling. 

   more » « less
3. Holistically Budgeting Processing Graphs

   Tong, Zelin ; Ahmed, Shareef ; Anderson, James H. ( December 2023 , Proceedings of the 44th IEEE Real-Time Systems Symposium)

   To certify the schedulability of a system, valid per-task worst-case execution-time (WCET) estimates are almost always required. Unfortunately, on multicore machines, deriving WCET estimates through static analysis that is not highly pessimistic may never be a practical reality. The alternative is to determine WCETs via a measurement process, but such a process cannot correctly produce accurate WCET estimates with certainty. This lack of certainty necessitates the use of overrun-handling mechanisms, such as budget-enforcement techniques, to preserve temporal correctness at runtime. In many systems of interest today, tasks are interconnected to form processing graphs, which can be quite large. The simplest (and perhaps most common) approach to budget enforcement in this case is to abort an entire graph invocation whenever any node (task) overruns its budget. However, such an approach can result in a high abort rate at the graph level even when the per-node abort rate is low. To remedy this situation, this paper presents a holistic budget-management strategy for directed acyclic graphs (DAGs) that involves reallocating per-node budgets to overrunning nodes to avoid DAG-level aborts. To enable the effects of aborts to be studied analytically, a probabilistic analysis is presented to derive a DAG’s abort rate under the proposed budget-management strategy. Experimental results are also presented to demonstrate the utility of budgeting graphs holistically 

   more » « less
4. Leveraging Unstructured Statistical Knowledge in a Probabilistic Language of Thought

   Lew, Alexander K. ; Tessler, Michael Henry ; Mansinghka, Vikash K. ; Tenenbaum, Joshua B. ( January 2020 , Proceedings of the Annual Conference of the Cognitive Science Society)

   One hallmark of human reasoning is that we can bring to bear a diverse web of common-sense knowledge in any situation. The vastness of our knowledge poses a challenge for the practical implementation of reasoning systems as well as for our cognitive theories – how do people represent their common-sense knowledge? On the one hand, our best models of sophisticated reasoning are top-down, making use primarily of symbolically-encoded knowledge. On the other, much of our understanding of the statistical properties of our environment may arise in a bottom-up fashion, for example through asso- ciationist learning mechanisms. Indeed, recent advances in AI have enabled the development of billion-parameter language models that can scour for patterns in gigabytes of text from the web, picking up a surprising amount of common-sense knowledge along the way—but they fail to learn the structure of coherent reasoning. We propose combining these approaches, by embedding language-model-backed primitives into a state- of-the-art probabilistic programming language (PPL). On two open-ended reasoning tasks, we show that our PPL models with neural knowledge components characterize the distribution of human responses more accurately than the neural language models alone, raising interesting questions about how people might use language as an interface to common-sense knowledge, and suggesting that building probabilistic models with neural language-model components may be a promising approach for more human-like AI. 

   more » « less
5. The origins of cognitive flexibility in chimpanzees

   https://doi.org/10.1111/desc.13266  

   Cantwell, Averill ; Buckholtz, Joshua W. ; Atencia, Rebeca ; Rosati, Alexandra G. ( April 2022 , Developmental Science)

   Cognitive flexibility is a core component of executive function, a suite of cognitive capacities that enables individuals to update their behavior in dynamic environments. Human executive functions are proposed to be enhanced compared to other species, but this inference is based primarily on neuroanatomical studies. To address this, we examined the nature and origins of cognitive flexibility in chimpanzees, our closest living relatives. Across three studies, we examined different components of cognitive flexibility using reversal learning tasks where individuals first learned one contingency and then had to shift responses when contingencies flipped. In Study 1, we tested n = 82 chimpanzees ranging from juvenility to adulthood on a spatial reversal task, to characterize the development of basic shifting skills. In Study 2, we tested how n = 24 chimpanzees use spatial versus arbitrary perceptual information to shift, a proposed difference between human and nonhuman cognition. In Study 3, we tested n = 40 chimpanzees on a probabilistic reversal task. We found an extended developmental trajectory for basic shifting and shifting in response to probabilistic feedback—chimpanzees did not reach mature performance until late in ontogeny. Additionally, females were faster to shift than males were. We also found that chimpanzees were much more successful when using spatial versus perceptual cues, and highly perseverative when faced with probabilistic versus consistent outcomes. These results identify both core features of chimpanzee cognitive flexibility that are shared with humans, as well as constraints on chimpanzee cognitive flexibility that may represent evolutionary changes in human cognitive development. 

   more » « less