skip to main content


Title: Option Tracing: Beyond Correctness Analysis in Knowledge Tracing
Award ID(s):
1917713
NSF-PAR ID:
10237350
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
AIED 21: International Conference on Artificial Intelligence in Education
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Background and context. “Explain in Plain English” (EiPE) questions ask students to explain the high-level purpose of code, requiring them to understand the macrostructure of the program’s intent. A lot is known about techniques that experts use to comprehend code, but less is known about how we should teach novices to develop this capability. Objective. Identify techniques that can be taught to students to assist them in developing their ability to comprehend code and contribute to the body of knowledge of how novices develop their code comprehension skills. Method. We developed interventions that could be taught to novices motivated by previous research about how experts comprehend code: prompting students to identify beacons, identify the role of variables, tracing, and abstract tracing. We conducted think-aloud interviews of introductory programming students solving EiPE questions, varying which interventions each student was taught. Some participants were interviewed multiple times throughout the semester to observe any changes in behavior over time. Findings. Identifying beacons and the name of variable roles were rarely helpful, as they did not encourage students to integrate their understanding of that piece in relation to other lines of code. However, prompting students to explain each variable’s purpose helped them focus on useful subsets of the code, which helped manage cognitive load. Tracing was helpful when students incorrectly recognized common programming patterns or made mistakes comprehending syntax (text-surface). Prompting students to pick inputs that potentially contradicted their current understanding of the code was found to be a simple approach to them effectively selecting inputs to trace. Abstract tracing helped students see high-level, functional relationships between variables. In addition, we observed student spontaneously sketching algorithmic visualizations that similarly helped them see relationships between variables. Implications. Because students can get stuck at many points in the process of code comprehension, there seems to be no silver bullet technique that helps in every circumstance. Instead, effective instruction for code comprehension will likely involve teaching a collection of techniques. In addition to these techniques, meta-knowledge about when to apply each technique will need to be learned, but that is left for future research. At present, we recommend teaching a bottom-up, concrete-to-abstract approach. 
    more » « less
  2. Roll, I ; McNamara, D ; Sosnovsky, S ; Luckin, R ; Dimitrova, V. (Ed.)
    Knowledge tracing refers to a family of methods that estimate each student’s knowledge component/skill mastery level from their past responses to questions. One key limitation of most existing knowledge tracing methods is that they can only estimate an overall knowledge level of a student per knowledge component/skill since they analyze only the (usually binary-valued) correctness of student responses. Therefore, it is hard to use them to diagnose specific student errors. In this paper, we extend existing knowledge tracing methods beyond correctness prediction to the task of predicting the exact option students select in multiple choice questions. We quantitatively evaluate the performance of our option tracing methods on two large-scale student response datasets. We also qualitatively evaluate their ability in identifying common student errors in the form of clusters of incorrect options across different questions that correspond to the same error. 
    more » « less
  3. In the United States and around the world, gun violence has become a long-standing public safety concern and a security threat, due to violent gun-related crimes, injuries, and fatalities. Although legislators and lawmakers have attempted to mitigate its threats through legislation, research on gun violence confirms the need for a comprehensive approach to gun violence prevention. This entails addressing the problem in as many ways as possible, such as through legislation, new technological advancements, re-engineering supply, and administrative protocols, among others. The research focuses on the technological, supply, and administrative aspects, in which we propose a manner of managing gun-related data efficiently from the point of manufacture/sale, as well as at points of transfers between secondary sellers for the improvement of criminal investigation processes. Making data more readily available with greater integrity will facilitate successful investigations and prosecutions of gun crimes. Currently, there is no single and uniform platform for firearm manufacturers, dealers, and other stakeholders involved in firearm sales, dissemination, management, and investigation. With the help of Blockchain technology, gun registry, ownership, transfers, and, most importantly, investigations, when crimes occur, can all be managed efficiently, breaking the cycle of gun violence. The identification of guns, gun tracing, and identification of gun owners/possessors rely on accuracy, integrity, and consistency in related systems to influence gun crime investigation processes. Blockchain technology, which uses a consensus-based approach to improve processes and transactions, is demonstrated in this study as a way to enhance these procedures. To the best of our knowledge, this is the first study to explore and demonstrate the utility of Blockchain for gun-related criminal investigations using a design science approach. 
    more » « less
  4. ABSTRACT

    Coordinated spawning of marine animals releases millions of planktonic eggs into the environment, known as egg boons. Eggs are rich in essential fatty acids and may be an important lipid subsidy to egg consumers. Our aim was to validate the application of fatty acid and stable isotope tracers of egg consumption to potential egg consumers and to confirm egg consumption by the selected species. We conducted feeding experiments with ctenophores, crustaceans and fishes. We fed these animals a common diet of Artemia or a commercial feed (Otohime) and simulated egg boons for half of them by intermittently supplementing the common diet with red drum (Sciaenops ocellatus) eggs for 10–94 days. Controls did not receive eggs. Fatty acid profiles of consumers fed eggs were significantly different from those of controls 24 h after the last egg-feeding event. Consumers took on fatty acid characteristics of eggs. In fishes and ctenophores, fatty acid markers of egg consumption did not persist 2–5 days after the last egg-feeding event, but markers of egg consumption persisted in crustaceans for at least 5–10 days. Additionally, consumption of eggs, which had high values of δ15N, led to δ15N enrichment in crustaceans and a fish. We conclude that fatty acids and nitrogen stable isotope can be used as biomarkers of recent egg consumption in marine animals, validating their use for assessing exploitation of egg boons in nature.

     
    more » « less