An official website of the United States government
Knowledge tracing is a method to model students’ knowledge and enable personalized education in many STEM disciplines such as mathematics and physics, but has so far still been a challenging task in computing disciplines. One key obstacle to successful knowledge tracing in computing education lies in the accurate extraction of knowledge components (KCs), since multiple intertwined KCs are practiced at the same time for programming problems. In this paper, we address the limitations of current methods and explore a hybrid approach for KC extraction, which combines automated code parsing with an expert-built ontology. We use an introductory (CS1) Java benchmark dataset to compare its KC extraction performance with the traditional extraction methods using a state-of-the-art evaluation approach based on learning curves. Our preliminary results show considerable improvement over traditional methods of student modeling. The results indicate the opportunity to improve automated KC extraction in CS education by incorporating expert knowledge into the process.
more »
« less
Tracking academic contributions to policymaking for autonomous vehicles
In this preliminary analysis, we see support for the theoretical framework delineating two approaches to using knowledge in policy making – expert and convening. In addition, we confirm expectations that academic knowledge is more influential in the expert approach to learning. However, academic knowledge is still drawn upon in the convening approach. This preliminary analysis will be extended and elaborated in the full conference presentation.
more »
« less
- Award ID(s):
- 2001455
- PAR ID:
- 10503442
- Publisher / Repository:
- Zenodo
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Large scale analysis of source code, and in particular scientific source code, holds the promise of better understanding the data science process, identifying analytical best practices, and providing insights to the builders of scientific toolkits. However, large corpora have remained unanalyzed in depth, as descriptive labels are absent and require expert domain knowledge to generate. We propose a novel weakly supervised transformer-based architecture for computing joint representations of code from both abstract syntax trees and surrounding natural language comments. We then evaluate the model on a new classification task for labeling computational notebook cells as stages in the data analysis process from data import to wrangling, exploration, modeling, and evaluation. We show that our model, leveraging only easily-available weak supervision, achieves a 38% increase in accuracy over expert-supplied heuristics and outperforms a suite of baselines. Our model enables us to examine a set of 118,000 Jupyter Notebooks to uncover common data analysis patterns. Focusing on notebooks with relationships to academic articles, we conduct the largest study of scientific code to date and find that notebooks which devote an higher fraction of code to the typically labor-intensive process of wrangling data in expectation exhibit decreased citation counts for corresponding papers. We also show significant differences between academic and non-academic notebooks, including that academic notebooks devote substantially more code to wrangling and exploring data, and less on modeling.more » « less
-
Frank, Brian W.; Jones, Dyan L.; Ryan, Qing X. (Ed.)Many of the activities and cognitive processes that physicists use while solving problems are "invisible" to students, which can hinder their acquisition of important expert-like skills. Whereas the detailed calculations performed by researchers are often published in journals and textbooks, other activities such as those undertaken while planning how to approach a problem are rarely discussed in published research. Hence, these activities are especially hidden from students. To better understand how physicists solve problems in their professional research, we leveraged the framework of cognitive task analysis to conduct semi-structured interviews with theoretical physicists (N=11). Here we elucidate the role of planning and preliminary analysis in theorists' work. Theorists described using a variety of activities in order to decide if their project was doable while also generating possible solution paths. These actions included doing cursory calculations, reflecting on previous knowledge, gaining intuition and understanding by studying prior work, and reproducing previous results. We found that theorists typically did not pursue projects unless they had a clear idea of what the outcome of their project would be, or at least knew that they would be able to make progress on the problem. Thus, this preliminary design and analysis phase was highly important for theorists despite being largely hidden from students. We conclude by suggesting potential ways to incorporate our findings into the classroom to give students more numerous opportunities to engage in these expert-like practices.more » « less
-
null (Ed.)We report on the development of a new instrument for measuring teachers' knowledge of language as an epistemic tool in science classes. Language is essential for science learning, as all learning requires the use of language to constitute one's own ideas and to engage with others' ideas. Teachers with knowledge of language as an epistemic tool can recognize the ways that language allows students to generate and validate knowledge for themselves, rather than to replicate canonical knowledge transmitted by other sources.We used a construct‐driven development approach with iterations of domain analysis, item revision, teacher feedback, expert review, and item piloting to address the content, substance, and structure aspects of validity. Data from 158 preservice and in‐service teachers on 27 preliminary items were collected. Findings from Rasch measurement modeling indicate a single dimension fits the items well and can distinguish teachers of higher and lower knowledge. We revised and selected 15 items for an updated instrument. This contributes to ongoing measurement projects and provides a potential instrument for future, broader use by the field to gauge teachers' knowledge of language as an epistemic tool.more » « less
-
Abstract Physics laboratory courses (PLC) have been recently the topic of several research studies examining their effectiveness at reaching their goals. As a result, a discussion about the effectiveness of traditional PLC for students’ content knowledge, skills, and “expert-thinking” acquisition has developed. Critical for the investigation of students learning in those settings has been the development of research-based assessments tools. An example of those is the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS). Recently, we translated the E-CLASS into German and set up a centralized survey administration system for instructors, allowing data acquisition and automated data analysis. Previously, we described this process and presented the preliminary results of the study of the introductory PLC at the University of Potsdam (UP). Here, we present an extended study that allows us to make stronger conclusions about students’ views about experimental physics at the UP. Overall, we find that students at US institutions have a higher level of “expert-like” views than students at the UP.more » « less
