An official website of the United States government
Concept inventories are standardized assessments that evaluate student understanding of key concepts within academic disciplines. While prevalent across STEM fields, their development lags for advanced computer science topics like dynamic programming (DP)---an algorithmic technique that poses significant conceptual challenges for undergraduates. To fill this gap, we developed and validated a Dynamic Programming Concept Inventory (DPCI). We detail the iterative process used to formulate multiple-choice questions targeting known student misconceptions about DP concepts identified through prior research studies. We discuss key decisions, tradeoffs, and challenges faced in crafting probing questions to subtly reveal these conceptual misunderstandings. We conducted a preliminary psychometric validation by administering the DPCI to 172 undergraduate CS students finding our questions to be of appropriate difficulty and effectively discriminating between differing levels of student understanding. Taken together, our validated DPCI will enable instructors to accurately assess student mastery of DP. Moreover, our approach for devising a concept inventory for an advanced theoretical computer science concept can guide future efforts to create assessments for other under-evaluated areas currently lacking coverage.
more »
« less
Towards Attention-Based Automatic Misconception Identification in Introductory Programming Courses
Identifying misconceptions in student programming solutions is an important step in evaluating their comprehension of fundamental programming concepts. While misconceptions are latent constructs that are hard to evaluate directly from student programs, logical errors can signal their existence in students’ understanding. Tracing multiple occurrences of related logical bugs over different problems can provide strong evidence of students’ misconceptions. This study presents preliminary results of utilizing an interpretable state-ofthe- art Abstract Syntax Tree-based embedding neural network to identify logical mistakes in student code. In this study, we show a proof-of-concept of the errors identified in student programs by classifying correct versus incorrect programs. Our preliminary results show that our framework is able to automatically identify misconceptions without designing and applying a detailed rubric. This approach shows promise for improving the quality of instruction in introductory programming courses by providing educators with a powerful tool that offers personalized feedback while enabling accurate modeling of student misconceptions.
more »
« less
- PAR ID:
- 10501957
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- Special Interest Group on Computer Science Education bulletin
- ISBN:
- 9798400704246
- Page Range / eLocation ID:
- 1680 to 1681
- Format(s):
- Medium: X
- Location:
- Portland OR USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This work-in-progress paper details preliminary results from a qualitative study exploring faculty developers’ interactions with and perceptions of engineering instructional faculty (EIF) at Hispanic-Serving Institutions (HSIs). One potential resource for supporting EIF’s educational innovation efforts is their institutions’ center for teaching and learning (CTL). Through CTLs, and similarly named offices, faculty developers provide EIF and other faculty with professional development opportunities, such as pedagogy workshops, consultations, and seminars. By engaging in services provided by faculty developers, EIF can draw on new ideas, energy, and perspectives for instruction that they can incorporate into their beliefs and practices. This is particularly relevant at HSIs, which play a crucial role in enhancing the education of Latinx engineering students. This study aims to understand HSI faculty developers’ perceptions of EIF’s motivation to participate in professional development programming around instruction. Leveraging the self-determination theory of motivation, our preliminary results suggest that faculty developers recognize how extrinsic and intrinsic factors play an important role in EIF’s decisions to engage in instructional development programming. Based on our preliminary results, we encourage the faculty development community to leverage the identity of EIF as problem-solving engineers, identify and correct misconceptions about the role of faculty developers, and be intentional about how their programming responds to the factors intrinsically and extrinsically motivating EIF.more » « less
-
programming concepts in programming assignments in a CS1 course. We seek to answer the following research questions: RQ1. How effectively can large language models identify knowledge components in a CS1 course from programming assignments? RQ2. Can large language models be used to extract program-level knowledge components, and how can the information be used to identify students’ misconceptions? Preliminary results demonstrated a high similarity between course-level knowledge components retrieved from a large language model and that of an expert-generated list.more » « less
-
Student perceptions of programming can impact their experiences in introductory computer science (CS) courses. For example, some students negatively assess their own ability in response to moments that are natural parts of expert practice, such as using online resources or getting syntax errors. Systems that automatically detect these moments from interaction log data could help us study these moments and intervene when the occur. However, while researchers have analyzed programming log data, few systems detect pre-defined moments, particularly those based on student perceptions. We contribute a new approach and system for detecting programming moments that students perceive as important from interaction log data. We conducted retrospective interviews with 41 CS students in which they identified moments that can prompt negative self-assessments. Then we created a qualitative codebook of the behavioral patterns indicative of each moment, and used this knowledge to build an expert system. We evaluated our system with log data collected from an additional 33 CS students. Our results are promising, with F1 scores ranging from 66% to 98%. We believe that this approach can be applied in many domains to understand and detect student perceptions of learning experiences.more » « less
-
Misconceptions about core linguistic concepts like mutable variables, mutable compound data, and their interaction with scope and higher-order functions seem to be widespread. But how do we detect them, given that experts have blind spots and may not realize the myriad ways in which students can misunderstand programs? Furthermore, once identified, what can we do to correct them? In this paper, we present a curated list of misconceptions, and an instrument to detect them. These are distilled from student work over several years and match and extend prior research. We also present an automated, self-guided tutoring system. The tutor builds on strategies in the education literature and is explicitly designed around identifying and correcting misconceptions. We have tested the tutor in multiple settings. Our data consistently show that (a) the misconceptions we tackle are widespread, and (b) the tutor appears to improve understanding.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3626253.3635575
