An official website of the United States government
Since intermediate CS students can use a variety of control structures, why do their choices often not match experts' Students may not realize what choices expert prefer, find non-expert choices easier to read, or simply forget to write with expert structure. To disentangle these explanations, we surveyed 328 2nd and 3rd semester undergraduates, with tasks including writing short functions, selecting which structure was most readable or best styled, and comprehension questions. Questions focused on seven control structure topics that were important to instructors (e.g., factoring out repeated code between an if-block and its else). Students frequently wrote with non-expert structure, and, for five topics, at least 1/3 of students (48% - 71%) thought a non-expert structure was more readable than the expert one. However, students often made one choice when writing code, but preferred a different choice when reading it. Additionally, for more complex topics, students often failed to notice (or understand) differences in execution caused by changes in structure. Together, these results suggest that instruction and practice for choosing control structures should be context-specific, and that assessment focused only on code writing may miss underlying misunderstandings.
more »
« less
Improving Assessment of Programming Pattern Knowledge through Code Editing and Revision
Abstract: How well do code-writing tasks measure students’ knowledge of programming patterns and anti-patterns? How can we assess this knowledge more accurately? To explore these questions, we surveyed 328 intermediate CS students and measured their performance on different types of tasks, including writing code, editing someone else’s code, and, if applicable, revising their own alternatively-structured code. Our tasks targeted returning a Boolean expression and using unique code within an if and else.We found that code writing sometimes under-estimated student knowledge. For tasks targeting returning a Boolean expression, over 55% of students who initially wrote with non-expert structure successfully revised to expert structure when prompted - even though the prompt did not include guidance on how to improve their code. Further, over 25% of students who initially wrote non-expert code could properly edit someone else’s non-expert code to expert structure. These results show that non-expert code is not a reliable indicator of deep misconceptions about the structure of expert code. Finally, although code writing is correlated with code editing, the relationship is weak: a model with code writing as the sole predictor of code editing explains less than 15% of the variance. Model accuracy improves when we include additional predictors that reflect other facets of knowledge, namely the identification of expert code and selection of expert code as more readable than non-expert code. Together, these results indicate that a combination of code writing, revising, editing, and identification tasks can provide a more accurate assessment of student knowledge of programming patterns than code writing alone.
more »
« less
- Award ID(s):
- 1948519
- PAR ID:
- 10451951
- Date Published:
- Journal Name:
- 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET)
- Page Range / eLocation ID:
- 58 to 69
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
University students have begun to use Artificial Intelligence (AI) in many different ways in their undergraduate education, some beneficial to their learning, and some simply expedient to completing assignments with as little work as possible. This exploratory qualitative study examines how undergraduate students used AI in a large General Education course on sustainability and technology at a research university in the United States in 2023. Thirty-nine students documented their use of AI in their final course project, which involved analyzing conceptual networks connecting core sustainability concepts. Through iterative qualitative coding, we identified key patterns in students’ AI use, including higher-order writing tasks (understanding complex topics, finding evidence), lower-order writing tasks (revising, editing, proofreading), and other learning activities (efficiency enhancement, independent research). Students primarily used AI to improve communication of their original ideas, though some leveraged it for more complex tasks like finding evidence and developing arguments. Many students expressed skepticism about AI-generated content and emphasized maintaining their intellectual independence. While some viewed AI as vital for improving their work, others explicitly distinguished between AI-assisted editing and their original thinking. This analysis provides insight into how students navigate AI use when it is explicitly permitted in coursework, with implications for effectively integrating AI into higher education to support student learning.more » « less
-
null (Ed.)Experts often use particular control flow structures to make their code easier to read and modify, such as using the logical operator AND to conjoin conditions rather than nesting separate if statements. Within Boolean expressions, experts take advantage of short-circuit evaluation by ordering their conditions to avoid errors (such as checking that an index is within the bounds of an array before examining the value at that index). How well do students understand these structures? We investigate students' use and understanding of conjoined versus separate conditions within a larger assessment of 125 undergraduate students at the end of their second- and third-semester CS courses (in algorithms & data structures and introductory software engineering). The assessment asked students to: write code where an edge case error could be avoided with short-circuit evaluation, revise their code with nudges towards expert structure, and answer comprehension questions involving code tracing. When writing, students frequently forgot to check for a key edge case. When that case was included, the check was often separated in its own if-statement rather than conjoined with the other conditions. This could indicate a stylistic choice or a belief that the check had to be separated for functionality. Notably, students who included all necessary conditions rarely exhibited the error of ordering them incorrectly. However, with code comprehension, students demonstrated significant misunderstandings about the effects of condition ordering. Students were more accurate on comprehension tasks with nested ifs than conjoined conditions, and this effect was most pronounced when the ordering of the conditions would lead to errors. When conditions were conjoined in a single expression, only 35% of students recognized that checking a value at an index before checking that the index was in bounds would lead to an error. However, 54% of students recognized the problem when the conditions were separated into individual if-statements. This demonstrates a subtlety in code execution that intermediate students may not have mastered and emphasizes the challenges in assessing students' understanding solely via the way they write code.more » « less
-
Nehm, Ross (Ed.)Although many scientists agree that evolution does not make claims about God/god(s), students might assume that evolution is atheistic, and this may lead to lower evolution acceptance. In study 1, we surveyed 1081 college biology students at one university about their religiosity and evolution acceptance and asked what religious ideas someone would have to reject if that person were to accept evolution. Approximately half of students wrote that a person cannot believe in God/religion and accept evolution, indicating that these students may have atheistic perceptions of evolution. Religiosity was not related to whether a student wrote that evolution is atheistic, but writing that evolution is atheistic was associated with lower evolution acceptance among the more religious students. In study 2, we collected data from 1898 students in eight states in the United States using a closed-ended survey. We found that 56.5% of students perceived that evolution is atheistic even when they were given the option to choose an agnostic perception of evolution. Further, among the most religious students, those who thought evolution is atheistic were less accepting of evolution, less comfortable learning evolution, and perceived greater conflict between their personal religious beliefs and evolution than those who thought evolution is agnostic.more » « less
-
Adebisi, John (Ed.)Non-expert users can now program robots using various end-user robot programming methods, which have widened the use of robots and lowered barriers preventing robot use by laypeople. Kinesthetic teaching is a common form of end-user robot programming, allowing users to forgo writing code by physically guiding the robot to demonstrate behaviors. Although it can be more accessible than writing code, kinesthetic teaching is difficult in practice because of users’ unfamiliarity with kinematics or limitations of robots and programming interfaces. Developing good kinesthetic demonstrations requires physical and cognitive skills, such as the ability to plan effective grasps for different task objects and constraints, to overcome programming difficulties. How to help users learn these skills remains a largely unexplored question, with users conventionally learning through self-guided practice. Our study compares how self-guided practice compares with curriculum-based training in building users’ programming proficiency. While we found no significant differences between study participants who learned through practice compared to participants who learned through our curriculum, our study reveals insights into factors contributing to end-user robot programmers’ confidence and success during programming and how learning interventions may contribute to such factors. Our work paves the way for further research on how to best structure training interventions for end-user robot programmers.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICSE-SEET58685.2023.00012
