An official website of the United States government
Introduction: The emergence and widespread adoption of generative AI (GenAI) chatbots such as ChatGPT, and programming assistants such as GitHub Copilot, have radically redefined the landscape of programming education. This calls for replication of studies and reexamination of findings from pre-GenAI CS contexts to understand the impact on students. Objectives: Achievement Goals are well studied in computing education and can be predictive of student interest and exam performance. The objective in this study is to compare findings from prior achievement goal studies in CS1 courses with new CS1 courses that emphasize the use of human-GenAI collaborative coding. Methods: In a CS1 course that integrates GenAI, we use linear regression to explore the relationship between achievement goals and prior experience on student interest, exam performance, and perceptions of GenAI. Results: As with prior findings in traditional CS1 classes, Mastery goals are correlated with interest in computing. Contradicting prior CS1 findings, normative goals are correlated with exam scores. Normative and mastery goals correlate with students’ perceptions of learning with GenAI. Mastery goals weakly correlate with reading and testing code output from GenAI.
more »
« less
Achievement Goals in CS1: Replication and Extension
Replication research is rare in CS education. For this reason, it is often unclear to what extent our findings generalize beyond the context of their generation. The present paper is a replication and extension of Achievement Goal Theory research on CS1 students. Achievement goals are cognitive representations of desired competence (e.g., topic mastery, outperforming peers) in achievement settings, and can predict outcomes such as grades and interest. We study achievement goals and their effects on CS1 students at six institutions in four countries. Broad patterns are maintained --- mastery goals are beneficial while appearance goals are not --- but our data additionally admits fine-grained analyses that nuance these findings. In particular, students' motivations for goal pursuit can clarify relationships between performance goals and outcomes.
more »
« less
- Award ID(s):
- 1712508
- PAR ID:
- 10065488
- Date Published:
- Journal Name:
- Proceedings of the 49th ACM Technical Symposium on Computer Science Education
- Page Range / eLocation ID:
- 687 to 692
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Students’ view of intelligence (i.e., their mindset beliefs) has been found to be related to their self-efficacy and goal orientations as well as to influence their course outcomes. Comparisons of students’ chemistry mindset between different groups found that organic chemistry I students held more of a growth mindset than general chemistry I students at the beginning of a term. Additionally, men tended to hold more growth mindset beliefs than women. Given these differences, structural equation modeling was used to explore the relations between students’ mindset, self-efficacy, and goal orientations, along with their relation to achievement outcomes within a course. An indirect effect of mindset on summative achievement was found to be mediated through performance-avoidance goals, whereas the relation between self-efficacy and summative achievement was mediated through performance-approach, mastery-avoidance, and performance-avoidance goal orientations. While mindset was not found to be directly or indirectly related to formative achievement outcomes, self-efficacy was found to have an indirect effect on formative achievement through mastery-approach and mastery-avoidance goal orientations. Additionally, an interaction between mindset and self-efficacy was found to be related to performance-avoidance goals, as has been suggested in prior studies. These results point to the importance of mindset on achievement outcomes while also considering influences from self-efficacy and goal orientations. Future work is encouraged to investigate how these variables are related when they are measured throughout a term.more » « less
-
Well-designed instructional videos are powerful tools for helping students learn and prompting students to use generative strategies while learning from videos further bolsters their effectiveness. However, little is known about how individual differences in motivational factors, such as achievement goals, relate to how students learn within multimedia environments that include instructional videos and generative strategies. Therefore, in this study, we explored how achievement goals predicted undergraduate students’ behaviors when learning with instructional videos that required students to answer practice questions between videos, as well as how those activities predicted subsequent unit exam performance one week later. Additionally, we tested the best measurement models for modeling achievement goals between traditional confirmatory factor analysis and bifactor confirmatory factor analysis. The bifactor model fit our data best and was used for all subsequent analyses. Results indicated that stronger mastery goal endorsement predicted performance on the practice questions in the multimedia learning environment, which in turn positively predicted unit exam performance. In addition, students’ time spent watching videos positively predicted practice question performance. Taken together, this research emphasizes the availing role of adaptive motivations, like mastery goals, in learning from instructional videos that prompt the use of generative learning strategies.more » « less
-
Course-based undergraduate research experiences (CUREs) provide students opportunities to engage in research in a course. Aspects of CURE design, such as providing students opportunities to make discoveries, collaborate, engage in relevant work, and iterate to solve problems are thought to contribute to outcome achievement in CUREs. Yet how each of these elements contributes to specific outcomes is largely unexplored. This lack of understanding is problematic, because we may unintentionally underemphasize important aspects of CURE design that allow for achievement of highly valued outcomes when designing or teaching our courses. In this work, we take a qualitative approach and leverage unique circumstances in two offerings of a CURE to investigate how these design elements influence outcome achievement. One offering experienced many research challenges that increased engagement in iteration. This level of research challenge ultimately prevented achievement of predefined research goals. In the other offering, students experienced fewer research challenges and ultimately achieved predefined research goals. Our results suggest that, when students encounter research challenges and engage in iteration, they have the potential to increase their ability to navigate scientific obstacles. In addition, our results suggest roles for collaboration and autonomy, or directing one’s own work, in outcome achievement.more » « less
-
Ko, A. K. (Ed.)There are significant participation gaps in computing, and the way to address these participation gaps lies not simply in getting students from underrepresented groups into a CS1 classroom, but supporting students to pursue their interest in computing further beyond CS1. There are many factors that may influence students’ pursuit of computing beyond introductory courses, including their sense that they can do what CS courses require of them (their self-efficacy) and positive emotional experiences in CS courses. When interest has been addressed in computing education, research has treated it mostly as an outcome of particular pedagogical approaches or curricula; what has not been studied is how students’ longer-term interest develops through more granular experiences that students have as they begin to engage with computing. In this paper, we present the results of a study designed to investigate how students’ interest in computing develops as a product of their momentary self-efficacy and affective experiences. Using a methodology that is relatively uncommon to computer science education—the experience sampling method, which involves frequently asking students brief, unobtrusive questions about their experiences—we surveyed CS1 students every week over the course of a semester to capture the nuances of their experiences. 74 CS1 students responded 14-18 times over the course of a semester about their self-efficacy, frustration, and situational interest. With this data, we used a multivariate, multi-level statistical model that allowed us to estimate how students’ granular, momentary experiences (measured through the experience sampling method surveys) and initial interest, self-efficacy, and self-reported gender (measured through traditional surveys) relate to their longer-term interest and achievement in the course. We found that students’ momentary experiences have a significant impact on their interest in computing and course outcomes, even controlling for the self-efficacy and interest students reported at the beginning of the semester. We also found significant gender differences in students’ momentary experiences, however, these were reduced substantially when students’ self-efficacy was added to the model, suggesting that gender gaps could instead be self-efficacy gaps. These results suggest that students’ momentary experiences in CS1, how they experience the course week to week, have an impact on their longer-term interest and learning outcomes. Furthermore, we found that male and female students reported different experiences, suggesting that improving the CS1 experiences that students have could help to close gender-related participation gaps. In all, this study shows that the granular experiences students have in CS1 matter for key outcomes of interest to computing education researchers and educators and that the experience sampling method, more common in fields adjacent to computer science education, provides one way for researchers to integrate the experiences students have into our accounts of why students become interested in computing.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3159450.3159452
