skip to main content


Title: An Examination of the Correlation of Exploring Computer Science Course Performance and the Development of Programming Expertise
This study investigated patterns in the development of computational thinking and programming expertise in the context of the Exploring Computer Science (ECS) program, a high school introductory CS course and professional development program designed to foster deep engagement through equitable inquiry around CS concepts. Prior research on programming expertise has identified three general areas of development — program comprehension, program planning, and program generation. The pedagogical practices in ECS are consistent with problem solving approaches that support the development of programming expertise. The study took place in a large urban district during the 2016–17 school year with 28 ECS teachers and 1,931 students. A validated external assessment was used to measure the development of programming expertise. The results indicate that there were medium-sized, statistically significant increases from pretest to posttest, and there were no statistically significant differences by gender or race/ethnicity. After controlling for prior academic achievement, performance in the ECS course correlated with performance on the posttest. With respect to specific programming concepts, the results also provide evidence on the progression of the development of programming expertise. Students seem to develop comprehension and planning expertise prior to expertise in program generation. In addition, students seem to develop expertise with concrete tasks prior to abstract tasks.  more » « less
Award ID(s):
1738691 1543217 1542971 1738776 1738572
NSF-PAR ID:
10088736
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
SIGCSE '19 Proceedings of the 50th ACM Technical Symposium on Computer Science Education
Page Range / eLocation ID:
1067 to 1073
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This study investigated patterns in the development of computational thinking practices in the context of the Exploring Computer Science (ECS) program, a high school introductory CS course and professional development program designed to foster deep engagement through equitable inquiry around CS concepts. Past research indicates that the personal relevance of the ECS experience influences students' expectancy-value towards computer science. Expectancy-value is a construct that is predictive of career choices. We extended our research to examine whether expectancy-value influences the development of computational thinking practices. This study took place in the context of two ECS implementation projects across two states. Twenty teachers, who implemented ECS in 2016–17, participated in the research. There were 906 students who completed beginning and end of year surveys and assessments. The surveys included demographic questions, a validated expectancy-value scale, and questions about students' course experiences. The assessments were developed and validated by SRI International as a companion to the ECS course. Overall, student performance statistically increased from pretest to posttest with effect size of 0.74. There were no statistically significant differences in performance by gender or race/ethnicity. These results are consistent with earlier findings that a personally relevant course experience positively influences students' expectancy for success. These results expanded on prior research by indicating that students' expectancy-value for computer science positively influenced student learning. 
    more » « less
  2. One of the goals of undergraduate education is to prepare students to adapt to a challenging career that requires continual learning and application of knowledge. Working professionals should have deep conceptual knowledge that they can apply in a range of contexts and possess the attitudes and skills of lifelong learners. The literature suggests the concept of Adaptive Expertise (AE), which can be defined as the ability to apply and extend knowledge and skills to new situations, describes some of these characteristics. Survey data concerning the level of AE displayed by various populations is extremely limited in most contexts, be it education or working professionals. As such, data concerning the level of adaptiveness displayed among various groups needs to be measured if activities designed to promote the development of AE are to be created and then tested in terms of their efficacy. This investigation provides this critical baseline data for future studies as we track the AE development of individual, first-year college students through their undergraduate program of study, with a focus on low-income students as a means to support retention. In this work, we assessed adaptive expertise among low-income STEM students using surveys and interviews. Low-income STEM students from various stages of their four year undergraduate program (n=208) completed an adaptive expertise survey in spring 2022. Following the survey, 24 of the low-income students (6 per year, 3 male, 3 female) were selected for targeted qualitative interviews to better understand the differences displayed by low and high AE students. Survey results from prior studies were used to draw comparisons between adaptiveness of low-income and non-low-income students. Results of the AE survey indicated no statistically-significant differences between low-income and non-low-income first-year students in terms of their level of adaptiveness. In addition, the level of AE displayed by low-income students increased through the program in a manner similar to that of non-low-income STEM students. Themes that emerged from the interviews included a general understanding of the importance and likelihood of learning new concepts continually while working in a professional role, and that students expressed growth in understanding the acceptance of reaching out for assistance from other students and faculty after exploring information on their own as they work through challenges in their academic assignments. Two dominant and divergent metacognitive processes were also observed: teaching/explaining concepts to others (highly adaptive) or primarily relying on exam/course grades for feedback on learning (low adaptiveness). Data gathered from interviews demonstrate the need for a greater emphasis on metacognitive practice to promote various aspects of AE. 
    more » « less
  3. Hands-on experiments using the Low-Cost Desktop Learning Modules (LCDLMs) have been implemented in dozens of classrooms to supplement student learning of heat transfer and fluid mechanics concepts with students of varying prior knowledge. The prior knowledge of students who encounter these LCDLMs in the classroom may impact the degree to which students learn from these interactive pedagogies. This paper reports on the differences in student cognitive learning between groups with low and high prior knowledge of the concepts that are tested. Student conceptual test results for venturi, hydraulic loss, and double pipe heat exchanger LCDLMs are analyzed by grouping the student data into two bins based on pre-test score, one for students scoring below 50% and another for those scoring above and comparing the improvement from pretest to posttest between the two groups. The analysis includes data from all implementations of each LCDLM for the 2020-2021 school year. Results from each of the three LCDLMs were analyzed separately to compare student performance on different fluid mechanics or heat exchanger concepts. Then, the overall pre- and posttest scores for all three LCDLMs were analyzed to examine how this interactive pedagogy impacts cognitive gains. Results showed statistically significant differences in improvement between low prior knowledge groups and high prior knowledge groups. Additional findings showed statistically significant results suggesting that the gaps in performance between low prior knowledge and high prior knowledge groups on pre-tests for the LCDLMs were decreased on the posttest. Findings showed that students with lower prior knowledge show a greater overall improvement in cognitive gains than those with higher prior knowledge on all three low-cost desktop learning modules. 
    more » « less
  4. Hands-on experiments using the Low-Cost Desktop Learning Modules (LCDLMs) have been implemented in dozens of classrooms to supplement student learning of heat transfer and fluid mechanics concepts with students of varying prior knowledge. The prior knowledge of students who encounter these LCDLMs in the classroom may impact the degree to which students learn from these interactive pedagogies. This paper reports on the differences in student cognitive learning between groups with low and high prior knowledge of the concepts that are tested. Student conceptual test results for venturi, hydraulic loss, and double pipe heat exchanger LCDLMs are analyzed by grouping the student data into two bins based on pre-test score, one for students scoring below 50% and another for those scoring above and comparing the improvement from pretest to posttest between the two groups. The analysis includes data from all implementations of each LCDLM for the 2020-2021 school year. Results from each of the three LCDLMs were analyzed separately to compare student performance on different fluid mechanics or heat exchanger concepts. Then, the overall pre- and posttest scores for all three LCDLMs were analyzed to examine how this interactive pedagogy impacts cognitive gains. Results showed statistically significant differences in improvement between low prior knowledge groups and high prior knowledge groups. Additional findings showed statistically significant results suggesting that the gaps in performance between low prior knowledge and high prior knowledge groups on pre-tests for the LCDLMs were decreased on the posttest. Findings showed that students with lower prior knowledge show a greater overall improvement in cognitive gains than those with higher prior knowledge on all three low-cost desktop learning modules. 
    more » « less
  5. null (Ed.)
    The AP Computer Science A course and exam continually exhibit inequity among over- and under-represented populations. This paper explored three years of AP CS A data in the Chicago Public School district (CPS) from 2016-2019 (N = 561). We analyzed the impact of teacher and student-level variables to determine the extent AP CS A course taking and exam passing differences existed between over- and under-represented populations. Our analyses suggest four prominent findings: (1) CPS, in collaboration with their Research-Practice Partnership (Chicago Alliance for Equity in Computer Science; CAFÉCS), is broadening participation for students taking the AP CS A course; (2) Over- and under- represented students took the AP CS A exam at statistically comparable rates, suggesting differential encouragement to take or not take the AP CS A exam was not prevalent among these demographics; (3) After adjusting for teacher and student-level prior experience, there were no significant differences among over- and under-represented racial categorizations in their likelihoods to pass the AP CS A exam, albeit Female students were 3.3 times less likely to pass the exam than Males overall; (4) Taking the Exploring Computer Science course before AP CS A predicted students being 3.5 times more likely to pass the AP CS A exam than students that did not take ECS before AP CS A. Implications are discussed around secondary computer science course sequencing and lines of inquiry to encourage even greater broadening of participation in the AP CS A course and passing of the AP CS A exam. 
    more » « less