Search for: All records

Tests serve an important role in computing education, measuring achievement and differentiating between learners with varying knowledge. But tests may have flaws that confuse learners or may be too difficult or easy, making test scores less valid and reliable. We analyzed the Second Computer Science 1 (SCS1) concept inventory, a widely used assessment of introductory computer science (CS1) knowledge, for such flaws. The prior validation study of the SCS1 used Classical Test Theory and was unable to determine whether differences in scores were a result of question properties or learner knowledge. We extended this validation by modeling question difficulty and learner knowledge separately with Item Response Theory (IRT) and performing expert review on problematic questions. We found that three questions measured knowledge that was unrelated to the rest of the SCS1, and four questions were too difficult for our sample of 489 undergrads from two universities. 

A primary goal of computing education research is to discover designs that produce better learning of computing. In this pursuit, we have increasingly drawn upon theories from learning science and education research, recognizing the potential benefits of optimizing our search for better designs by leveraging the predictions of general theories of learning. In this paper, we contribute an argument that theory can also inhibit our community's search for better designs. We present three inhibitions: 1) our desire to both advance explanatory theory and advance design splits our attention, which prevents us from excelling at both; 2) our emphasis on applying and refining general theories of learning is done at the expense of domain-specific theories of computer science knowledge, and 3) our use of theory as a critical lens in peer review prevents the publication of designs that may accelerate design progress. We present several recommendations for how to improve our use of theory, viewing it as just one of many sources of design insight in pursuit of improving learning of computing. 

About half of recent computer and information science graduates attended community college at some point. Prior work on transfer students in general suggests that the transfer process can engage people from underrepresented communities, but can also be academically and socially "shocking". However, we know little about the experiences of transfer students in computer science in particular. We used the Laanan-Transfer Student Questionnaire (L-TSQ) to survey 25 transfer students and 135 native (non-transfer) students and conducted follow-up interviews with 8 transfer students attending a large public 4-year university in a city with significant technology industry presence. We found that while transfer students were more diverse demographically, the support of the university for transfer student orientation tended to mitigate social shocks of transferring. This did not, however, eliminate gaps in academic performance. These findings suggest that there are other non-social factors that influence academic performance that CS programs must support to equitably engage students who transfer. 

We propose and evaluate a lightweight strategy for tracing code that can be efficiently taught to novice programmers, building off of recent findings on "sketching" when tracing. This strategy helps novices apply the syntactic and semantic knowledge they are learning by encouraging line-by-line tracing and providing an external representation of memory for them to update. To evaluate the effect of teaching this strategy, we conducted a block-randomized experiment with 24 novices enrolled in a university-level CS1 course. We spent only 5-10 minutes introducing the strategy to the experimental condition. We then asked both conditions to think-aloud as they predicted the output of short programs. Students using this strategy scored on average 15% higher than students in the control group for the tracing problems used the study (p<0.05). Qualitative analysis of think-aloud and interview data showed that tracing systematically (line-by-line and "sketching" intermediate values) led to better performance and that the strategy scaffolded and encouraged systematic tracing. Students who learned the strategy also scored on average 7% higher on the course midterm. These findings suggest that in <1 hour and without computer-based tools, we can improve CS1 students' tracing abilities by explicitly teaching a strategy. 

Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene. In this paper, we examine the feasibility of predicting abandonment with machine-learned classifiers. Using interaction logs from an online programming game, we extracted a collection of features that are potentially related to learner abandonment and engagement, then developed classifiers for each level. Across the first five levels of the game, our classifiers successfully predicted 61% to 76% of learners who did not complete the next level, achieving an average AUC of 0.68. In these classifiers, features negatively associated with abandonment included account activation and help-seeking behaviors, whereas features positively associated with abandonment included features indicating difficulty and disengagement. These findings highlight the feasibility of providing timely intervention to learners likely to quit. 

Influencing adolescent interest in computing is key to engaging diverse teens in computer science learning. Prior work suggests that informal mentorship may be a powerful way to trigger and maintain interest in computing, but we still know little about how mentoring relationships form, how mentors trigger and maintain interest, or what qualities adolescents value in informal mentors. In a 3-week career exploration class with 18 teens from underrepresented groups, we had students write extensively about their informal computing mentors. In analyzing their writing, we found that most teens had informal computing mentors, that mentors were typically teachers, friends, and older siblings (and not parents or school counselors), and that what teens desired most were informal mentors that were patient, helpful, inspiring, and knowledgeable. These findings suggest that computing mentors can come in many forms, that they must be patient, helpful, and inspiring, but that they also require content knowledge about computing to be meaningful. Future work might explore what knowledge of computing is sufficient to empower teachers, parents, peers, and family to be effective computing mentors. 

Coding bootcamps are a new and understudied way of training new software developers. To learn about the barriers bootcamp students face, we interviewed twenty-six coding bootcamp students and analyzed the interviews using the Communities of Practice framework. We found that bootcamps can be part of an alternate path into the software industry and they provided a second chance for those who missed computing education opportunities earlier, particularly for women. While bootcamps represented a second chance, students entering the industry through bootcamps faced great personal costs and risks, often including significant time, money and effort spent before, during, and after their bootcamps. Though the coursework of bootcamps only ranged from three to six months, career change could take students a year or more, with some students even attending sections of multiple bootcamps. 

Prior work on adolescent interest development shows that mentorship can promote interest in a subject while reshaping beliefs about the subject. To what extent do these same effects occur in computing, where interest and beliefs have traditionally been negative? We conducted two studies of the Puget Sound region in the United States, surveying and teaching 57 diverse adolescents with interests in computing. In the first study, we found that interest in computing was strongly related to having a mentoring relationship and not to gender or socioeconomic status. Teens with mentors also engaged in significantly more computing education and had more diverse beliefs about peers who engaged in computing education. The second study reinforced this finding, showing that teens who took a class from an instructor who aimed to become students' teacher-mentor had significantly greater positive changes in interest in computing than those who already had a mentor. These findings, while correlational, suggest that mentors can play a key role in promoting adolescent interest in computing.