skip to main content

Explore Research Products in the NSF-PAR

Title: An Approach for Detecting Student Perceptions of the Programming Experience from Interaction Log Data
Student perceptions of programming can impact their experiences in introductory computer science (CS) courses. For example, some students negatively assess their own ability in response to moments that are natural parts of expert practice, such as using online resources or getting syntax errors. Systems that automatically detect these moments from interaction log data could help us study these moments and intervene when the occur. However, while researchers have analyzed programming log data, few systems detect pre-defined moments, particularly those based on student perceptions. We contribute a new approach and system for detecting programming moments that students perceive as important from interaction log data. We conducted retrospective interviews with 41 CS students in which they identified moments that can prompt negative self-assessments. Then we created a qualitative codebook of the behavioral patterns indicative of each moment, and used this knowledge to build an expert system. We evaluated our system with log data collected from an additional 33 CS students. Our results are promising, with F1 scores ranging from 66% to 98%. We believe that this approach can be applied in many domains to understand and detect student perceptions of learning experiences.  more » « less
Award ID(s):
1755628
NSF-PAR ID:
10352595
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
International Conference on Artificial Intelligence in Education
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ( , Proceedings of the 2020 ACM Conference on International Computing Education Research)
    null (Ed.)
    Undergraduate computer science (CS) programs often suffer from high dropout rates. Recent research suggests that self-efficacy -- an individual's belief in their ability to complete a task -- can influence whether students decide to persist in CS. Studies show that students' self-assessments affect their self-efficacy in many domains, and in CS, researchers have found that students frequently assess their programming ability based on their expectations about the programming process. However, we know little about the specific programming experiences that prompt the negative self-assessments that lead to lower self-efficacy. In this paper, we present findings from a survey study with 214 CS1 students from three universities. We used vignette-style questions to describe thirteen programming moments which may prompt negative self-assessments, such as getting syntax errors and spending time planning. We found that many students across all three universities reported that they negatively self-assess at each of the thirteen moments, despite the differences in curriculum and population. Furthermore, those who report more frequent negative self-assessments tend to have lower self-efficacy. Finally, our findings suggest that students' perceptions of professional programming practice may influence their expectations and negative self-assessments. By reducing the frequency that students self-assess negatively while programming, we may be able to improve self-efficacy and decrease dropout rates in CS. 
    more » « less
  2. ; ; ( , 2021 ASEE Virtual Annual Conference Content Access)
    null (Ed.)
    Engineering instructors often use physical manipulatives such as foam beams, rolling cylinders, and large representations of axis systems to demonstrate mechanics concepts and help students visualize systems. Additional benefits are possible when manipulatives are in the hands of individual students or small teams of students who can explore concepts at their own pace and focus on their specific points of confusion. Online learning modalities require new strategies to promote spatial visualization and kinesthetic learning. Potential solutions include creating videos of the activities, using CAD models to demonstrate the principles, programming computer simulations, and providing hands-on manipulatives to students for at-home use. This Work-in-Progress paper discusses our experiences with this last strategy in statics courses two western community colleges and a western four-year university where we supplied students with their own hands-on kits. We have previously reported on the successful implementation of a hands-on statics kit consisting of 3D printed components and standard hardware. The kit was originally designed for use by teams of students during class to engage with topics such as vectors, moments, and rigid body equilibrium. With the onset of the COVID-19 pandemic and shift to online instruction, the first author developed a scaled down version of the kit for at-home use by individual students and modified the associated activity worksheets accordingly. For the community college courses, local students picked up their models at the campus bookstore. We also shipped some of the kits to students who were unable to come to campus, including some in other countries. Due to problems with printing and availability of materials, only 18 kits were available for the class of 34 students at the university implementation. Due to this circumstance, students were placed in teams and asked to work together virtually, one student showing the kit to the other student as they worked through the worksheet prompts. One community college instructor took this approach as well for a limited number of international students who did not receive their kits in a timely manner due to shipping problems. Two instructors assigned the hands-on kits as asynchronous learning activities in their respective online courses, with limited guidance on their use. The third used the kits primarily in synchronous online class meetings. We found that students’ reaction to the models varied by pilot site and presume that implementation differences contributed to this variation. In all cases, student feedback was less positive than it has been for face-to-face courses that used the models from which the take home kit was adapted. Our main conclusion is that implementation matters. Doing hands-on learning in an online course requires some fundamental rethinking about how the learning is structured and scaffolded. 
    more » « less
  3. ; ; ; ; ; ( , ASEE Annual Conference proceedings)
    Barriers to broadening participation in engineering to rural and Appalachian youth include misalignment with family and community values, lack of opportunities, and community misperceptions of engineering. While single interventions are unlikely to stimulate change in these areas, more sustainable interventions that are co-designed with local relevance appear promising. Through our NSF ITEST project, we test the waters of this intervention model through partnership with school systems and engineering industry to implement a series of engineering-themed, standards-aligned lessons for the middle school science classroom. Our mixed methods approach includes collection of interview and survey data from administrators, teachers, engineers, and university affiliates as well as observation and student data from the classroom. We have utilized theory from learning science and organizational collaboration to structure and inform our analysis and explore the impact of our project. The research is guided by the following questions: RQ 1: How do participants conceptualize engineering careers? How and why do such perceptions shift throughout the project? RQ 2: What elements of the targeted intervention affect student motivation towards engineering careers specifically with regard to developing competencies and ability beliefs regarding engineering? RQ 3: How can strategic collaboration between K12 and industry promote a shift in teacher’s conceptions of engineers and increased self-efficacy in building and delivering engineering curriculum? RQ 4: How do stakeholder characteristics, perceptions, and dynamics affect the likelihood of sustainability in strategic collaborations between K12 and industry stakeholders? How do prevailing institutional and collaborative conditions mediate sustainability? In year one, we involved nine 6th grade teachers, three engineering companies, and over 500 students. In year two, we expanded to include 7th grade teachers in our partner schools and the new students moving up to 6th grade. Lessons aligned with students' everyday experiences and connected to industry. For example, students created bouncy balls and tested their effectiveness on materials produced from partner manufacturing facilities. From preliminary analysis of data collected in the first two years of the project (e.g, the Draw an Engineer Test and teacher interviews), we have begun to see evidence of positive student and teacher impact. Additionally, our application of collaborative theory to the investigation of stakeholder perceptions of the project has revealed implications for partnering with school systems and engineering industry. For example, key individuals at each organization may serve as important conduits for program communication and collaborative work. 
    more » « less
  4. ; ; ( , Teachers College Record: The Voice of Scholarship in Education)
    Background: There has been a dearth of research on intersectional identities in STEM, including the fields of computing and engineering. In computing education research, much work has been done on broadening participation, but there has been little investigation into how the field of computer science (CS) presents opportunities for students with strong intersectional identities. This study explores the strengths and connections among the unique identities and the symbiotic relationships that elementary Latina students hold in CS identity attainment. Purpose: The aim of this article is to better understand how predominantly low-income, multilingual Latina students experience identity development through the lens of diverse group membership. We examine how young Latinas, through their participation in a yearlong culturally and linguistically responsive CS curriculum, leverage their intersecting identities to rewrite the formula of what a computer scientist is and can be, leaving space to include and invite other strong identities as well. Research Design: An explanatory sequential mixed-methods design was used that analyzed data from predominantly low-income, multilingual Latinas in upper elementary grades, including pre- and post-CS identity surveys (N = 50) delivered before and after implementation of the curriculum, and eight individual semi-structured student interviews. Findings: We found that Latina students developed significantly stronger identification with the field of CS from the beginning to the end of the school year with regard to their experiences with CS, perception of themselves as computer scientists, family support for CS and school, and friend support for CS and school. Interviews revealed that perception of their CS ability greatly influenced identification with CS and that girls’ self-perceptions stemmed from their school, cultural, and home learning environments. Conclusion: Our results highlight the wealth of resources that Latinas bring to the classroom through their home- and community-based assets, which are characterized by intersecting group membership. Students did not report on the intersection between language and CS identity development, which warrants further investigation. 
    more » « less
  5. ; ; ( , Teachers College record)
    Background: There has been a dearth of research on intersectional identities in STEM, including the fields of computing and engineering. In computing education research, much work has been done on broadening participation, but there has been little investigation into how the field of computer science (CS) presents opportunities for students with strong intersectional identities. This study explores the strengths and connections among the unique identities and the symbiotic relationships that elementary Latina students hold in CS identity attainment. Purpose: The aim of this article is to better understand how predominantly lowincome, multilingual Latina students experience identity development through the lens of diverse group membership. We examine how young Latinas, through their participation in a yearlong culturally and linguistically responsive CS curriculum, leverage their intersecting identities to rewrite the formula of what a computer scientist is and can be, leaving space to include and invite other strong identities as well. Research Design: An explanatory sequential mixed-methods design was used that analyzed data from predominantly low-income, multilingual Latinas in upper elementary grades, including pre- and post-CS identity surveys (N = 50) delivered before and after implementation of the curriculum, and eight individual semistructured student interviews. Findings: We found that Latina students developed significantly stronger identification with the field of CS from the beginning to the end of the school year with regard to their experiences with CS, perception of themselves as computer scientists, family support for CS and school, and friend support for CS and school. Interviews revealed that perception of their CS ability greatly influenced identification with CS and that girls’ self-perceptions stemmed from their school, cultural, and home learning environments. Conclusion: Our results highlight the wealth of resources that Latinas bring to the classroom through their home- and community-based assets, which are characterized by intersecting group membership. Students did not report on the intersection between language and CS identity development, which warrants further investigation. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email