More Like this

1. You asked, now what? Modeling Students’ Help-Seeking and Coding actions from Re- quest to Resolution

   Zhikai Gao ; Bradley Erickson ; Yiqiao Xu ; Collin Lynch ; Sarah Heckman ; Tiffany Barnes ( December 2022 , Journal of educational data mining)

   Nigel Bosch ; Antonija Mitrovic ; Agathe Merceron (Ed.)

   Demand for education in Computer Science has increased markedly in recent years. With increased demand has come to an increased need for student support, especially for courses with large programming projects. Instructors commonly provide online post forums or office hours to address this massive demand for help requests. Identifying what types of questions students are asking in those interactions and what triggers their help requests can in turn assist instructors in better managing limited help-providing resources. In this study, we aim to explore students’ help-seeking actions from the two separate approaches we mentioned before and investigate their coding actions before help requests to understand better what motivates students to seek help in programming projects. We collected students’ help request data and commit logs from two Fall offerings of a CS2 course. In our analysis, we first believe that different types of questions should be related to different behavioral patterns. Therefore, we first categorized students’ help requests based on their content (e.g., Implementation, General Debugging, or Addressing Teaching Staff (TS) Test Failures). We found that General Debugging is the most frequently asked question. Then we analyzed how the popularity of each type of request changed over time. Our results suggest that implementation is more popular in the early stage of the project cycle, and it changes to General Debugging and Addressing TS Failures in the later stage. We also calculated the accuracy of students’ commit frequency one hour before their help requests; the results show that before Implementation requests, the commit frequency is significantly lower, and before TS failure requests, the frequency is significantly higher. Moreover, we checked before any help request whether students changed their source code or test code. The results show implementation requests related to higher chances of source code changes and coverage questions related to more test code changes. Moreover, we use a Markov Chain model to show students’ action sequences before, during, and after the requests. And finally, we explored students’ progress after the office hours interaction and found that over half of the students improved the correctness of their code after 20 minutes of their office hours interaction addressing TS failures ends. 

   more » « less
2. The concerns and perceived challenges students faced when traditional in-person engineering courses suddenly transitioned to remote learning

   Yu, Fan ; Milord, Johanna ; Orton, Sarah L. ; Flores, Lisa. Y. ; & Marra, Rose. M. ( June 2022 , 2022 ASEE Annual Conference)

   This research evaluates the impact of switching college engineering courses from in-person instruction to emergency remote learning among engineering students at a university in the Midwest. The study aimed to answer the question: What were the concerns and perceived challenges students faced when traditional in-person engineering courses suddenly transitioned to remote learning? The goal of this study is to uncover the challenges students were facing in engineering online courses and to understand students’ concerns. Our findings can help improve teaching instruction to provide students with previously unavailable educational assistance for online engineering courses. We collected online survey responses during weeks 8 and 9 of the academic semester, shortly after the COVID-19 shutdown and emergency transition to remote learning in Spring 2020. The survey included two open-ended questions which inquired about students’ feedback about moving the class online, and one two-item scale which assessed students’ confidence in online engineering learning. Data analysis for the open-ended questions was guided by the theoretical framework - Social Cognitive Career Theory [1] that explores how context, person factors and social cognitions contribute to career goals, interests and actions. A phenomenological approach [2] was conducted to understand the experience of these students. Open coding and axial coding [2] methods were used to create initial categories then themes related to students' concerns and challenges. Data from the two-item scale was evaluated using descriptive statistics: means, standard deviations, and ranges. Four main themes with separate sub-categories emerged from the student responses: 1) Instructor’s ability to teach course online (Instructional limitations, Seeking help, Increased Workload), 2) Student’s ability to learn online (Time Management, Lower engagement and motivation, Harder to absorb material, Hard to focus, Worry about performance), 3) Difficulties outside of class (Technology issues), and 4) No concerns. Students seemed more concerned about their ability to learn the material (48% of responses) than the instructor’s ability to teach the material (36% of responses). The instructional limitations or lack of instructional support (22% of responses) and time management (12% of responses) were among the major concerns in the sub-categories. The results from two-item scale indicated participants' s confidence in their ability to master their classroom knowledge was at an intermediate level via online instruction (6/10), and participants' confidence in the instructor's ability to teach knowledge in online classes is moderate to high (7/10). The results align with the open-ended question response in which students were somewhat more concerned about their ability to learn than the instructor’s ability to teach. The themes and analysis will be a valuable tool to help institutions and instructors improve student learning experiences. 

   more » « less
3. Exploring the Evolution of Engineering Students™ Feelings of Inclusion in Their College and the Broader Scientific Community

   Melissa Lynn Morris, University of ; Joseph Dygert, West Virginia ; Robin A.M Hensel, West Virginia ( July 2021 , 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference)

   null (Ed.)

   This complete research paper discusses how students’ feelings of inclusion change throughout their undergraduate career. Student responses acquired through focus groups and one-on-one interviews were examined to determine how included the students felt in their engineering college and also the broader scientific community. A small group of non-calculus ready engineering students enrolled in a large land grant institution in the Mid-Atlantic region consented to participate in the study. The student cohort participated in an NSF S-STEM funded program aimed at fostering a sense of inclusion in engineering by implementing a curriculum focused on cohort formation, career exploration, and professional development. The AcES, consisting of a weeklong pre-fall bridge experience, two common courses, and a variety of co-curricular activities, has been operating for eight years. Students who receive S-STEM funded scholarships participate in three focus groups and two one-on-one interviews each semester throughout their undergraduate studies. Student responses from the one-on-one interviews and focus groups conducted from 2017-2020 were examined with qualitative coding methods. Questions examined in this work include: 1) Did the engineering in history course help make you feel like you belong in engineering at WVU and that you are included in engineering at WVU?, 2) Do you feel part of the group when working on projects in your engineering courses?, 3) Do you consider yourself a member of the scientific and engineering community here at WVU? Why or why not, and 4) Do you consider yourself a member of the broader scientific and engineering community? During the exploratory coding phase three codes were established to represent the degree of inclusion felt by students: Edge of Inclusion, Slight Inclusion, and Feelings of Inclusion. Edge of Inclusion was characterized by student responses such as “almost there but not totally”, “just starting to be”, and “no, well maybe a bit” while student responses such as “yes, but only a little” and “in some classes or situations” were recognized as Slight Inclusion. Examples of student responses such as “yes, I do feel part of it”, “absolutely, since I’ve…”, and “I would consider myself part of . . .” were classified with the code Feelings of Inclusion. Since the sample size was limited by scholarship funding, statistically significant results weren’t obtainable, but clear themes emerged that can be used to influence engineering curricula and serve as justification for an expanded study. Participating in an internship emerged as a major contributor to students feeling included in the broader scientific community. Interestingly, a decrease in the average degree of inclusion occurred after the students’ first semester, prior to increasing in later semesters. It is hypothesized that the emphasis on cohort formation, career exploration, and planned co-curricular activities during the first semester in the AcES program bolstered the initial feelings of inclusion. A student’s feeling of inclusion is known to be a contributing factor in retention. The findings of this research indicate that internships should not only be strongly encouraged, but university resources should be invested in helping students be prepared for, apply to, and obtain internships. The researchers suggest the study be expanded beyond the AcES program to examine a broader sample and greater number of students. 

   more » « less
4. Characterizing Student Development Progress: Validating Student Adherence to Project Milestones

   Erickson, Bradley ; Heckman, Sarah ; Lynch, Collin F. ( March 2022 , Proceedings of the 53rd ACM Technical Symposium on Com- puter Science Education)

   Merkle, Larry ; Doyle, Maureen ; Sheard, Judithe ; Soh, Leen-Kiat ; Dorn, Brian (Ed.)

   As enrollment in CS programs have risen, it has become increasingly difficult for teaching staff to provide timely and detailed guidance on student projects. To address this, instructors use automated assessment tools to evaluate students’ code and processes as they work. Even with automation, understanding students’ progress, and more importantly, if students are making the ‘right’ progress toward the solution is challenging at scale. To help students manage their time and learn good software engineering processes, instructors may create intermediate deadlines, or milestones, to support progress. However, student’s adherence to these processes is opaque and may hinder student success and instructional support. Better understanding of how students follow process guidance in practice is needed to identify the right assignment structures to support development of high-quality process skills. We use data collected from an automated assessment tool, to calculate a set of 15 progress indicators to investigate which types of progress are being made during four stages of two projects in a CS2 course. These stages are split up by milestones to help guide student activities. We show how looking at which progress indicators are triggered significantly more or less during each stage validates whether students are adhering to the goals of each milestone. We also find students trigger some progress indicators earlier on the second project suggesting improving processes over time. 

   more » « less
5. Gradual Soundness: Lessons from Static Python

   https://doi.org/10.22152/programming-journal.org/2023/7/2  

   Lu, Kuang-Chen ; Greenman, Ben ; Meyer, Carl ; Viehland, Dino ; Panse, Aniket ; Krishnamurthi, Shriram ( June 2022 , The Art, Science, and Engineering of Programming)

   Context: Gradually-typed languages allow typed and untyped code to interoperate, but typically come with significant drawbacks. In some languages, the types are unreliable; in others, communication across type boundaries can be extremely expensive; and still others allow only limited forms of interoperability. The research community is actively seeking a sound, fast, and expressive approach to gradual typing. Inquiry: This paper describes Static Python, a language developed by engineers at Instagram that has proven itself sound, fast, and reasonably expressive in production. Static Python’s approach to gradual types is essentially a programmer-tunable combination of the concrete and transient approaches from the literature. Concrete types provide full soundness and low performance overhead, but impose nonlocal constraints. Transient types are sound in a shallow sense and easier to use; they help to bridge the gap between untyped code and typed concrete code. Approach: We evaluate the language in its current state and develop a model that captures the essence of its approach to gradual types. We draw upon personal communication, bug reports, and the Static Python regression test suite to develop this model. Knowledge: Our main finding is that the gradual soundness that arises from a mix of concrete and transient types is an effective way to lower the maintenance cost of the concrete approach. We also find that method-based JIT technology can eliminate the costs of the transient approach. On a more technical level, this paper describes two contributions: a model of Static Python and a performance evaluation of Static Python. The process of formalization found several errors in the implementation, including fatal errors. Grounding: Our model of Static Python is implemented in PLT Redex and tested using property-based soundness tests and 265 tests from the Static Python regression suite. This paper includes a small core of the model to convey the main ideas of the Static Python approach and its soundness. Our performance claims are based on production experience in the Instagram web server. Migrations to Static Python in the server have caused a 3.7\% increase in requests handled per second at maximum CPU load. Importance: Static Python is the first sound gradual language whose piece-meal application to a realistic codebase has consistently improved performance. Other language designers may wish to replicate its approach, especially those who currently maintain unsound gradual languages and are seeking a path to soundness. 

   more » « less