More Like this

1. Who Uses Office Hours? A Comparison of In-Person and Virtual Office Hours Utilization

   Gao, Zhikai ; Heckman, Sarah ; Lynch, Collin ( March 2022 , Proceedings of the 53rd ACM Technical Symposium on Computer Science Education)

   Merkle, Larry ; Doyle, Maureen ; Sheard, Judithe ; Soh, Leen-Kiat ; Dorn, Brian (Ed.)

   In Computer Science (CS) education, instructors use office hours for one-on-one help-seeking. Prior work has shown that traditional in-person office hours may be underutilized. In response many instructors are adding or transitioning to virtual office hours. Our research focuses on comparing in-person and online office hours to investigate differences between performance, interaction time, and the characteristics of the students who utilize in-person and virtual office hours. We analyze a rich dataset covering two semesters of a CS2 course which used in-person office hours in Fall 2019 and virtual office hours in Fall 2020. Our data covers students' use of office hours, the nature of their questions, and the time spent receiving help as well as demographic and attitude data. Our results show no relationship between student's attendance in office hours and class performance. However we found that female students attended office hours more frequently, as did students with a fixed mindset in computing, and those with weaker skills in transferring theory to practice. We also found that students with low confidence in or low enjoyment toward CS were more active in virtual office hours. Finally, we observed a significant correlation between students attending virtual office hours and an increased interest in CS study; while students attending in-person office hours tend to show an increase in their growth mindset. 

   more » « less
2. Too long to wait and not much to do: Modeling student behaviors while waiting for help in online office hours.

   Gao, Zhikai ; Lynch, Collin ; Heckman, Sarah ( July 2023 , Proceedings of the 7th Educational Data Mining in Computer Science Education (CSEDM) Workshop)

   Akram, Bita ; Shi, Yang ; Brusilovsky, Peter ; I-han Hsiao, Sharon ; Leinonen, Juho (Ed.)

   Promptly addressing students’ help requests on their programming assignments has become more and more challenging in computer science education. Since the pandemic, most instructors use online office hours to answer questions. Prior studies have shown increased student participation with online office hours. This popularity has led to significantly longer wait times in the office hours queue, and various strategies for selecting the next student to help may impact wait time. For example, prioritizing students who have not been seen on the day of the deadline will extend the wait time for students who are frequently rejoining the queue. To better understand this problem, we explored students’ behavior when they are waiting in the queue. We investigate the amount of time students are willing to wait in the queue by modeling the distribution of cancellation time. We find that after waiting for 49 minutes, most students will cancel their help request. Then, we looked at students’ coding actions during the waiting period and found that only 21% of students have commits while waiting. Surprisingly, students who waited for hours did not commit their work for automated feedback. Our findings suggest that time in the queue should be considered in addition to other factors like last interaction when selecting the next student to help during office hours to minimize canceled interactions. 

   more » « less
3. What Drives Students to Office Hours: Individual Differences and Similarities

   https://doi.org/10.1145/3545945.3569777  

   Ko, Shao-Heng ; Stephens-Martinez, Kristin ( March 2023 , 54th ACM Technical Symposium on Computer Science Education)

   Undergraduate teaching assistants (UTAs) office hours are an approachable way for students to get help, but little is known about why and for what do the students choose to attend office hours. We sought to understand what kind of help the students believe they need by analyzing the problem-solving step students self-reported when joining the office hours queue app. We used the UPIC framework to aggregate course specific problem-solving steps to enable comparing between seven data sets from a CS1 and a data science course across four semesters. We then compared the class-level and student-level phase distributions to understand the differences between the two courses and the two levels in the courses. We found most students have a "primary phase" where a majority of their interactions fall, and there are significant individual differences in their phase distributions. Moreover, we did not find either students' demographics or the context of their first visits to significantly impact their individual differences in the phase distributions, suggesting students may have fixed beliefs on how to approach office hours. Finally, a strong majority of interactions happen within 3 days of the deadline, such that the UPIC distribution for those days looks like the class-level phase distribution. 

   more » « less
4. You asked, now what? Modeling Students’ Help-Seeking and Coding actions from Re- quest to Resolution

   Zhikai Gao ; Bradley Erickson ; Yiqiao Xu ; Collin Lynch ; Sarah Heckman ; Tiffany Barnes ( December 2022 , Journal of educational data mining)

   Nigel Bosch ; Antonija Mitrovic ; Agathe Merceron (Ed.)

   Demand for education in Computer Science has increased markedly in recent years. With increased demand has come to an increased need for student support, especially for courses with large programming projects. Instructors commonly provide online post forums or office hours to address this massive demand for help requests. Identifying what types of questions students are asking in those interactions and what triggers their help requests can in turn assist instructors in better managing limited help-providing resources. In this study, we aim to explore students’ help-seeking actions from the two separate approaches we mentioned before and investigate their coding actions before help requests to understand better what motivates students to seek help in programming projects. We collected students’ help request data and commit logs from two Fall offerings of a CS2 course. In our analysis, we first believe that different types of questions should be related to different behavioral patterns. Therefore, we first categorized students’ help requests based on their content (e.g., Implementation, General Debugging, or Addressing Teaching Staff (TS) Test Failures). We found that General Debugging is the most frequently asked question. Then we analyzed how the popularity of each type of request changed over time. Our results suggest that implementation is more popular in the early stage of the project cycle, and it changes to General Debugging and Addressing TS Failures in the later stage. We also calculated the accuracy of students’ commit frequency one hour before their help requests; the results show that before Implementation requests, the commit frequency is significantly lower, and before TS failure requests, the frequency is significantly higher. Moreover, we checked before any help request whether students changed their source code or test code. The results show implementation requests related to higher chances of source code changes and coverage questions related to more test code changes. Moreover, we use a Markov Chain model to show students’ action sequences before, during, and after the requests. And finally, we explored students’ progress after the office hours interaction and found that over half of the students improved the correctness of their code after 20 minutes of their office hours interaction addressing TS failures ends. 

   more » « less
5. Student Perceptions of the Complete Online Transition of Two CS Courses in Response to the COVID-19 Pandemic

   Farghally, M. F. ( July 2021 , 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference)

   null (Ed.)

   Student perceptions of the complete online transition of two CS courses in response to the COVID-19 pandemic Due to the COVID-19 pandemic, universities across the globe switched from traditional Face-to-Face (F2F) course delivery to completely online. Our university declared during our Spring break that students would not return to campus, and that all courses must be delivered fully online starting two weeks later. This was challenging to both students and instructors. In this evidence-based practice paper, we present results of end-of-semester student surveys from two Spring 2020 CS courses: a programming intensive CS2 course, and a senior theory course in Formal Languages and Automata (FLA). Students indicated course components they perceived as most beneficial to their learning, before and then after the online transition, and preferences for each regarding online vs. F2F. By comparing student reactions across courses, we gain insights on which components are easily adapted to online delivery, and which require further innovation. COVID was unfortunate, but gave a rare opportunity to compare students’ reflections on F2F instruction with online instructional materials for half a semester vs. entirely online delivery of the same course during the second half. The circumstances are unique, but we were able to acquire insights for future instruction. Some course components were perceived to be more useful either before or after the transition, and preferences were not the same in the two courses, possibly due to differences in the courses. Students in both courses found prerecorded asynchronous lectures significantly less useful than in-person lectures. For CS2, online office hours were significantly less useful than in-person office hours, but we found no significant difference in FLA. CS2 students felt less supported by their instructor after the online transition, but no significant difference was indicated by FLA students. FLA students found unproctored online exams offered through Canvas more stressful than in-person proctored exams, but the opposite was indicated by CS2 students. CS2 students indicated that visual materials from an eTextbook were more useful to them after going online than before, but FLA students indicated no significant difference. Overall, students in FLA significantly preferred the traditional F2F version of the course, while no significant difference was detected for CS2 students. We did not find significant effects from gender on the preference of one mode over the other. A serendipitous outcome was learning that some changes forced by circumstance should be considered for long term adoption. Offering online lab sessions and online exams where the questions are primarily multiple choice are possible candidates. However, we found that students need to feel the presence of their instructor to feel properly supported. To determine what course components need further improvement before transitioning to fully online mode, we computed a logistic regression model. The dependent variable is the student's preference for F2F or fully online. The independent variables are the course components before and after the online transition. For both courses, in-person lectures were a significant factor negatively affecting students' preferences of the fully online mode. Similarly, for CS2, in-person labs and in-person office hours were significant factors pushing students’ preferences toward F2F mode. 

   more » « less