More Like this

1. Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development Projects

   https://doi.org/10.1145/3593592  

   Hundhausen, Christopher ; Conrad, Phill ; Adesope, Olusola ; Tariq, Ahsun ( September 2023 , ACM Transactions on Computing Education)

   Assessing team software development projects is notoriously difficult and typically based on subjective metrics. To help make assessments more rigorous, we conducted an empirical study to explore relationships between subjective metrics based on peer and instructor assessments, and objective metrics based on GitHub and chat data. We studied 23 undergraduate software teams (n= 117 students) from two undergraduate computing courses at two North American research universities. We collected data on teams’ (a) commits and issues from their GitHub code repositories, (b) chat messages from their Slack and Microsoft Teams channels, (c) peer evaluation ratings from the CATME peer evaluation system, and (d) individual assignment grades from the courses. We derived metrics from (a) and (b) to measure both individual team members’contributionsto the team, and theequalityof team members’ contributions. We then performed Pearson analyses to identify correlations among the metrics, peer evaluation ratings, and individual grades. We found significant positive correlations between team members’ GitHub contributions, chat contributions, and peer evaluation ratings. In addition, the equality of teams’ GitHub contributions was positively correlated with teams’ average peer evaluation ratings and negatively correlated with the variance in those ratings. However, no such positive correlations were detected between the equality of teams’ chat contributions and their peer evaluation ratings. Our study extends previous research results by providing evidence that (a) team members’ chat contributions, like their GitHub contributions, are positively correlated with their peer evaluation ratings; (b) team members’ chat contributions are positively correlated with their GitHub contributions; and (c) the equality of team’ GitHub contributions is positively correlated with their peer evaluation ratings. These results lend further support to the idea that combining objective and subjective metrics can make the assessment of team software projects more comprehensive and rigorous. 

   more » « less
2. A Reflection on Action Approach to Teamwork Facilitation

   Jaiswal, A. ; Patel, D. ; Zhu, Y. ; Lee, J. ; & Magana, A.J. ( January 2022 , ASEE annual conference exposition proceedings)

   Working in teams has been recognized as an essential 21st-century skill. Introducing teamwork in the undergraduate classroom is crucial as it allows the students to work with individuals with diverse skillsets and learn from one another. It is important to note that just creating a team and allowing the students to work does not foster teamwork skills. Inculcating teamwork skills requires a consciousness on the part of the instructor and the teaching assistants. Pedagogies such as cooperative learning have been recognized as effective in helping students develop teamwork skills. We introduced a joint reflection on action approach to developing teamwork skills among novice students as part of a sophomore-level systems analysis and design course. In this evidence-based practice paper, we report on students’ reflections regarding their perceptions of teamwork. This study approaches the following research questions: What are students' reflections about the role of communication while working in teams in a cooperative project-based learning environment? The guiding pedagogical framework for this course is cooperative learning. The course requires the students to work in teams in a semester-long software development project. To elicit reflection on action about their teamwork experience. Specifically, we exposed students to concrete experiences as part of their teamwork interactions, which became the basis for observations and reflections. For this, the semester-long project was complemented with one reflection-on-action activity. In the activity, students were asked to watch a video of secrets of successful teamwork and were asked to reflect on their perceptions about the role of communication within teams. The students’ reflections on the activity were analyzed using qualitative inductive thematic analysis to understand the students’ perceptions regarding teamwork and communication within teams. 

   more » « less
3. A Reflection on Action Approach to Teamwork Facilitation

   Jaiswal, A. ; Patel, D. ; Zhu, Y. ; Lee, J. ; Magana, A.J. ( January 2022 , Proceedings ASEE annual conference)

   Working in teams has been recognized as an essential 21st-century skill. Introducing teamwork in the undergraduate classroom is crucial as it allows the students to work with individuals with diverse skillsets and learn from one another. It is important to note that just creating a team and allowing the students to work does not foster teamwork skills. Inculcating teamwork skills requires a consciousness on the part of the instructor and the teaching assistants. Pedagogies such as cooperative learning have been recognized as effective in helping students develop teamwork skills. We introduced a joint reflection on action approach to developing teamwork skills among novice students as part of a sophomore-level systems analysis and design course. In this evidence-based practice paper, we report on students’ reflections regarding their perceptions of teamwork. This study approaches the following research questions: What are students' reflections about the role of communication while working in teams in a cooperative project-based learning environment? The guiding pedagogical framework for this course is cooperative learning. The course requires the students to work in teams in a semester-long software development project. To elicit reflection on action about their teamwork experience. Specifically, we exposed students to concrete experiences as part of their teamwork interactions, which became the basis for observations and reflections. For this, the semester-long project was complemented with one reflection-on-action activity. In the activity, students were asked to watch a video of secrets of successful teamwork and were asked to reflect on their perceptions about the role of communication within teams. The students’ reflections on the activity were analyzed using qualitative inductive thematic analysis to understand the students’ perceptions regarding teamwork and communication within teams. 

   more » « less
4. GitterCom: A Dataset of Open Source Developer Communications in Gitter

   https://doi.org/10.1145/3379597.3387494  

   Parra, Esteban ; Ellis, Ashley ; Haiduc, Sonia ( June 2020 , Proceedings of the 17th International Conference on Mining Software Repositories)

   null (Ed.)

   Team communication is essential for the development of modern software systems. For distributed software development teams, such as those found in many open source projects, this communication usually takes place using electronic tools. Among these, modern chat platforms such as Gitter are becoming the de facto choice for many software projects due to their advanced features geared towards software development and effective team communication. Gitter channels contain numerous messages exchanged by developers regarding the state of the project, issues and features of the system, team logistics, etc. These messages can contain important information to researchers studying open source software systems, developers new to a particular project and trying to get familiar with the software, etc. Therefore, uncovering what developers are communicating about through Gitter is an essential first step towards successfully understanding and leveraging this information. We present a new dataset, called GitterCom, which aims to enable research in this direction and represents the largest manually labeled and curated dataset of Gitter developer messages. The dataset is comprised of 10,000 messages collected from 10 Gitter communities associated with the development of open source software. Each message was manually annotated and verified by two of the authors, capturing the purpose of the communication expressed by the message. While the dataset has not yet been used in any publication, we discuss how it can enable interesting research opportunities. 

   more » « less
5. An Exploration of the Effects of Enterprise Social Media Use for Classroom Teams.

   Cherchiglia, L. ; Van Osch, W. ; Liang, Y. ; Averkiadi, E. ( December 2020 , SIGHCI 2020 Proceedings)

   null (Ed.)

   Cherchiglia et al. Effects of ESM Use for Classroom Teams Proceedings of the Nineteenth Annual Pre-ICIS Workshop on HCI Research in MIS, Virtual Conference, December 12, 2020 1 An Exploration of the Effects of Enterprise Social Media Use for Classroom Teams Leticia Cherchiglia Michigan State University leticia@msu.edu Wietske Van Osch HEC Montreal & Michigan State University wietske.van-osch@hec.ca Yuyang Liang Michigan State University liangyuy@msu.edu Elisavet Averkiadi Michigan State University averkiad@msu.edu ABSTRACT This paper explores the adoption of Microsoft Teams, a group-based Enterprise Social Media (ESM) tool, in the context of a hybrid Information Technology Management undergraduate course from a large midwestern university. With the primary goal of providing insights into the use and design of tools for group-based educational settings, we constructed a model to reflect our expectations that core ESM affordances would enhance students’ perceptions of Microsoft Teams’ functionality and efficiency, which in turn would increase both students’ perceptions of group productivity and students’ actual usage of Microsoft Teams for communication purposes. In our model we used three core ESM affordances from Treem and Leonardi (2013), namely editability (i.e., information can be created and/or edited after creation, usually in a collaborative fashion), persistence (i.e., information is stored permanently), and visibility (i.e., information is visible to other users). Analysis of quantitative (surveys, server-side; N=62) and qualitative (interviews; N=7) data led to intriguing results. It seems that although students considered that editability, persistency, and visibility affordances within Microsoft Teams were convenient functions of this ESM, problems when working collaboratively (such as connectivity, formatting, and searching glitches) might have prevented considerations of this ESM as fast and user-friendly (i.e., efficient). Moreover, although perceived functionality and efficiency were positively connected to group productivity, hidden/non-intuitive communication features within this ESM might help explain the surprising negative connection between efficiency and usage of this ESM for the purpose of group communication. Another explanation is that, given the plethora of competing tools specifically designed to afford seamless/optimal team communication, students preferred to use more familiar tools or tools perceived as more efficient for group communication than Microsoft Teams, a finding consistent with findings in organizational settings (Van Osch, Steinfield, and Balogh, 2015). Beyond theoretical contributions related to the impact that ESM affordances have on users’ interaction perceptions, and the impact of users’ interaction perceptions on team and system outcomes, from a strategic and practical point of view, our findings revealed several challenges for the use of Microsoft Teams (and perhaps ESM at large) in educational settings: 1) As the demand for online education grows, collaborative tools such as Microsoft Teams should strive to provide seamless experiences for multiple-user access to files and messages; 2) Microsoft Teams should improve its visual design in order to increase ease of use, user familiarity, and intuitiveness; 3) Microsoft Teams appears to have a high-learning curve, partially related to the fact that some features are hidden or take extra steps/clicks to be accessed, thus undermining their use; 4) Team communication is a complex topic which should be further studied because, given the choice, students will fall upon familiar tools therefore undermining the full potential for team collaboration through the ESM. We expect that this paper can provide insights for educators faced with the choice for an ESM tool best-suited for group-based classroom settings, as well as designers interested in adapting ESMs to educational contexts, which is a promising avenue for market expansion. 

   more » « less