skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Friday, July 12 until 2:00 AM ET on Saturday, July 13 due to maintenance. We apologize for the inconvenience.


Title: Predicting Team Function Using Bayesian and Cognitive Diagnostic Modeling Approaches
Team-based learning is commonly used in engineering introductory courses. As students of a team may be from vastly different backgrounds, academically and non-academically, it is important for faculty members to know what aid or hinder team success. The dataset that is used in this paper includes student personality inputs, self-and-peer-assessments of teamwork, and perceptions of teamwork outcomes. Using this information, we developed several bayesian models that are able to predict if a team is working well. We also constructed and estimated Q-matrices which are crucial in explaining the relationship between latent traits and students’ characteristics in cognitive diagnostic models. The prediction and diagnostic models are able to help faculty members and instructors to gain insights into finding ways to separate students into teams more effectively so that students have a positive team-based learning experience.  more » « less
Award ID(s):
2120252
NSF-PAR ID:
10446598
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
American Society for Engineering Education
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    Effectively facilitating teamwork experiences, particularly in the context of large-size courses, is difficult to implement. This study seeks to address the challenges of implementing effective teamwork experiences in large courses. This study integrated teamwork pedagogy to facilitate a semester-long project in the context of a large-size class comprising 118 students organized into 26 teams. The data for this study were collected from two online teamwork sessions when teams collaborated and self-recorded during the in-class time. The video recordings were qualitatively analyzed to identify patterns in team dynamics processes through visualizations. The study aims to provide insights into the different ways team members engaged in team dynamics processes during different phases of the semester.

    Results

    Findings suggest that members of teams were mostly active and passive during meetings and less constructive and interactive in their engagement. Team members mainly engaged in communication, team orientation, and feedback behaviors. Over time, team members' interactions with one another remained about the same, with feedback behaviors tending to diminish and coordination behaviors staying about the same or slightly increasing over time.

    Conclusion

    The implications of this study extend to both practice and theory. Practically, combining cooperative learning and scrum practices enabled a blend of collaborative and cooperative work, which suggests providing teams with tools and structures to coordinate teamwork processes and promote interaction among team members. From a theoretical perspective, this study contributes to the understanding of temporal aspects of teamwork dynamics by examining how team interactions evolve during working sessions at different points in time. Overall, this research provides valuable insights for educators, practitioners, and researchers aiming to enhance teamwork experiences in large courses, particularly in software development disciplines.

     
    more » « less
  2. The purpose of this research paper is to explore whether participation in an interdisciplinary collaboration program partnering Preservice Teachers (PST) and Undergraduate Engineering Students (UES) results in an increase in teamwork effectiveness. The interdisciplinary collaboration was designed as a service-learning project within existing undergraduate programs that included the development and delivery of engineering content to a K-12 audience. The collaborations were integrated into existing courses in two colleges, engineering and education. The Behaviorally Anchored Rating Scale (BARS) version of the Comprehensive Assessment of Team Member Effectiveness (CATME) was used midway and at the end of the project to evaluate teamwork effectiveness. Results of the analysis indicated that both PST and UES experienced a significant increase in team-member effectiveness over the course of the project in four of the five factors: interacting with team members, keeping the team on track, expecting quality, and having relevant knowledge, skills and abilities. A noticeable positive increase in student attitudes towards the task was also observed between the midway and the end of the project. Analysis also suggests that the gain in the teamwork effectiveness did not differ across majors, with both UES and PST showing similar gains. Findings from this study provide some preliminary evidence that an innovative interdisciplinary service learning experience partnering engineering and education students, had a positive impact on their teamwork skills. 
    more » « less
  3. Jeff Nichols (Ed.)
    Instructors using algorithmic team formation tools must decide which criteria (e.g., skills, demographics, etc.) to use to group students into teams based on their teamwork goals, and have many possible sources from which to draw these configurations (e.g., the literature, other faculty, their students, etc.). However, tools offer considerable flexibility and selecting ineffective configurations can lead to teams that do not collaborate successfully. Due to such tools’ relative novelty, there is currently little knowledge of how instructors choose which of these sources to utilize, how they relate different criteria to their goals for the planned teamwork, or how they determine if their configuration or the generated teams are successful. To close this gap, we conducted a survey (N=77) and interview (N=21) study of instructors using CATME Team-Maker and other criteria-based processes to investigate instructors’ goals and decisions when using team formation tools. The results showed that instructors prioritized students learning to work with diverse teammates and performed “sanity checks” on their formation approach’s output to ensure that the generated teams would support this goal, especially focusing on criteria like gender and race. However, they sometimes struggled to relate their educational goals to specific settings in the tool. In general, they also did not solicit any input from students when configuring the tool, despite acknowledging that this information might be useful. By opening the “black box” of the algorithm to students, more learner-centered approaches to forming teams could therefore be a promising way to provide more support to instructors configuring algorithmic tools while at the same time supporting student agency and learning about teamwork. 
    more » « less
  4. Team- and project-based pedagogies are increasingly normative in engineering education and beyond. Student teamwork holds the promise of developing collaborative skills deemed essential for new engineers by professional accreditation bodies such as ABET. The emphasis on these models, furthermore, reflects developments in pedagogical theory, stressing the importance of experiential learning and the social construction of knowledge, repositioning the instructor as a facilitator and guide. Teamwork in an educational context differs from that in professional contexts in that learning outcomes for all team members – both in terms of technical knowledge and team-working skills – are a primary goal of the activity, even while more tangible task-related outcomes might be the main concern of the students themselves. However, team-based learning also holds the potential for team members to have negative experiences, of which instructors may have little or no awareness, especially in real-time. Teams may achieve team-level outcomes required for successful completion, in spite of uneven levels of participation and contribution. Reduced participation on the part of an individual team member may have many causes, pro-active or reactive: it may be a deliberate refusal to engage, a lack of self-confidence, or a response to hostility from other members, among other possibilities. Inequitable team interactions will lead to uneven uptake of desired learning outcomes. Fostering equity in interactions and identifying inequitable practices among team members is therefore an important part of implementing team-based pedagogies, and an essential first step in identifying and challenging systematic patterns of inequity with regard to members of historically marginalized groups. This paper will therefore explore ways in which equity in group decision-making may be conceptualized and observed, laying the foundations for identifying and addressing inequities in the student experience. It will begin by considering different potential manifestations of interactional equity, surveying notions derived from prior education research in the fields of health, mathematics, engineering, and the natural sciences. These notions include: equity of participation on the basis of quantified vocal contributions (in terms of words, utterances, or clausal units); distribution and evolution of interactional roles; equity of idea endorsement and uptake; distribution of inchargeness and influence; equity of access to positional identities and discourse practices; and team member citizenship. In the paper’s empirical component, we trial measures of equity taken or developed from this literature on a small dataset of transcripts showing verbal interactions between undergraduate student team members in a first-year engineering design course. Some measures will be qualitative and others quantitative, depending on the particular form and manifestation of equity they are designed to examine. Measures include manual coding of speech acts and interactional ‘bids’, statistical measures of utterance frequency and length, and computational approaches to modeling interactional features such as social impact and receptivity. Results are compared with the students’ own reflections on the interactions, taken immediately afterward. Recommendations are made for the application of the measures, both from research and practice perspectives. Keywords: Teamwork, Equity, Interaction, Design 
    more » « less
  5. There is growing evidence of the effectiveness of project-based learning (PBL) in preparing students to solve complex problems. In PBL implementations in engineering, students are treated as professional engineers facing projects centered around real-world problems, including the complexity and uncertainty that influence such problems. Not only does this help students to analyze and solve an authentic real-world task, promoting critical thinking, but also students learn from each other, learning valuable communication and teamwork skills. Faculty play an important part by assuming non-conventional roles (e.g., client, senior professional engineer, consultant) to help students throughout this instructional and learning approach. Typically in PBLs, students work on projects over extended periods of time that culminate in realistic products or presentations. In order to be successful, students need to learn how to frame a problem, identify stakeholders and their requirements, design and select concepts, test them, and so on. Two different implementations of PBL projects in a fluid mechanics course are presented in this paper. This required, junior-level course has been taught since 2014 by the same instructor. The first PBL project presented is a complete design of pumped pipeline systems for a hypothetical plant. In the second project, engineering students partnered with pre-service teachers to design and teach an elementary school lesson on fluid mechanics concepts. With the PBL implementations, it is expected that students: 1) engage in a deeper learning process where concepts can be reemphasized, and students can realize applicability; 2) develop and practice teamwork skills; 3) learn and practice how to communicate effectively to peers and to those from other fields; and 4) increase their confidence working on open-ended situations and problems. The goal of this paper is to present the experiences of the authors with both PBL implementations. It explains how the projects were scaffolded through the entire semester, including how the sequence of course content was modified, how team dynamics were monitored, the faculty roles, and the end products and presentations. Students' experiences are also presented. To evaluate and compare students’ learning and satisfaction with the team experience between the two PBL implementations, a shortened version of the NCEES FE exam and the Comprehensive Assessment of Team Member Effectiveness (CATME) survey were utilized. Students completed the FE exam during the first week and then again during the last week of the semester in order to assess students’ growth in fluid mechanics knowledge. The CATME survey was completed mid-semester to help faculty identify and address problems within team dynamics, and at the end of the semester to evaluate individual students’ teamwork performance. The results showed that no major differences were observed in terms of the learned fluid mechanics content, however, the data showed interesting preliminary observations regarding teamwork satisfaction. Through reflective assignments (e.g., short answer reflections, focus groups), student perceptions of the PBL implementations are discussed in the paper. Finally, some of the challenges and lessons learned from implementing both projects multiple times, as well as access to some of the PBL course materials and assignments will be provided. 
    more » « less