The Department of Chemistry and Biochemistry at St. Mary's College of Maryland has scaffolded collaboration skills throughout the Biochemistry curriculum and developed several assessment tools to evaluate these skills. Biochemistry I and II have used team contracts at the beginning of extensive team projects where students identify their strengths, review expectations, and plan for group communication. At the conclusion of each project, each student assesses their own contributions and team members for various parts of the project. A common collaboration rubric was also applied in Biochemistry I and II as well as in two other courses, General Chemistry II Lab and Physical Chemistry I Lab, for students to evaluate themself and team members using the following subcategories: quality of work, commitment, leadership, communication, and analysis. In Biochemistry I and II, we used this rubric for multiple assignments that are part of the projects in the lecture courses. In the General Chemistry II Lab, we provided elements of this rubric within an evaluation form that reflects these collaboration attributes after each lab experience, so students can assess and report privately on their experiences as part of their collaboration grade for the course. A similar collaboration rubric is completed by students for each team‐based laboratory within Physical Chemistry I. We also demonstrate different ways that instructors can use the data from these assessment tools. In our department, we are using these tools to frame the importance of collaboration skills and collecting data to inform our teaching of these skills. Preliminary data suggest that our curriculum is successfully teaching students how to be good collaborators.
Exploring the process of group-based collaboration: A validation argument for a collaboration model and observation rubric for training explainable machine learning models.
Collaboration is an important learning process. During collaborative learning, students engage in group activities where they converge on goals, solve problems and make joint decisions. To understand the process of collaboration, we focused on how behavior and interaction patterns contribute to the social-relational space of collaboration. We have designed a multilayered conceptual model for the collaboration process and an observation rubric that identifies behaviors and interactions during collaboration that serves as the foundation for machine learning models that can provide behavioral insight into the process of collaboration. This study reports results on several validation studies performed to establish a validation argument for our collaboration conceptual model and collaboration rubric. Through disconfirming evidence, interrater reliability testing, expert reviews, and focus group interviews, we found that our stratified architecture of collaboration and rubric provide valid accounts and descriptions of human behavior and interactions that can be used to substantiate the collaboration process.
more »
« less
- Award ID(s):
- 2016849
- PAR ID:
- 10461795
- Date Published:
- Journal Name:
- Proceedings of the 16th International Conference of the Learning Sciences - ICLS 2022
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning.more » « less
-
Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning.more » « less
-
Large-group (n > 8) co-located collaboration has not been adequately studied because it demands different conceptual framings than those used to study small-group collaboration, while also posing pragmatic constraints on data collection. Working within these pragmatic constraints, we use video data to devise an indicator of the current possibilities for learner collaboration during large-group co-located interactions. We borrow conceptualizations from proxemics and social network analysis to construct collaborative opportunity networks, allowing us to define the concept of collaborative opportunity temperature (COT) readings: a “snapshot” of the current configuration of the different social subgroup structures within a large group, indicating emergent opportunities for collaboration (via talk or shared action) due to proximity. Using a case study of two groups of people (n = 11, n = 12) who interacted with a multi-user museum exhibit, we outline the processes of deriving COT. We show how to quickly detect differences in subgroup configurations that may result from educational interventions and how COT can triangulate with and complement other forms of data (audio transcripts and activity logs) during lengthier analyses. We also outline how COT readings can be used to supply formative feedback on social engagement to learners and be adapted to other learning environments.more » « less
-
Co-creation in academe can take multiple forms. In this research, the co-creation focus is on collaboration between faculty and graduate students to develop educational modules. This activity is designed to improve graduate education and prepare students for conducting graduate research. In previous work presented at ASEE 2022, we discussed benefits and challenges of participating in the co-creation process. This current paper focuses on how we took lessons from our first year and transformed them into a structure to better support interdisciplinary research, collaboration, and community building. We will discuss how we supported the process of co-creation by developing a series of workshops to scaffold student learning. Scaffolds are instructional methods and interventions that are designed to foster skill development by allowing for interactions between what students already know and what they have yet to learn. These workshops were designed using the tenets of the gold standard project-based learning (PjBL). The PjBL framework is itself a scaffold that is designed to build research competencies. Specifically, to introduce a challenging problem or question, we created multiple technical overviews of the cyber-physical system theme of interest that would constitute the eventual educational modules. We scaffolded sustained inquiry by developing a workshop using techniques from the Right Question Institute, and also through a workshop about crafting your message for different audiences. To support the PjBL idea of authenticity, we developed a workshop about core values to help students connect personally to their project topics. To further support collaboration and community building, we developed a workshop to introduce ideas of interdisciplinary collaboration, including developing community agreements and recognizing and responding to microaggressions. Periodic reinforcements of these topics were incorporated as students progressed in their co-creation project. We assessed how students applied these topics through student reflections. Scaffolding students’ learning helped to address co-creation challenges that were expressed by our pilot group, including not understanding the goals of the project and not feeling connected to the research. Observational data of the current groups suggests that students have better understanding of the co-creation process and are collaborating more effectively than our pilot group students, and focus group data confirmed these observations. We also collected feedback from students about the workshops to evaluate what is effective about them and what can be improved. Students felt skills taught in the workshops such as how to prioritize research questions, construct messages for specific audiences, and perform literature searches and reviews, were all effective and useful as they worked on their projects. For improvement, they suggested clearer objectives and more workshops that focus on technical aspects of the project work would be helpful.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
