skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Exploring the process of group-based collaboration: A validation argument for a collaboration model and observation rubric for training explainable machine learning models.
Title: Exploring the process of group-based collaboration: A validation argument for a collaboration model and observation rubric for training explainable machine learning models.
Collaboration is an important learning process. During collaborative learning, students engage in group activities where they converge on goals, solve problems and make joint decisions. To understand the process of collaboration, we focused on how behavior and interaction patterns contribute to the social-relational space of collaboration. We have designed a multilayered conceptual model for the collaboration process and an observation rubric that identifies behaviors and interactions during collaboration that serves as the foundation for machine learning models that can provide behavioral insight into the process of collaboration. This study reports results on several validation studies performed to establish a validation argument for our collaboration conceptual model and collaboration rubric. Through disconfirming evidence, interrater reliability testing, expert reviews, and focus group interviews, we found that our stratified architecture of collaboration and rubric provide valid accounts and descriptions of human behavior and interactions that can be used to substantiate the collaboration process.  more » « less
Award ID(s):
2016849
PAR ID:
10461795
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Editor(s):
Chinn, C.; Tan, E.; Chan, C.; Kali, Y.
Date Published:
Journal Name:
Proceedings of the 16th International Conference of the Learning Sciences - ICLS 2022
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, International Educational Data Mining Society)
    Benjamin, Paaßen; Carrie, Demmans Epp (Ed.)
    Collaborative game-based learning offers opportunities for students to participate in small group learning experiences that foster knowledge sharing, problem solving, and engagement. Student satisfaction with their collaborative experiences plays a pivotal role in shaping positive learning outcomes and is a critical factor in group success during learning. Gauging students申f satisfaction within collaborative learning contexts can offer insights into student engagement and participation levels while affording practitioners the ability to provide targeted interventions or scaffolding. In this paper,we propose a framework for inferring student collaboration satisfaction with multimodal learning analytics from collaborative interactions. Utilizing multimodal data collected from 50 middle school students engaged in collaborative game-based learning, we predict student collaboration satisfaction. We first evaluate the performance of baseline models on individual modalities for insight into which modalities are most informative. We then devise a multimodal deep learning model that leverages a cross-attention mechanism to attend to salient information across modalities to enhance collaboration satisfaction prediction. Finally,we conduct ablation and feature importance analysis to understand which combination of modalities and features is most effective. Findings indicate that various combinations of data sources are highly beneficial for student collaboration satisfaction prediction. 
    more » « less
  2. ; ; ; ; ; ; ; (, Practical assessment, research & evaluation)
    Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning. 
    more » « less
  3. ; ; ; ; ; ; ; (, Practical assessment research evaluation)
    Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning. 
    more » « less
  4. ; ; ; ; (, Computers and Education Open)
    To address the increasing demand for AI literacy, we introduced a novel active learning approach that leverages both teaching assistants (TAs) and generative AI to provide feedback during in-class exercises. This method was evaluated through two studies in separate Computer Science courses, focusing on the roles and impacts of TAs in this learning environment, as well as their collaboration with ChatGPT in enhancing student feedback. The studies revealed that TAs were effective in accurately determining students’ progress and struggles, particularly in areas such as “backtracking”, where students faced significant challenges. This intervention’s success was evident from high student engagement and satisfaction levels, as reported in an end-of-semester survey. Further findings highlighted that while TAs provided detailed technical assessments and identified conceptual gaps effectively, ChatGPT excelled in presenting clarifying examples and offering motivational support. Despite some TAs’ resistance to fully embracing the feedback guidelines-specifically their reluctance to provide encouragement-the collaborative feedback process between TAs and ChatGPT improved the quality of feedback in several aspects, including technical accuracy and clarity in explaining conceptual issues. These results suggest that integrating human and artificial intelligence in educational settings can significantly enhance traditional teaching methods, creating a more dynamic and responsive learning environment. Future research will aim to improve both the quality and efficiency of feedback, capitalizing on unique strengths of both human and AI to further advance educational practices in the field of computing. 
    more » « less
  5. ; ; (, Journal of learning analytics)
    Large-group (n > 8) co-located collaboration has not been adequately studied because it demands different conceptual framings than those used to study small-group collaboration, while also posing pragmatic constraints on data collection. Working within these pragmatic constraints, we use video data to devise an indicator of the current possibilities for learner collaboration during large-group co-located interactions. We borrow conceptualizations from proxemics and social network analysis to construct collaborative opportunity networks, allowing us to define the concept of collaborative opportunity temperature (COT) readings: a “snapshot” of the current configuration of the different social subgroup structures within a large group, indicating emergent opportunities for collaboration (via talk or shared action) due to proximity. Using a case study of two groups of people (n = 11, n = 12) who interacted with a multi-user museum exhibit, we outline the processes of deriving COT. We show how to quickly detect differences in subgroup configurations that may result from educational interventions and how COT can triangulate with and complement other forms of data (audio transcripts and activity logs) during lengthier analyses. We also outline how COT readings can be used to supply formative feedback on social engagement to learners and be adapted to other learning environments. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email