skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Exploring the process of group-based collaboration: a validation argument for a collaboration model and observation rubric for training explainable machine learning models.
Award ID(s):
2016849
PAR ID:
10347108
Author(s) / Creator(s):
Editor(s):
C. Chinn, E. Tan
Date Published:
Journal Name:
Proceedings of the 16th International Conference of the Learning Sciences - ICLS 2022
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. De_Vita, R; Espinal, X; Laycock, P; Shadura, O (Ed.)
    The Big Science projects common of multi-institute particle-physics collaborations generates unique needs for member management, including paper authorship tracking, shift assignments, subscription to mailing lists and access to 3rd party applications such as Github and Slack. For smaller collaborations under 200 people, often no facility for centralized member management is available and these needs are usually manually handled by long-term members despite the management becoming untenable as collaborations grow. To automate many of these tasks for the expanding XENON collaboration, we developed the XENONnT User Management Website, a web application that stores and updates data related to the collaboration members through the use of Node.js and MongoDB. We found that web frameworks are so mature and approachable such that a student can develop a good system to meet the unique needs of the collaboration. The application allows for the scheduling of shifts for members to coordinate between institutes. User manipulation of 3rd party applications are implemented using REST API integration. The XENONnT User Management Website is open source and is a show case of quick implementation of utility application using the web framework, which demonstrated the utility of web-based approaches for solving specific problems to aid the logistics of running Big Science collaborations. 
    more » « less
  2. Effective human-AI collaboration hinges not only on the AI agent’s ability to follow explicit instructions but also on its capacity to navigate ambiguity, incompleteness, invalidity, and irrelevance in communication. Gricean conversational and inference norms facilitate collaboration by aligning unclear instructions with cooperative principles. We propose a normative framework that integrates Gricean norms and cognitive frameworks—common ground, relevance theory, and theory of mind—into large language model (LLM) based agents. The normative framework adopts the Gricean maxims of quantity, quality, relation, and manner, along with inference, as Gricean norms to interpret unclear instructions, which are: ambiguous, incomplete, invalid, or irrelevant. Within this framework, we introduce Lamoids, GPT-4 powered agents designed to collaborate with humans. To assess the influence of Gricean norms in human- AI collaboration, we evaluate two versions of a Lamoid: one with norms and one without. In our experiments, a Lamoid collaborates with a human to achieve shared goals in a grid world (Doors, Keys, and Gems) by interpreting both clear and unclear natural language instructions. Our results reveal that the Lamoid with Gricean norms achieves higher task accuracy and generates clearer, more accurate, and contextually relevant responses than the Lamoid without norms. This improvement stems from the normative framework, which enhances the agent’s pragmatic reasoning, fostering effective human-AI collaboration and enabling context-aware communication in LLM-based agents. 
    more » « less
  3. The gold standard in human-AI collaboration is complementarity: when combined performance exceeds both the human and algorithm alone. We investigate this challenge in binary classification settings where the goal is to maximize 0-1 accuracy. Given two or more agents who can make calibrated probabilistic predictions, we show a No Free Lunch-style result. Any deterministic collaboration strategy (a function mapping calibrated probabilities into binary classifications) that does not essentially always defer to the same agent will sometimes perform worse than the least accurate agent. In other words, complementarity cannot be achieved for free. The result does suggest one model of collaboration with guarantees, where one agent identifies obvious errors of the other agent. We also use the result to understand the necessary conditions enabling the success of other collaboration techniques, providing guidance to human-AI collaboration. 
    more » « less