skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Early Chatbot Assistance Can Enhance Team Decision-Making by Promoting Cognitive Diversity and Information Elaboration
As AI increasingly assists teams in decision-making, the study examines how technology shapes team processes and performance. We conducted an online experiment of team decision-making assisted by chatbots and analyzed team interaction processes with computational methods. We found that teams assisted by a chatbot offering information in the first half of their decision-making process performed better than those assisted by the chatbot in the second half. The effect was explained by the variation in teams’ information-sharing process between the two chatbot conditions. When assisted by the chatbot in the first half of the decision-making task, teams showed higher levels of cognitive diversity (i.e., the difference in the information they shared) and information elaboration (i.e., exchange and integration of information). The findings demonstrate that if introduced early, AI can support team decision-making by acting as a catalyst to promote team information sharing.  more » « less
Award ID(s):
2105169
PAR ID:
10545069
Author(s) / Creator(s):
;
Publisher / Repository:
AMCIS 2023 Proceedings
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, Proceedings of the Hawaii International Conference on System Sciences)
    Whereas artificial intelligence (AI) is increasingly used to facilitate team decision-making, little is known about how the timing of AI assistance may impact team performance. The study investigates this question with an online experiment in which teams completed a new product development task with assistance from a chatbot. Information needed for making the decision was distributed among the team members. The chatbot shared information critical to the decision in either the first half or second half of team interaction. The results suggest that teams assisted by the chatbot in the first half of the decision-making task made better decisions than those assisted by the chatbot in the second half. Analysis of team member perceptions and interaction processes suggests that having a chatbot at the beginning of team interaction may have generated a ripple effect in the team that promoted information sharing among team members. 
    more » « less
  2. ; (, ICIS 2023 Proceedings)
    As the integration of artificial intelligence (AI) into team decision-making continues to expand, it is both theoretically and practically pressing for researchers to understand the impact of the technology on team dynamics and performance. To investigate this relationship, we conducted an online experiment in which teams made decisions supported by chatbots and employed computational methods to analyze team interaction processes. Our results indicated that compared to those assisted by chatbots in later phases, teams receiving chatbot assistance during the initial phase of their decision-making process exhibited increased cognitive diversity (i.e., diversity in shared information) and information elaboration (i.e., exchange and integration of information). Ultimately, teams assisted by chatbots early on performed better. These results imply that introducing AI at the beginning of the process can enhance team decision-making by promoting effective information sharing among team members. 
    more » « less
  3. ; ; (, Proceedings of the AAAI Conference on Artificial Intelligence)
    The increased integration of artificial intelligence (AI) technologies in human workflows has resulted in a new paradigm of AI-assisted decision making,in which an AI model provides decision recommendations while humans make the final decisions. To best support humans in decision making, it is critical to obtain a quantitative understanding of how humans interact with and rely on AI. Previous studies often model humans' reliance on AI as an analytical process, i.e., reliance decisions are made based on cost-benefit analysis. However, theoretical models in psychology suggest that the reliance decisions can often be driven by emotions like humans' trust in AI models. In this paper, we propose a hidden Markov model to capture the affective process underlying the human-AI interaction in AI-assisted decision making, by characterizing how decision makers adjust their trust in AI over time and make reliance decisions based on their trust. Evaluations on real human behavior data collected from human-subject experiments show that the proposed model outperforms various baselines in accurately predicting humans' reliance behavior in AI-assisted decision making. Based on the proposed model, we further provide insights into how humans' trust and reliance dynamics in AI-assisted decision making is influenced by contextual factors like decision stakes and their interaction experiences. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; ; et al (, ACM)
    Despite the growing interest in human-AI decision making, experimental studies with domain experts remain rare, largely due to the complexity of working with domain experts and the challenges in setting up realistic experiments. In this work, we conduct an in-depth collaboration with radiologists in prostate cancer diagnosis based on MRI images. Building on existing tools for teaching prostate cancer diagnosis, we develop an interface and conduct two experiments to study how AI assistance and performance feedback shape the decision making of domain experts. In Study 1, clinicians were asked to provide an initial diagnosis (human), then view the AI's prediction, and subsequently finalize their decision (human-AI team). In Study 2 (after a memory wash-out period), the same participants first received aggregated performance statistics from Study 1, specifically their own performance, the AI's performance, and their human-AI team performance, and then directly viewed the AI's prediction before making their diagnosis (i.e., no independent initial diagnosis). These two workflows represent realistic ways that clinical AI tools might be used in practice, where the second study simulates a scenario where doctors can adjust their reliance and trust on AI based on prior performance feedback. Our findings show that, while human-AI teams consistently outperform humans alone, they still underperform the AI due to under-reliance, similar to prior studies with crowdworkers. Providing clinicians with performance feedback did not significantly improve the performance of human-AI teams, although showing AI decisions in advance nudges people to follow AI more. Meanwhile, we observe that the ensemble of human-AI teams can outperform AI alone, suggesting promising directions for human-AI collaboration. 
    more » « less
  5. ; ; (, International Joint Conferences on Artificial Intelligence Organization)
    With the rapid development of decision aids that are driven by AI models, the practice of AI-assisted decision making has become increasingly prevalent. To improve the human-AI team performance in decision making, earlier studies mostly focus on enhancing humans' capability in better utilizing a given AI-driven decision aid. In this paper, we tackle this challenge through a complementary approach—we aim to train behavior-aware AI by adjusting the AI model underlying the decision aid to account for humans' behavior in adopting AI advice. In particular, as humans are observed to accept AI advice more when their confidence in their own judgement is low, we propose to train AI models with a human-confidence-based instance weighting strategy, instead of solving the standard empirical risk minimization problem. Under an assumed, threshold-based model characterizing when humans will adopt the AI advice, we first derive the optimal instance weighting strategy for training AI models. We then validate the efficacy and robustness of our proposed method in improving the human-AI joint decision making performance through systematic experimentation on synthetic datasets. Finally, via randomized experiments with real human subjects along with their actual behavior in adopting the AI advice, we demonstrate that our method can significantly improve the decision making performance of the human-AI team in practice. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email