%AYu, Li [School of Information Renmin University of China 59 Zhongguancun St, Haidian Beijing 100872 China]%AZhang, Dongsong [Belk College of Business The University of North Carolina at Charlotte 9201 University City Blvd Charlotte NC 28223‐0001 USA]%AFu, Zhe [School of Information Renmin University of China Beijing 100872 China]%BJournal Name: Decision Sciences; Journal Volume: 52; Journal Issue: 6; Related Information: CHORUS Timestamp: 2023-08-28 03:45:47 %D2020%IWiley-Blackwell %JJournal Name: Decision Sciences; Journal Volume: 52; Journal Issue: 6; Related Information: CHORUS Timestamp: 2023-08-28 03:45:47 %K %MOSTI ID: 10361018 %PMedium: X %TCE‐DIFF: An Approach to Identifying and Coping with Irregular Ratings in Collaborative Decision Making %XABSTRACT

A key to collaborative decision making is to aggregate individual evaluations into a group decision. One of its fundamental challenges lies in the difficulty in identifying and dealing with irregular or unfair ratings and reducing their impact on group decisions. Little research has attempted to identify irregular ratings in a collaborative assessment task, let alone develop effective approaches to reduce their negative impact on the final group judgment. In this article, based on the synergy theory, we propose a novel consensus‐based collaborative evaluation (CE) method called Collaborative Evaluation based on rating DIFFerence (CE‐DIFF) for identifying irregular ratings and mitigating their impact on collaborative decisions. CE‐DIFF determines and assigns different weights automatically to individual evaluators or ratings based on the level of consistency of one's ratings with the group assessment outcome through continuous iterations. We conducted two empirical experiments to evaluate the proposed method. The results show that CE‐DIFF has higher accuracy in dealing with irregular ratings than existing CE methods, such as arithmetic mean and trimmed mean. In addition, the effectiveness of CE‐DIFF is independent of group size. This study provides a new and more effective method for collaborative assessment, as well as novel theoretical insights and practical implications on how to improve collaborative assessment.

%0Journal Article