An official website of the United States government
ABSTRACT A cursory read of the popular press and the JDM research field suggest that they have very different beliefs regarding the consequences of confidence. And these beliefs have important consequences themselves. For individuals, how one views the consequences of confidence (and whether they are positive or negative) likely influences the extent to which one pursues the development of confidence and how one interprets confidence in others. For JDM researchers, their beliefs about the consequences of confidence inform research programs. For example, a belief that overconfidence leads to inappropriate medical treatments, legal advice, or investments suggests an emphasis on reducing overconfidence rather than on developing confidence. This paper aims to improve understanding of both the general public's and JDM researchers' beliefs about the consequences of confidence in knowledge. We present a general theoretical framework for thinking about the consequences of confidence, followed by two exploratory studies designed to access these beliefs, first with the general public and then with JDM researchers. We used structured, open‐ended questioning to generate a large dataset (over 10,000 responses) of potential consequences of low confidence, high confidence, overconfidence, and underconfidence. Qualitative coding identified a broad set of respondent‐generated beliefs regarding psychological and behavioral consequences, organized into antonym pairs (e.g., arrogant/high self‐image vs. low self‐image). Respondents made few distinctions between low confidence and underconfidence, viewing both negatively. However, the general public drew a sharp distinction between high confidence (described positively) and overconfidence (described negatively), a trend less prevalent among JDM researchers.
more »
« less
This content will become publicly available on December 1, 2026
Influencing Confidence: Testing Ways to Increase or Decrease Confidence in Knowledge
ABSTRACT This paper examines approaches for influencing people's confidence in their knowledge without influencing knowledge. Three studies examined the relative effectiveness of training and false feedback approaches. Participants chose which of two IKEA products they thought was more expensive and indicated their confidence in that judgment for 50 product pairs. In Study 1, participants took part in one of five conditions designed to manipulate their confidence: false feedback‐increasing, false feedback‐decreasing, training‐increasing, training‐decreasing, or control. For false feedback, we told participants they did very well or poorly on the task. For training‐increasing, we gave participants information about IKEA pricing that appeared useful but was difficult to implement. For training‐decreasing, we developed an automated calibration training technique that provided personalized calibration feedback consisting of a calibration diagram accompanied by textual summary information and advice. Neither the false feedback nor training approach increased confidence on 50 subsequent knowledge‐confidence judgments. However, both manipulations designed to reduce confidence were successful, with a substantially larger effect in the calibration training condition. In Study 2, we adapted the calibration training approach to provide false feedback indicating participants were either underconfident or overconfident. Both the original calibration training pproach and the new false feedback approach indicating overconfidence reduced confidence, and the false feedback approach indicating underconfidence increased confidence. Study 3 tested the effectiveness of this new false feedback approach on an on‐line rather than student sample, finding essentially the same results as those in Study 2. Throughout the three studies, the effects of the manipulations extended to overconfidence, overall calibration, and the Brier score. The results provide a potential tool for research and practice regarding confidence in knowledge.
more »
« less
- PAR ID:
- 10656209
- Publisher / Repository:
- Wiley
- Date Published:
- Journal Name:
- Journal of Behavioral Decision Making
- Volume:
- 38
- Issue:
- 5
- ISSN:
- 0894-3257
- Subject(s) / Keyword(s):
- confidence knowledge overconfidence calibration training false feedback
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Overconfidence is a common issue for deep neural networks, limiting their deployment in real-world applications. To better estimate confidence, existing methods mostly focus on fully-supervised scenarios and rely on training labels. In this paper, we propose the first confidence estimation method for a semi-supervised setting, when most training labels are unavailable. We stipulate that even with limited training labels, we can still reasonably approximate the confidence of model on unlabeled samples by inspecting the prediction consistency through the training process. We use training consistency as a surrogate function and propose a consistency ranking loss for confidence estimation. On both image classification and segmentation tasks, our method achieves state-of-the-art performances in confidence estimation. Furthermore, we show the benefit of the proposed method through a downstream active learning task.more » « less
-
Combining uncertainty information with AI recommendations supports calibration with domain knowledgeThe use of Artificial Intelligence (AI) decision support is increasing in high-stakes contexts, such as healthcare, defense, and finance. Uncertainty information may help users better leverage AI predictions, especially when combined with their domain knowledge. We conducted a human-subject experiment with an online sample to examine the effects of presenting uncertainty information with AI recommendations. The experimental stimuli and task, which included identifying plant and animal images, are from an existing image recognition deep learning model, a popular approach to AI. The uncertainty information was predicted probabilities for whether each label was the true label. This information was presented numerically and visually. In the study, we tested the effect of AI recommendations in a within-subject comparison and uncertainty information in a between-subject comparison. The results suggest that AI recommendations increased both participants’ accuracy and confidence. Further, providing uncertainty information significantly increased accuracy but not confidence, suggesting that it may be effective for reducing overconfidence. In this task, participants tended to have higher domain knowledge for animals than plants based on a self-reported measure of domain knowledge. Participants with more domain knowledge were appropriately less confident when uncertainty information was provided. This suggests that people use AI and uncertainty information differently, such as an expert versus second opinion, depending on their level of domain knowledge. These results suggest that if presented appropriately, uncertainty information can potentially decrease overconfidence that is induced by using AI recommendations.more » « less
-
Mulherkar, Shalaka (Ed.)Increasing balance confidence in older individuals is important towards improving their quality of life and reducing activity avoidance. Here, we investigated if balance confidence (perceived ability) and balance performance (ability) in older adults were related to one another and would improve after balance training. The relationship of balance confidence in conjunction with balance performance for varied conditions (such as limiting vision, modifying somatosensory cues, and also base of support) was explored. We sought to determine if balance confidence and ability, as well as their relationship, could change after several weeks of training. Twenty-seven healthy participants were trained for several weeks during standing and walking exercises. In addition, seven participants with a higher risk of imbalance leading to falls (survivors of stroke) were also trained. Prior to and after training, balance ability and confidence were assessed via the Balance Error Scoring System (BESS) and Activities Specific Balance Confidence (ABC) Scale, respectively. Both groups showed improvements in balance abilities (i.e., BESS errors significantly decreased after training). Balance confidence was significantly higher in the healthy group than in the stroke group; however, ABC results reflected that balance confidence did not significantly increase after training for each. The correlations between balance ability and balance confidence were explored. Encouragingly, healthy participants displayed a negative correlation between BESS errors and ABC (i.e., enhancements in balance confidence (increases in ABC Scale results) were related to improvements in balance ability (decreases in BESS errors)). For the stroke participants, despite improvements in balance ability, our results showed that there was no relation to balance confidence (i.e., no correlation between BESS errors and ABC) in this group.more » « less
-
To achieve a goal, people have to keep track of how much effort they are putting in (effort monitoring) and how well they are performing (performance monitoring), which can be informed by endogenous signals, or exogenous signals providing explicit feedback about whether they have met their goal. Interventions to improve performance often focus on adjusting feedback to direct the individual on how to better invest their efforts, but is it possible that this feedback itself plays a role in shaping the experience of how effortful the task feels? Here, we examine this question directly by assessing the relationship between effort monitoring and performance monitoring. Participants (N = 68) performed a task in which their goal was to squeeze a handgrip to within a target force level (not lower or higher) for a minimum duration. On most trials, they were given no feedback as to whether they met their goal, and were largely unable to detect how they had performed. On a subset of trials, however, we provided participants with (false) feedback indicating that they had either succeeded or failed at meeting their goal (positive vs. negative feedback blocks, respectively). Sporadically, participants rated their experience of effort exertion, fatigue, and confidence in having met the target grip force on that trial. Despite being non-veridical to their actual performance, we found that the type of feedback participants received influenced their experience of effort. When receiving negative (vs. positive) feedback, participants fatigued faster and adjusted their grip strength more for higher target force levels. We also found that confidence gradually increased with increasing positive feedback and decreased with increasing negative feedback, again despite feedback being uniformly uninformative. These results suggest differential influences of feedback on experiences related to effort and further shed light on the relationship between experiences related to performance monitoring and effort monitoring.more » « less
