Abstract A left digit effect has been broadly observed across judgment and decision‐making contexts ranging from product evaluation to medical treatment decisions to number line estimation. For example, $3.00 is judged to be a much greater cost than $2.99, and “801” is estimated strikingly too far to the right of “798” on a number line. Although the consequences of the effects for judgment and decision behavior have been documented, the sources of the effects are not well established. The goal of the current work is to extend investigations of the left digit effect to a new complex judgment activity and to assess whether the magnitude of the effect at the individual level can be predicted from performance on a simpler number skills task on which the left digit effect has also recently been observed. In three experiments (N = 434), adults completed a judgment task in which they rated the strength of hypothetical applicants for college admission and a self‐paced number line estimation task. In all experiments, a small or medium left digit effect was found in the college admissions task, and a large effect was found in number line estimation. Individual‐level variation was observed, but there was no relationship between the magnitudes of the effects in the two tasks. These findings provide evidence of a left digit effect in a novel multiattribute judgment task but offer no evidence that such performance can be predicted from a simple number skills task such as number line estimation.
more »
« less
Judgment Sieve: Reducing Uncertainty in Group Judgments through Interventions Targeting Ambiguity versus Disagreement
When groups of people are tasked with making a judgment, the issue of uncertainty often arises. Existing methods to reduce uncertainty typically focus on iteratively improving specificity in the overall task instruction. However, uncertainty can arise from multiple sources, such as ambiguity of the item being judged due to limited context, or disagreements among the participants due to different perspectives and an under-specified task. A one-size-fits-all intervention may be ineffective if it is not targeted to the right source of uncertainty. In this paper we introduce a new workflow, Judgment Sieve, to reduce uncertainty in tasks involving group judgment in a targeted manner. By utilizing measurements that separate different sources of uncertainty during an initial round of judgment elicitation, we can then select a targeted intervention adding context or deliberation to most effectively reduce uncertainty on each item being judged. We test our approach on two tasks: rating word pair similarity and toxicity of online comments, showing that targeted interventions reduced uncertainty for the most uncertain cases. In the top 10% of cases, we saw an ambiguity reduction of 21.4% and 25.7%, and a disagreement reduction of 22.2% and 11.2% for the two tasks respectively. We also found through a simulation that our targeted approach reduced the average uncertainty scores for both sources of uncertainty as opposed to uniform approaches where reductions in average uncertainty from one source came with an increase for the other.
more »
« less
- Award ID(s):
- 2137469
- PAR ID:
- 10487973
- Publisher / Repository:
- ACM
- Date Published:
- Journal Name:
- Proceedings of the ACM on Human-Computer Interaction
- Volume:
- 7
- Issue:
- CSCW2
- ISSN:
- 2573-0142
- Page Range / eLocation ID:
- 1 to 26
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Emerging infectious diseases have caused population declines and biodiversity loss. The ability of pathogens to survive in the environment, independent of their host, can exacerbate disease impacts and increase the likelihood of species extinction. Control of pathogens with environmental stages remains a significant challenge for conservation and effective management strategies are urgently needed.We examined the effectiveness of managing environmental exposure to reduce the impacts of an emerging infectious disease of bats, white‐nose syndrome (WNS). We used a chemical disinfectant, chlorine dioxide (ClO2), to experimentally reducePseudogymnoascus destructans, the fungal pathogen causing WNS, in the environment. We combined laboratory experiments with 3 years of field trials at four abandoned mines to determine whether ClO2could effectively removeP. destructansfrom the environment, reduce host infection and limit population impacts.ClO2was effective at killingP. destructansin vitro across multiple concentrations. In field settings, higher concentrations of ClO2treatment were needed to sufficiently reduce viableP. destructansconidia in the environment.The reduction in the environmental reservoir at treatment sites resulted in lower fungal loads on bats compared to untreated control populations. Survival following treatment was also higher in little brown bats (Myotis lucifugus), and trended higher for tricolored bats (Perimyotis subflavus).Synthesis and applications. Our results highlight that targeted management of sources for environmental transmission can be an effective control strategy for wildlife disease. We found that successfully reducing pathogen in the environment decreased disease severity and increased survival, but required higher treatment exposure than was effective in laboratory experiments, and the effects varied among species. More broadly, our findings have implications for other emerging wildlife diseases with free‐living pathogen stages by highlighting how the degree of environmental contamination can have cascading impacts on hosts, presenting an opportunity for intervention.more » « less
-
Uncertainty decomposition refers to the task of decomposing the total uncertainty of a predictive model into aleatoric (data) uncertainty, resulting from inherent randomness in the data-generating process, and epistemic (model) uncertainty, resulting from missing information in the model’s training data. In large language models (LLMs) specifically, identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability, but remains an important open research question. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling, which can be applied to any pre-trained LLM. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions. We show that, when aleatoric uncertainty arises from ambiguity or under-specification in LLM inputs, this approach makes it possible to factor an (un-clarified) LLM’s predictions into separate aleatoric and epistemic terms, using a decomposition similar to the one employed by Bayesian neural networks. Empirical evaluations demonstrate that input clarification ensembling provides accurate and reliable uncertainty quantification on several language processing tasks.more » « less
-
Age Differences In Retrieval-Related Reinstatement Reflect Age-Related Dedifferentiation At EncodingAbstract Age-related reductions in neural selectivity have been linked to cognitive decline. We examined whether age differences in the strength of retrieval-related cortical reinstatement could be explained by analogous differences in neural selectivity at encoding, and whether reinstatement was associated with memory performance in an age-dependent or an age-independent manner. Young and older adults underwent fMRI as they encoded words paired with images of faces or scenes. During a subsequent scanned memory test participants judged whether test words were studied or unstudied and, for words judged studied, also made a source memory judgment about the associated image category. Using multi-voxel pattern similarity analyses, we identified robust evidence for reduced scene reinstatement in older relative to younger adults. This decline was however largely explained by age differences in neural differentiation at encoding; moreover, a similar relationship between neural selectivity at encoding and retrieval was evident in young participants. The results suggest that, regardless of age, the selectivity with which events are neurally processed at the time of encoding can determine the strength of retrieval-related cortical reinstatement.more » « less
-
Engineering judgment is critical to both engineering education and engineering practice, and the ability to practice or participate in engineering judgment is often considered central to the formation of professional engineering identities. In practice, engineers must make difficult judgments that evaluate potentially competing objectives, ambiguity, uncertainty, incomplete information, and evolving technical knowledge. Nonetheless, while engineering judgment is implicit in engineering work and so central to identification with the profession, educators and practitioners have few actionable frameworks to employ when considering how to develop and assess this capacity in students. In this paper, we propose a theoretical framework designed to inform both educators and researchers that positions engineering judgment at the intersection of the cognitive dimensions of naturalistic decision-making, and discursive dimensions of identity. Our proposed theory positions engineering judgment not only as an individual capacity practiced by individual engineers alone but also as the capacity to position oneself within the discursive community so as to participate in the construction of engineering judgments among a group of professionals working together. Our theory draws on several strands of existing research to theorize a working framework for engineering judgment that considers the cognitive processes associated with making judgments and the inextricable discursive practices associated with negotiating those judgments in context. In constructing this theory, we seek to provide engineering education practitioners and researchers with a framework that can inform the design of assignments, curricula, or experiences that are intended to foster students’ participation in the development and practice of engineering judgment.more » « less
An official website of the United States government

