skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: The Precision-Bias Distinction for Evaluating Visual Decision Aids for Risk Perception
Risk communication is critically important, for both patients and providers. However, people struggle to understand risks because there are inherent biases and limitations to reasoning under uncertainty. A common strategy to enhance risk communication is the use of decision aids, such as charts or graphs, that depict the risk visually. A problem with prior research on visual decision aids is that it used a metric of performance that confounds 2 underlying constructs: precision and bias. Precision refers to a person’s sensitivity to the information, whereas bias refers to a general tendency to overestimate (or underestimate) the level of risk. A visual aid is effective for communicating risk only if it enhances precision or, once precision is suitably high, reduces bias. This article proposes a methodology for evaluating the effectiveness of visual decision aids. Empirical data further illustrate how the new methodology is a significant advancement over more traditional research designs.  more » « less
Award ID(s):
1632222
PAR ID:
10547373
Author(s) / Creator(s):
 
Publisher / Repository:
SAGE Publications
Date Published:
Journal Name:
Medical Decision Making
Volume:
40
Issue:
6
ISSN:
0272-989X
Format(s):
Medium: X Size: p. 846-853
Size(s):
p. 846-853
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; (, Medical Decision Making)
    null (Ed.)
    Background Decision aid developers have to convey complex task-specific numeric information in a way that minimizes bias and promotes understanding of the options available within a particular decision. Whereas our companion paper summarizes fundamental issues, this article focuses on more complex, task-specific aspects of presenting numeric information in patient decision aids. Methods As part of the International Patient Decision Aids Standards third evidence update, we gathered an expert panel of 9 international experts who revised and expanded the topics covered in the 2013 review working in groups of 2 to 3 to update the evidence, based on their expertise and targeted searches of the literature. The full panel then reviewed and provided additional revisions, reaching consensus on the final version. Results Five of the 10 topics addressed more complex task-specific issues. We found strong evidence for using independent event rates and/or incremental absolute risk differences for the effect size of test and screening outcomes. Simple visual formats can help to reduce common judgment biases and enhance comprehension but can be misleading if not well designed. Graph literacy can moderate the effectiveness of visual formats and hence should be considered in tool design. There is less evidence supporting the inclusion of personalized and interactive risk estimates. Discussion More complex numeric information. such as the size of the benefits and harms for decision options, can be better understood by using incremental absolute risk differences alongside well-designed visual formats that consider the graph literacy of the intended audience. More research is needed into when and how to use personalized and/or interactive risk estimates because their complexity and accessibility may affect their feasibility in clinical practice. 
    more » « less
  2. ; ; ; ; ; ; ; ; (, Medical Decision Making)
    null (Ed.)
    Background Shared decision making requires evidence to be conveyed to the patient in a way they can easily understand and compare. Patient decision aids facilitate this process. This article reviews the current evidence for how to present numerical probabilities within patient decision aids. Methods Following the 2013 review method, we assembled a group of 9 international experts on risk communication across Australia, Germany, the Netherlands, the United Kingdom, and the United States. We expanded the topics covered in the first review to reflect emerging areas of research. Groups of 2 to 3 authors reviewed the relevant literature based on their expertise and wrote each section before review by the full authorship team. Results Of 10 topics identified, we present 5 fundamental issues in this article. Although some topics resulted in clear guidance (presenting the chance an event will occur, addressing numerical skills), other topics (context/evaluative labels, conveying uncertainty, risk over time) continue to have evolving knowledge bases. We recommend presenting numbers over a set time period with a clear denominator, using consistent formats between outcomes and interventions to enable unbiased comparisons, and interpreting the numbers for the reader to meet the needs of varying numeracy. Discussion Understanding how different numerical formats can bias risk perception will help decision aid developers communicate risks in a balanced, comprehensible manner and avoid accidental “nudging” toward a particular option. Decisions between probability formats need to consider the available evidence and user skills. The review may be useful for other areas of science communication in which unbiased presentation of probabilities is important. 
    more » « less
  3. ; ; (, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    In this paper we present the promise of the Cognitive Work Analysis (CWA) methodology, particularly abstraction hierarchy modeling, in the foster care domain. There is increasing interest in applying machine learning decision aids to foster care decision making, but that interest is accompanied by concerns that those aids may perpetuate systemic bias or be largely context-blind. Modeling the work conducted at different levels of the domain offers unique insights into where bias may enter the system as well as possible design implications for these future decision aids. This project models two major areas of work in the domain, management of individual cases and management of overall programs offered. These work areas are then considered in the first 3 levels of the abstraction hierarchy to display the promise that this model can hold for the domain in future work, particularly when supported with more naturalistic studies. 
    more » « less
  4. ; ; (, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    Insurance loss prevention survey, specifically windstorm risk inspection survey is the process of investigating potential damages associated with a building or structure in the event of an extreme weather condition such as a hurricane or tornado. This process is performed by a trained windstorm risk engineer who physically goes to a facility to assess the wind vulnerabilities associated with it. This process is highly subjective, and the accuracy of findings depends on the experience and skillsets of the engineer. Although using sensors and automation enabled systems help engineers gather data, their ability to make sense of this information is vital. Further, their Situation Awareness (SA) can be affected by the use of such systems. Using a between-subjects experimental design, this study explored the use of various context-based visualization strategies to support the SA requirements and performance of windstorm risk engineers. The independent variable included in this study is the type of context-based visualizations used (with 3 levels: no visual aids, checklist based and predictive display based visual aids). We measured SA using SAGAT and performance using a questionnaire. SA and performance were found to be higher for the predictive display and checklist based conditions. The findings from this study will inform the design of context-based decision aids to support the SA of risk engineers. 
    more » « less
  5. ; (, Visual Languages and Human-Centered Computing 2022)
    Confirmation bias is a type of cognitive bias that involves seeking and prioritizing information that conforms to a pre-existing view or hypothesis that can negatively affect the decision-making process. We investigate the manifestation and mitigation of confirmation bias with an emphasis on the use of visualization. In a series of Amazon Mechanical Turk studies, participants selected evidence that supported or refuted a given hypothesis. We demonstrated the presence of confirmation bias and investigated the use of five simple visual representations, using color, positional, and length encodings for mitigating this bias. We found that at worst, visualization had no effect in the amount of confirmation bias present, and at best, it was successful in mitigating the bias. We discuss these results in light of factors that can complicate visual debiasing in non-experts. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email