skip to main content


Title: An Objective Measure of Decisional Clarity to Assess Decision Aid Effectiveness in Situations with Equipoise: A Randomized Trial
Background

Decision aids can help patients make medical decisions, which is especially advantageous in situations with equipoise. However, when there is no correct answer, it is difficult to assess whether a decision aid is helpful. The goal of this research is to propose and validate an objective method for measuring decision aid effectiveness by quantifying the clarity participants achieved when making decisions.

Design

The measure of decisional clarity was tested in a convenience sample of 131 college-aged students making hypothetical decisions about 2 treatment options for depression and anxiety. The treatments varied with respect to potential benefits and harms. Information was presented numerically or with an accompanying data visualization (an icon array) that is known to aid decision making.

Results

Decisional clarity was better with the icon arrays. Furthermore, the results showed that decisional clarity can be used to identify situations for which patients will be more likely to struggle making their decision. These included situations for which financial considerations were relevant to the decision and situations for which the probabilities of potential benefits were higher.

Limitations

The measure of decisional clarity and the situations identified as lacking clarity should be validated with a larger, more representative sample.

Conclusions

These findings demonstrate that decisional clarity can be used to both empirically evaluate the effectiveness of a decision aid as well as test factors that can cloud clarity and disrupt medical decision making.

Implications

Researchers and medical providers interested in developing decision aids for situations with equipoise can use decisional clarity as an objective measure to assess the effectiveness of their decision aid. Financial considerations and higher probabilities may also cloud judgments.

Highlights

An objective measure of decisional clarity is supported. Decisional clarity can be used to evaluate decision aids in the context of equipoise for which there is no objectively correct choice. Decisional clarity can also be used to identify scenarios for which patients are likely to struggle to make a medical decision.

 
more » « less
NSF-PAR ID:
10368918
Author(s) / Creator(s):
 
Publisher / Repository:
SAGE Publications
Date Published:
Journal Name:
Medical Decision Making
Volume:
42
Issue:
6
ISSN:
0272-989X
Page Range / eLocation ID:
p. 822-831
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Background

    Objective numeracy appears to support better medical decisions and health outcomes. The more numerate generally understand and use numbers more and make better medical decisions, including more informed medical choices. Numeric self-efficacy—an aspect of subjective numeracy that is also known as numeric confidence—also relates to decision making via emotional reactions to and inferences from experienced difficulty with numbers and via persistence linked with numeric comprehension and healthier behaviors over time. Furthermore, it moderates the effects of objective numeracy on medical outcomes.

    Purpose

    We briefly review the numeracy and decision-making literature and then summarize more recent literature on 3 separable effects of numeric self-efficacy. Although dual-process theories can account for the generally superior decision making of the highly numerate, they have neglected effects of numeric self-efficacy. We discuss implications for medical decision-making (MDM) research and practice. Finally, we propose a modification to dual-process theories, adding a “motivational mind” to integrate the effects of numeric self-efficacy on decision-making processes (i.e., inferences from experienced difficulty with numbers, greater persistence, and greater use of objective-numeracy skills) important to high-quality MDM.

    Conclusions

    The power of numeric self-efficacy (confidence) has been little considered in MDM, but many medical decisions and behaviors require persistence to be successful over time (e.g., comprehension, medical-recommendation adherence). Including numeric self-efficacy in research and theorizing will increase understanding of MDM and promote development of better decision interventions.

    Highlights

    Research demonstrates that objective numeracy supports better medical decisions and health outcomes. The power of numeric self-efficacy (aka numeric confidence) has been little considered but appears critical to emotional reactions and inferences that patients and others make when encountering numeric information (e.g., in decision aids) and to greater persistence in medical decision-making tasks involving numbers. The present article proposes a novel modification to dual-process theory to account for newer findings and to describe how numeracy mechanisms can be better understood. Because being able to adapt interventions to improve medical decisions depends in part on having a good theory, future research should incorporate numeric self-efficacy into medical decision-making theories and interventions.

     
    more » « less
  2. In emergency situations, such as the current COVID-19 pandemic, less immediate concerns such as cybersecurity and long-term economic impact can fall by the wayside. This paper presents a discussion of the impact of cybersecurity issues that occur during and are attributable to pandemics and other emergency situations. This discussion is facilitated by a simulation tool, the Disaster Vulnerability Threat and Impact Simulator System (DVTISS). DVTISS simulates the network structure, security measures, user characteristics and demographics, data, and devices of an organization or region’s computing infrastructure. The system is provided input parameters and performs analysis to identify the combined results of numerous different decisions, which are made in concert, to identify the types of vulnerabilities that may be present and the impact of their exploitation. The impacts of system unavailability are considered. This can aid businesses, governments and others in determining the level of prioritization that should be given to cybersecurity considerations. The simulator can also be used for disaster preparedness and planning, evaluating particular response strategies and the evaluation of laws and policies that impact IT decision making during emergencies. This paper uses the DVTISS tool to consider organizational responses to several example emergency situations. It demonstrates the utility of the tool as well as its efficacy for decision making support. Based on the example emergencies, the paper also discusses key areas of vulnerability during emergency situations and their financial, data and system outage impacts. © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. 
    more » « less
  3. Abstract Expert testimony varies in scientific quality and jurors have a difficult time evaluating evidence quality (McAuliff et al., 2009). In the current study, we apply Fuzzy Trace Theory principles, examining whether visual and gist aids help jurors calibrate to the strength of scientific evidence. Additionally we were interested in the role of jurors’ individual differences in scientific reasoning skills in their understanding of case evidence. Contrary to our preregistered hypotheses, there was no effect of evidence condition or gist aid on evidence understanding. However, individual differences between jurors’ numeracy skills predicted evidence understanding. Summary Poor-quality expert evidence is sometimes admitted into court (Smithburn, 2004). Jurors’ calibration to evidence strength varies widely and is not robustly understood. For instance, previous research has established jurors lack understanding of the role of control groups, confounds, and sample sizes in scientific research (McAuliff, Kovera, & Nunez, 2009; Mill, Gray, & Mandel, 1994). Still others have found that jurors can distinguish weak from strong evidence when the evidence is presented alone, yet not when simultaneously presented with case details (Smith, Bull, & Holliday, 2011). This research highlights the need to present evidence to jurors in a way they can understand. Fuzzy Trace Theory purports that people encode information in exact, verbatim representations and through “gist” representations, which represent summary of meaning (Reyna & Brainerd, 1995). It is possible that the presenting complex scientific evidence to people with verbatim content or appealing to the gist, or bottom-line meaning of the information may influence juror understanding of that evidence. Application of Fuzzy Trace Theory in the medical field has shown that gist representations are beneficial for helping laypeople better understand risk and benefits of medical treatment (Brust-Renck, Reyna, Wilhelms, & Lazar, 2016). Yet, little research has applied Fuzzy Trace Theory to information comprehension and application within the context of a jury (c.f. Reyna et. al., 2015). Additionally, it is likely that jurors’ individual characteristics, such as scientific reasoning abilities and cognitive tendencies, influence their ability to understand and apply complex scientific information (Coutinho, 2006). Methods The purpose of this study was to examine how jurors calibrate to the strength of scientific information, and whether individual difference variables and gist aids inspired by Fuzzy Trace Theory help jurors better understand complicated science of differing quality. We used a 2 (quality of scientific evidence: high vs. low) x 2 (decision aid to improve calibration - gist information vs. no gist information), between-subjects design. All hypotheses were preregistered on the Open Science Framework. Jury-eligible community participants (430 jurors across 90 juries; Mage = 37.58, SD = 16.17, 58% female, 56.93% White). Each jury was randomly assigned to one of the four possible conditions. Participants were asked to individually fill out measures related to their scientific reasoning skills prior to watching a mock jury trial. The trial was about an armed bank robbery and consisted of various pieces of testimony and evidence (e.g. an eyewitness testimony, police lineup identification, and a sweatshirt found with the stolen bank money). The key piece of evidence was mitochondrial DNA (mtDNA) evidence collected from hair on a sweatshirt (materials from Hans et al., 2011). Two experts presented opposing opinions about the scientific evidence related to the mtDNA match estimate for the defendant’s identification. The quality and content of this mtDNA evidence differed based on the two conditions. The high quality evidence condition used a larger database than the low quality evidence to compare to the mtDNA sample and could exclude a larger percentage of people. In the decision aid condition, experts in the gist information group presented gist aid inspired visuals and examples to help explain the proportion of people that could not be excluded as a match. Those in the no gist information group were not given any aid to help them understand the mtDNA evidence presented. After viewing the trial, participants filled out a questionnaire on how well they understood the mtDNA evidence and their overall judgments of the case (e.g. verdict, witness credibility, scientific evidence strength). They filled this questionnaire out again after a 45-minute deliberation. Measures We measured Attitudes Toward Science (ATS) with indices of scientific promise and scientific reservations (Hans et al., 2011; originally developed by National Science Board, 2004; 2006). We used Drummond and Fischhoff’s (2015) Scientific Reasoning Scale (SRS) to measure scientific reasoning skills. Weller et al.’s (2012) Numeracy Scale (WNS) measured proficiency in reasoning with quantitative information. The NFC-Short Form (Cacioppo et al., 1984) measured need for cognition. We developed a 20-item multiple-choice comprehension test for the mtDNA scientific information in the cases (modeled on Hans et al., 2011, and McAuliff et al., 2009). Participants were shown 20 statements related to DNA evidence and asked whether these statements were True or False. The test was then scored out of 20 points. Results For this project, we measured calibration to the scientific evidence in a few different ways. We are building a full model with these various operationalizations to be presented at APLS, but focus only on one of the calibration DVs (i.e., objective understanding of the mtDNA evidence) in the current proposal. We conducted a general linear model with total score on the mtDNA understanding measure as the DV and quality of scientific evidence condition, decision aid condition, and the four individual difference measures (i.e., NFC, ATS, WNS, and SRS) as predictors. Contrary to our main hypotheses, neither evidence quality nor decision aid condition affected juror understanding. However, the individual difference variables did: we found significant main effects for Scientific Reasoning Skills, F(1, 427) = 16.03, p <.001, np2 = .04, Weller Numeracy Scale, F(1, 427) = 15.19, p <.001, np2 = .03, and Need for Cognition, F(1, 427) = 16.80, p <.001, np2 = .04, such that those who scored higher on these measures displayed better understanding of the scientific evidence. In addition there was a significant interaction of evidence quality condition and scores on the Weller’s Numeracy Scale, F(1, 427) = 4.10, p = .04, np2 = .01. Further results will be discussed. Discussion These data suggest jurors are not sensitive to differences in the quality of scientific mtDNA evidence, and also that our attempt at helping sensitize them with Fuzzy Trace Theory-inspired aids did not improve calibration. Individual scientific reasoning abilities and general cognition styles were better predictors of understanding this scientific information. These results suggest a need for further exploration of approaches to help jurors differentiate between high and low quality evidence. Note: The 3rd author was supported by an AP-LS AP Award for her role in this research. Learning Objective: Participants will be able to describe how individual differences in scientific reasoning skills help jurors understand complex scientific evidence. 
    more » « less
  4. Theory—understanding mental processes that drive decisions—is important to help patients and providers make decisions that reflect medical advances and personal values. Building on a 2008 review, we summarize current tenets of fuzzy-trace theory (FTT) in light of new evidence that provides insight regarding mental representations of options and how such representations connect to values and evoke emotions. We discuss implications for communicating risks, preventing risky behaviors, discouraging misinformation, and choosing appropriate treatments. Findings suggest that simple, fuzzy but meaningful gist representations of information often determine decisions. Within minutes of conversing with their doctor, reading a health-related web post, or processing other health information, patients rely on gist memories of that information rather than verbatim details. This fuzzy-processing preference explains puzzles and paradoxes in how patients (and sometimes providers) think about probabilities (e.g., “50-50” chance), outcomes of treatment (e.g., with antibiotics), experiences of pain, end-of-life decisions, memories for medication instructions, symptoms of concussion, and transmission of viruses (e.g., in AIDS and COVID-19). As examples, participation in clinical trials or seeking treatments with low probabilities of success (e.g., with antibiotics or at the end of life) may indicate a defensibly different categorical gist perspective on risk as opposed to simply misunderstanding probabilities or failing to make prescribed tradeoffs. Thus, FTT explains why people avoid precise tradeoffs despite computing them. Facilitating gist representations of information offers an alternative approach that goes beyond providing uninterpreted “neutral” facts versus persuading or shifting the balance between fast versus slow thinking (or emotion vs. cognition). In contrast to either taking mental shortcuts or deliberating about details, gist processing facilitates application of advanced knowledge and deeply held values to choices. Highlights Fuzzy-trace theory (FTT) supports practical approaches to improving health and medicine. FTT differs in important respects from other theories of decision making, which has implications for how to help patients, providers, and health communicators. Gist mental representations emphasize categorical distinctions, reflect understanding in context, and help cue values relevant to health and patient care. Understanding the science behind theory is crucial for evidence-based medicine. 
    more » « less
  5. Objective

    Patients have a poor understanding of outcomes related to total knee replacement (TKR) surgery, with most patients underestimating the potential benefits and overestimating the risk of complications. In this study, we sought to compare the impacts of descriptive information alone or in combination with an icon array, experience condition (images), or spinner on participants’ preference forTKR.

    Methods

    A total of 648 members of an online arthritis network were randomized to 1 of 4 outcome presentation formats: numeric only, numeric with an icon array, numeric with a set of 50 images, or numeric with a functional spinner. Preferences forTKRwere measured before and immediately after viewing the outcome information using an 11‐point numeric rating scale. Knowledge was assessed by asking participants to report the frequency of each outcome.

    Results

    Participants randomized to the icon array, images, and spinner had stronger preferences forTKR(after controlling for baseline preferences) compared to those viewing the numeric only format (P< 0.05 for all mean differences). Knowledge scores were highest in participants randomized to the icon array; however, knowledge did not mediate the association between format and change in preference forTKR.

    Conclusion

    Decision support at the point‐of‐care is being increasingly recognized as a vital component of care. Our findings suggest that adding graphic information to descriptive statistics strengthens preferences forTKR. Although experience formats using images may be too complex to use in clinical practice, icon arrays and spinners may be a viable and easily adaptable decision aid to support communication of probabilistic information.

     
    more » « less