An official website of the United States government
The visualization community has seen a rise in the adoption of user studies. Empirical user studies systematically test the assumptions that we make about how visualizations can help or hinder viewers’ performance of tasks. Although the increase in user studies is encouraging, it is vital that research on human reasoning with visualizations be grounded in an understanding of how the mind functions. Previously, there were no sufficient models that illustrate the process of decision-making with visualizations. However, Padilla et al. [41] recently proposed an integrative model for decision-making with visualizations, which expands on modern theories of visualization cognition and decision-making. In this paper, we provide insights into how cognitive models can accelerate innovation, improve validity, and facilitate replication efforts, which have yet to be thoroughly discussed in the visualization community. To do this, we offer a compact overview of the cognitive science of decision-making with visualizations for the visualization community, using the Padilla et al. [41] cognitive model as a guiding framework. By detailing examples of visualization research that illustrate each component of the model, this paper offers novel insights into how visualization researchers can utilize a cognitive framework to guide their user studies. We provide practical examples of each component of the model from empirical studies of visualizations, along with visualization implications of each cognitive process, which have not been directly addressed in prior work. Finally, this work offers a case study in utilizing an understanding of human cognition to generate a novel solution to a visualization reasoning bias in the context of hurricane forecast track visualizations.
more »
« less
This content will become publicly available on March 15, 2026
A Practical Tool for Visualizing and Measuring Model Selection Uncertainty
ABSTRACT Accounting for uncertainty in model selection is crucial for statistical inference and data‐driven decision‐making, particularly with high‐dimensional data. While multiple studies have focused on constructing model confidence sets, a practical and informative visualization tool to assist in decision‐making under such uncertainty has been lacking. This paper introduces an intuitive visualization tool, the graph of ranking from solution paths (GRASP), designed to provide instant insights into model selection uncertainty. Additionally, GRASP accounts for the uncertainty of variable importance, enabling decision‐makers to assess each variable under uncertainty. Based on an innovative selection procedure that utilizes the entire solution path, a feature importance score and bootstrap techniques, GRASP effectively visualizes the uncertainty of model selection, as demonstrated by our numerical examples. Furthermore, we propose a novel measure of uncertainty based on GRASP, providing a single‐number summary of selection uncertainty. This measure incorporates the concept of the flat norm, traditionally used in geometry and physics. Our simulation studies and numerical examples confirm that this measure accurately and robustly quantifies uncertainty.
more »
« less
- Award ID(s):
- 2053668
- PAR ID:
- 10650461
- Publisher / Repository:
- Wiley
- Date Published:
- Journal Name:
- Stat
- Volume:
- 14
- Issue:
- 2
- ISSN:
- 2049-1573
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Corlu, C G; Hunter, S R; Lam, H; Onggo, B S; Shortle, J; Biller, B (Ed.)Screening procedures for ranking and selection have received less attention than selection procedures, yet they serve as a cheap and powerful tool for decision making under uncertainty. Research on screening procedures has been less active in recent years, just as the advent of parallel computing has dramatically reshaped how selection procedures are designed and implemented. As a result, screening procedures used in modern practice continue to largely operate offline on fixed data. In this tutorial, we provide an overview of screening procedures with the goal of clarifying the current state of research and laying out opportunities for future development. We discuss several guarantees delivered by screening procedures and their role in different decision-making settings and investigate their impact on screening power and sampling efficiency in numerical experiments. We also study the implementation of screening procedures in parallel computing environments and how they can be combined with selection procedures.more » « less
-
Responding to the challenges of societal transformation in the face of climate change, efforts to integrate behaviorally rich models of adaptation decision-making into large-scale macroeconomic and Earth system models are growing and agent-based models (ABMs) are an effective tool for doing so. However, behavioral richness in ABMs has been limited to implementations of single decision models for all agents in a simulated population. The main goals of this study were to: 1) implement the ‘building-block processes’ (BBPs) approach for decision model heterogeneity; 2) demonstrate the application of sensitivity and uncertainty analyses to quantify the scope of structural uncertainty produced by alternative decision models under variable price and climate conditions; and 3) apply the Observing System Simulation Experiment (OSSE) approach to validate such a behaviorally rich BBPs model at the level of individual agent decisions. Using an ABM of agricultural producers’ decision-making, we demonstrated that uncertainty in crop and farm management decisions introduced by heterogeneous decision models was equal to and in some instances greater than that due to variable price or precipitation conditions. Unrealistically rapid or stagnant behavioral dynamics were evident in model versions implementing single decision models for all agents. Moreover, interactions among agents with diverse decision models in the same population produced consistently more accurate outcomes and realistic behavioral dynamics. The BBPs framework and accompanying sensitivity and uncertainty analyses demonstrated here offer a path forward for increasing behavioral richness in ABMs, which is key to understanding processes of adaptation central to societal responses to climate change.more » « less
-
This paper explores the micro Energy-Water-Hydrogen (m-EWH) nexus, an engineering system designed to reduce carbon emissions in the power sector. The m-EWH nexus leverages renewable energy sources (RES) to produce hydrogen via electrolysis, which is then combined with carbon captured from fossil fuel power plants to mitigate emissions. To address the uncertainty challenges posed by RES, this paper proposes a real-time decision-making framework for the m-EWH nexus, which requires the rapid solution of large-scale mixed-integer convex programming (MICP) problems. To this end, we develop a machine learning-accelerated solution method for real-time optimization (MARO), comprising three key modules: (1) an active constraint and integer variable prediction module that rapidly solves MICP problems using historical optimization data; (2) an optimal strategy selection module based on feasibility ranking to ensure solution feasibility; and (3) a feature space extension and refinement module to improve solution accuracy by generating new features and refining existing ones. The effectiveness of the MARO method is validated through two case studies of the m-EWH nexus, demonstrating its capability to swiftly and accurately solve MICP problems for this complex system.more » « less
-
The rise of machine learning-driven decision-making has sparked a growing emphasis on algorithmic fairness. Within the realm of clustering, the notion of balance is utilized as a criterion for attaining fairness, which characterizes a clustering mechanism as fair when the resulting clusters maintain a consistent proportion of observations representing individuals from distinct groups delineated by protected attributes. Building on this idea, the literature has rapidly incorporated a myriad of extensions, devising fair versions of the existing frequentist clustering algorithms, e.g., k-means, k-medioids, etc., that aim at minimizing specific loss functions. These approaches lack uncertainty quantification associated with the optimal clustering configuration and only provide clustering boundaries without quantifying the probabilities associated with each observation belonging to the different clusters. In this article, we intend to offer a novel probabilistic formulation of the fair clustering problem that facilitates valid uncertainty quantification even under mild model misspecifications, without incurring substantial computational overhead. Mixture model-based fair clustering frameworks facilitate automatic uncertainty quantification, but tend to showcase brittleness under model misspecification and involve significant computational challenges. To circumnavigate such issues, we propose a generalized Bayesian fair clustering framework that inherently enjoys decision-theoretic interpretation. Moreover, we devise efficient computational algorithms that crucially leverage techniques from the existing literature on optimal transport and clustering based on loss functions. The gain from the proposed technology is showcased via numerical experiments and real data examples.more » « less
