An official website of the United States government
In the field of visualization, understanding users’ analytical reasoning is important for evaluating the effectiveness of visualization applications. Several studies have been conducted to capture and analyze user interactions to comprehend this reasoning process. However, few have successfully linked these interactions to users’ reasoning processes. This paper introduces an approach that addresses the limitation by correlating semantic user interactions with analysis decisions using an interactive wire transaction analysis system and a visual state transition matrix, both designed as visual analytics applications. The system enables interactive analysis for evaluating financial fraud in wire transactions. It also allows mapping captured user interactions and analytical decisions back onto the visualization to reveal their decision differences. The visual state transition matrix further aids in understanding users’ analytical flows, revealing their decision-making processes. Classification machine learning algorithms are applied to evaluate the effectiveness of our approach in understanding users’ analytical reasoning process by connecting the captured semantic user interactions to their decisions (i.e., suspicious, not suspicious, and inconclusive) on wire transactions. With the algorithms, an average of 72% accuracy is determined to classify the semantic user interactions. For classifying individual decisions, the average accuracy is 70%. Notably, the accuracy for classifying ‘inconclusive’ decisions is 83%. Overall, the proposed approach improves the understanding of users’ analytical decisions and provides a robust method for evaluating user interactions in visualization tools.
more »
« less
Odds and Insights: Decision Quality in Exploratory Data Analysis Under Uncertainty
Recent studies have shown that users of visual analytics tools can have difficulty distinguishing robust findings in the data from statistical noise, but the true extent of this problem is likely dependent on both the incentive structure motivating their decisions, and the ways that uncertainty and variability are (or are not) represented in visualisations. In this work, we perform a crowd-sourced study measuring decision-making quality in visual analytics, testing both an explicit structure of incentives designed to reward cautious decision-making as well as a variety of designs for communicating uncertainty. We find that, while participants are unable to perfectly control for false discoveries as well as idealised statistical models such as the Benjamini-Hochberg, certain forms of uncertainty visualisations can improve the quality of participants’ decisions and lead to fewer false discoveries than not correcting for multiple comparisons. We conclude with a call for researchers to further explore visual analytics decision quality under different decision-making contexts, and for designers to directly present uncertainty and reliability information to users of visual analytics tools. The supplementary materials are available at: https://osf.io/xtsfz/.
more »
« less
- Award ID(s):
- 2211939
- PAR ID:
- 10547799
- Publisher / Repository:
- Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
- Date Published:
- ISBN:
- 9798400703300
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Objective In this study, we extend the impact of mindfulness to the concept of least-worst decision-making. Least-worst decisions involve high-uncertainty and require the individual to choose between a number of potentially negative courses of action. Research is increasingly exploring least-worst decisions, and real-world events (such as the COVID-19 pandemic) show the need for individuals to overcome uncertainty and commit to a least-worst course of action. From sports to business, researchers are increasingly showing that “being mindful” has a range of positive performance-related benefits. We hypothesized that mindfulness would improve least-worst decision-making because it would increase self-reflection and value identification. However, we also hypothesized that trait maximization (the tendency to attempt to choose the “best” course of action) would negatively interact with mindfulness. Methods Three hundred and ninety-eight participants were recruited using Amazon MTurk and exposed to a brief mindfulness intervention or a control intervention (listening to an audiobook). After this intervention, participants completed the Least-Worst Uncertain Choice Inventory for Emergency Responders (LUCIFER). Results As hypothesized, mindfulness increased decision-making speed and approach-tendencies. Conversely, for high-maximizers, increased mindfulness caused a slowing of the decision-making process and led to more avoidant choices. Conclusions This study shows the potential positive and negative consequences of mindfulness for least-worst decision-making, emphasizing the critical importance of individual differences when considering both the effect of mindfulness and interventions aimed at improving decision-making.more » « less
-
Abstract Contemporary wildlife disease management is complex because managers need to respond to a wide range of stakeholders, multiple uncertainties, and difficult trade‐offs that characterize the interconnected challenges of today. Despite general acknowledgment of these complexities, managing wildlife disease tends to be framed as a scientific problem, in which the major challenge is lack of knowledge. The complex and multifactorial process of decision‐making is collapsed into a scientific endeavor to reduce uncertainty. As a result, contemporary decision‐making may be oversimplified, rely on simple heuristics, and fail to account for the broader legal, social, and economic context in which the decisions are made. Concurrently, scientific research on wildlife disease may be distant from this decision context, resulting in information that may not be directly relevant to the pertinent management questions. We propose reframing wildlife disease management challenges as decision problems and addressing them with decision analytical tools to divide the complex problems into more cognitively manageable elements. In particular, structured decision‐making has the potential to improve the quality, rigor, and transparency of decisions about wildlife disease in a variety of systems. Examples of management of severe acute respiratory syndrome coronavirus 2, white‐nose syndrome, avian influenza, and chytridiomycosis illustrate the most common impediments to decision‐making, including competing objectives, risks, prediction uncertainty, and limited resources.more » « less
-
In this review, we discuss the role of sensor analytics point solutions (SNAPS), a reduced complexity machine-assisted decision support tool. We summarize the approaches used for mobile phone-based chemical/biological sensors, including general hardware and software requirements for signal transduction and acquisition. We introduce SNAPS, part of a platform approach to converge sensor data and analytics. The platform is designed to consist of a portfolio of modular tools which may lend itself to dynamic composability by enabling context-specific selection of relevant units, resulting in case-based working modules. SNAPS is an element of this platform where data analytics, statistical characterization and algorithms may be delivered to the data either via embedded systems in devices, or sourced, in near real-time, from mist, fog or cloud computing resources. Convergence of the physical systems with the cyber components paves the path for SNAPS to progress to higher levels of artificial reasoning tools (ART) and emerge as data-informed decision support, as a service for general societal needs. Proof of concept examples of SNAPS are demonstrated both for quantitative data and qualitative data, each operated using a mobile device (smartphone or tablet) for data acquisition and analytics. We discuss the challenges and opportunities for SNAPS, centered around the value to users/stakeholders and the key performance indicators users may find helpful, for these types of machine-assisted tools.more » « less
-
The combination of Visual Guidance and Extended Reality (XR) technology holds the potential to greatly improve the performance of human workforces in numerous areas, particularly industrial environments. Focusing on virtual assembly tasks and making use of different forms of supportive visualisations, this study investigates the potential of XR Visual Guidance. Set in a web-based immersive environment, our results draw from a heterogeneous pool of 199 participants. This research is designed to significantly differ from previous exploratory studies, which yielded conflicting results on user performance and associated human factors. Our results clearly show the advantages of XR Visual Guidance based on an over 50% reduction in task completion times and mistakes made; this may further be enhanced and refined using specific frameworks and other forms of visualisations/Visual Guidance. Discussing the role of other factors, such as cognitive load, motivation, and usability, this paper also seeks to provide concrete avenues for future research and practical takeaways for practitioners.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
