More Like this

1. Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco

   https://doi.org/http://doi.ieeecomputersociety.org/10.1109/TVCG.2018.2865240  

   Moritz, Dominik; Wang, Chenglong; Nelson, Gregory L; Lin, Haiden; Smith, Adam M; Howe, Bill; Heer, Jeff (July 2018, IEEE transactions on visualization and computer graphics)

   There exists a gap between visualization design guidelines and their application in visualization tools. While empirical studies can provide design guidance, we lack a formal framework for representing design knowledge, integrating results across studies, and applying this knowledge in automated design tools that promote effective encodings and facilitate visual exploration. We propose modeling visualization design knowledge as a collection of constraints, in conjunction with a method to learn weights for soft constraints from experimental data. Using constraints, we can take theoretical design knowledge and express it in a concrete, extensible, and testable form: the resulting models can recommend visualization designs and can easily be augmented with additional constraints or updated weights. We implement our approach in Draco, a constraint-based system based on Answer Set Programming (ASP). We demonstrate how to construct increasingly sophisticated automated visualization design systems, including systems based on weights learned directly from the results of graphical perception experiments. 

   more » « less
2. Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco

   Moritz, Dominik; Wang, Chenglong; Nelson, Greg L.; Lin, Halden; Smith, Adam M.; Howe, Bill; Heer, Jeffrey (April 2019, InfoVis)

   There exists a gap between visualization design guidelines and their application in visualization tools. While empirical studies can provide design guidance, we lack a formal framework for representing design knowledge, integrating results across studies, and applying this knowledge in automated design tools that promote effective encodings and facilitate visual exploration. We propose modeling visualization design knowledge as a collection of constraints, in conjunction with a method to learn weights for soft constraints from experimental data. Using constraints, we can take theoretical design knowledge and express it in a concrete, extensible, and testable form: the resulting models can recommend visualization designs and can easily be augmented with additional constraints or updated weights. We implement our approach in Draco, a constraint-based system based on Answer Set Programming (ASP). We demonstrate how to construct increasingly sophisticated automated visualization design systems, including systems based on weights learned directly from the results of graphical perception experiments. 

   more » « less
3. My Model is Unfair, Do People Even Care? Visual Design Affects Trust and Perceived Bias in Machine Learning

   https://doi.org/10.1109/TVCG.2023.3327192  

   Gaba, Aimen; Kaufman, Zhanna; Cheung, Jason; Shvakel, Marie; Hall, Kyle Wm; Brun, Yuriy; Bearfield, Cindy Xiong (January 2024, IEEE transactions on visualization and computer graphics)

   Machine learning technology has become ubiquitous, but, unfortunately, often exhibits bias. As a consequence, disparate stakeholders need to interact with and make informed decisions about using machine learning models in everyday systems. Visualization technology can support stakeholders in understanding and evaluating trade-offs between, for example, accuracy and fairness of models. This paper aims to empirically answer “Can visualization design choices affect a stakeholder's perception of model bias, trust in a model, and willingness to adopt a model?” Through a series of controlled, crowd-sourced experiments with more than 1,500 participants, we identify a set of strategies people follow in deciding which models to trust. Our results show that men and women prioritize fairness and performance differently and that visual design choices significantly affect that prioritization. For example, women trust fairer models more often than men do, participants value fairness more when it is explained using text than as a bar chart, and being explicitly told a model is biased has a bigger impact than showing past biased performance. We test the generalizability of our results by comparing the effect of multiple textual and visual design choices and offer potential explanations of the cognitive mechanisms behind the difference in fairness perception and trust. Our research guides design considerations to support future work developing visualization systems for machine learning. 

   more » « less
4. VisuaLint: Sketchy In Situ Annotations of Chart Construction Errors

   https://doi.org/10.1111/cgf.13975  

   Hopkins, Aspen K; Correll, Michael; Satyanarayan, Arvind. (January 2020, Computer graphics forum)

   Chart construction errors, such as truncated axes or inexpressive visual encodings, can hinder reading a visualization, or worse, imply misleading facts about the underlying data. These errors can be caught by critical readings of visualizations, but readers must have a high level of data and design literacy and must be paying close attention. To address this issue, we introduce VisuaLint: a technique for surfacing chart construction errors in situ. Inspired by the ubiquitous red wavy underline that indicates spelling mistakes, visualization elements that contain errors (e.g., axes and legends) are sketchily rendered and accompanied by a concise annotation. VisuaLint is unobtrusive — it does not interfere with reading a visualization — and its di- rect display establishes a close mapping between erroneous elements and the expression of error. We demonstrate five examples of VisualLint and present the results of a crowdsourced evaluation (N = 62) of its efficacy. These results contribute an empiri- cal baseline proficiency for recognizing chart construction errors, and indicate near-universal difficulty in error identification. We find that people more reliably identify chart construction errors after being shown examples of VisuaLint, and prefer more verbose explanations for unfamiliar or less obvious flaws. 

   more » « less
5. CALVI: Critical Thinking Assessment for Literacy in Visualizations

   https://doi.org/10.1145/3544548.3581406  

   Ge, Lily W.; Cui, Yuan; Kay, Matthew (April 2023, CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems)

   Visualization misinformation is a prevalent problem, and combating it requires understanding people’s ability to read, interpret, and reason about erroneous or potentially misleading visualizations, which lacks a reliable measurement: existing visualization literacy tests focus on well-formed visualizations. We systematically develop an assessment for this ability by: (1) developing a precise definition of misleaders (decisions made in the construction of visualizations that can lead to conclusions not supported by the data), (2) constructing initial test items using a design space of misleaders and chart types, (3) trying out the provisional test on 497 participants, and (4) analyzing the test tryout results and refining the items using Item Response Theory, qualitative analysis, a wrong-due-to-misleader score, and the content validity index. Our final bank of 45 items shows high reliability, and we provide item bank usage recommendations for future tests and different use cases. Related materials are available at: https://osf.io/pv67z/. 

   more » « less