skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Using Close Reading as a Method for Evaluating Visualizations
Visualization research and practice that incorporates the arts make claims to being more effective in connecting with users on a human level. However, these claims are difficult to measure quantitatively. In this paper, we present a follow-on study to use close reading, a humanities method from literary studies, to evaluate visualizations created using artistic processes [Bares 2020]. Close reading is a method in literary studies that we've previously explored as a method for evaluating visualizations. To use close reading as an evaluation method, we guide participants through a series of steps designed to prompt them to interpret the visualization's formal, informational, and contextual features. Here we elaborate on our motivations for using close reading as a method to evaluate visualizations, and enumerate the procedures we used in the study to evaluate a 2D visualization, including modifications made because of the COVID-19 pandemic. Key findings of this study include that close reading is an effective formative method to elicit information related to interpretation and critique; user subject position; and suspicion or skepticism. Information gained through close reading is valuable in the visualization design and iteration processes, both related to designing features and other formal elements more effectively, as well as in considering larger questions of context and framing.  more » « less
Award ID(s):
1704604
PAR ID:
10252329
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2020 IEEE Workshop on Evaluation and Beyond - Methodological Approaches to Visualization (BELIV)
Page Range / eLocation ID:
29 to 37
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Many studies have claimed to find that reading fiction leads to improvements in social cognition. But this work has left open the critical question of whether any type of narrative, fictional or nonfictional, might have similar effects. To address this question, as well as to test whether framing a narrative as fiction matters, the current studies presented participants ( N  = 268 in Study 1; N  = 362 in Study 2) with literary fiction texts, narrative nonfiction texts, expository nonfiction texts, or no texts. We tested their theory-of-mind abilities using the picture-based Reading the Mind in the Eyes task and a text-based test of higher-order social cognition. Reading anything was associated with higher scores compared to reading nothing, but the effects of framing and text type were inconsistent. These results suggest that prior claims regarding positive effects of reading fiction on mentalizing should be seen as tenuous; other mechanisms may be driving previously published effects. 
    more » « less
  2. Data literacy, an important goal for social studies education, involves teaching students how to comprehend, analyze, interpret, evaluate, create, and argue with data and data visualizations such as timelines, maps, and graphs. Digital data visualizations support rapid inquiry and explorations that would be difficult on paper - such as adding data to an existing data visualization or creating multiple data visualizations of the same variable at multiple times or places for easy comparison. The process of asking questions and finding answers through data exploration is an important part of social studies education. While the authors recognize that there are many online data visualization tools that can be used in classrooms, they designed their tools, DV4L and Timeline Builder, specifically for use in social studies classrooms and based on feedback from prospective and current teachers. The authors have taken care to include features that teachers said would be useful in social studies data inquiry lessons and activities. These are still in a prototype phase, but are free to use and online. 
    more » « less
  3. Three-dimensional (3D) holographic technology, like virtual, augmented, and mixed reality technology, is an emerging technology designed to improve learning outcomes in science, technology, engineering, and math (STEM) disciplines. Holograms provide unique opportunities to enhance students’ understanding of intractable concepts and processes using engaging visualization methods. Portable 3D holographic fans allow for the improved visualization of molecules, structures, pathways, and other STEM - related content that have the potential to elevate information acquisition in novel ways that extend beyond 2-D presentations and textbook figures. While the potentiality of this innovative technology is exciting, adopting 3D holographic materials in the STEM pedagogical and research environment requires producing literary evidence to justify usage in specific contexts and sufficient guidance on safety protocols. A review of 3D hologram technology revealed an inadequate amount of efficacy research. Quantitative and qualitative research studies involving STEM majors, faculty, and researchers constitute the engine that will drive the utilization of 3D hologram visualizations in STEM undergraduate, graduate, and professional school classrooms and laboratories. The current article reviews relevant research findings and discusses the potential impacts of 3D hologram technology in teaching, research, distance learning, and medical contexts. 
    more » « less
  4. Three-dimensional (3D) holographic technology, like virtual, augmented, and mixed reality technology, is an emerging technology designed to improve learning outcomes in science, technology, engineering, and math (STEM) disciplines. Holograms provide unique opportunities to enhance students’ understanding of intractable concepts and processes using engaging visualization methods. Portable 3D holographic fans allow for the improved visualization of molecules, structures, pathways, and other STEM-related content that have the potential to elevate information acquisition in novel ways that extend beyond 2-D presentations and textbook figures. While the potentiality of this innovative technology is exciting, adopting 3D holographic materials in the STEM pedagogical and research environment requires producing literary evidence to justify usage in specific contexts and sufficient guidance on safety protocols. A review of 3D hologram technology revealed an inadequate amount of efficacy research. Quantitative and qualitative research studies involving STEM majors, faculty, and researchers constitute the engine that will drive the utilization of 3D hologram visualizations in STEM undergraduate, graduate, and professional school classrooms and laboratories. The current article reviews relevant research findings and discusses the potential impacts of 3D hologram technology in teaching, research, distance learning, and medical contexts. 
    more » « less
  5. People naturally bring their prior beliefs to bear on how they interpret the new information, yet few formal models exist for accounting for the influence of users' prior beliefs in interactions with data presentations like visualizations. We demonstrate a Bayesian cognitive model for understanding how people interpret visualizations in light of prior beliefs and show how this model provides a guide for improving visualization evaluation. In a first study, we show how applying a Bayesian cognition model to a simple visualization scenario indicates that people's judgments are consistent with a hypothesis that they are doing approximate Bayesian inference. In a second study, we evaluate how sensitive our observations of Bayesian behavior are to different techniques for eliciting people subjective distributions, and to different datasets. We find that people don't behave consistently with Bayesian predictions for large sample size datasets, and this difference cannot be explained by elicitation technique. In a final study, we show how normative Bayesian inference can be used as an evaluation framework for visualizations, including of uncertainty. 
    more » « less