skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Data@Hand: Fostering Visual Exploration of Personal Data on Smartphones Leveraging Speech and Touch Interaction
Most mobile health apps employ data visualization to help people view their health and activity data, but these apps provide limited support for visual data exploration. Furthermore, despite its huge potential benefits, mobile visualization research in the personal data context is sparse. This work aims to empower people to easily navigate and compare their personal health data on smartphones by enabling flexible time manipulation with speech. We designed and developed Data@Hand, a mobile app that leverages the synergy of two complementary modalities: speech and touch. Through an exploratory study with 13 long-term Fitbit users, we examined how multimodal interaction helps participants explore their own health data. Participants successfully adopted multimodal interaction (i.e., speech and touch) for convenient and fluid data exploration. Based on the quantitative and qualitative findings, we discuss design implications and opportunities with multimodal interaction for better supporting visual data exploration on mobile devices.  more » « less
Award ID(s):
1753452
PAR ID:
10315476
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
2021 CHI Conference on Human Factors in Computing Systems
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Proceedings of the ACM on Human-Computer Interaction)
    Speech as a natural and low-burden input modality has great potential to support personal data capture. However, little is known about how people use speech input, together with traditional touch input, to capture different types of data in self-tracking contexts. In this work, we designed and developed NoteWordy, a multimodal self-tracking application integrating touch and speech input, and deployed it in the context of productivity tracking for two weeks (N = 17). Our participants used the two input modalities differently, depending on the data type as well as personal preferences, error tolerance for speech recognition issues, and social surroundings. Additionally, we found speech input reduced participants' diary entry time and enhanced the data richness of the free-form text. Drawing from the findings, we discuss opportunities for supporting efficient personal data capture with multimodal input and implications for improving the user experience with natural language input to capture various self-tracking data. 
    more » « less
  2. ; ; ; ; (, Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems)
    Eating disorders (EDs) constitute a mental illness with the highest mortality. Today, mobile health apps provide promising means to ED patients for managing their condition. Apps enable users to monitor their eating habits, thoughts, and feelings, and offer analytic insights for behavior change. However, not only have scholars critiqued the clinical validity of these apps, their underlying design principles are not well understood. Through a review of 34 ED apps, we uncovered 11 different data types ED apps collect, and 9 strategies they employ to support collection and reflection. Drawing upon personal health informatics and visualization frameworks, we found that most apps did not adhere to best practices on what and how data should be collected from and reflected to users, or how data-driven insights should be communicated. Our review offers suggestions for improving the design of ED apps such that they can be useful and meaningful in ED recovery. 
    more » « less
  3. ; ; ; ; ; ; (, Proceedings of the ACM on Human-Computer Interaction)
    Data physicalizations (3D printed terrain models, anatomical scans, or even abstract data) can naturally engage both the visual and haptic senses in ways that are difficult or impossible to do with traditional planar touch screens and even immersive digital displays. Yet, the rigid 3D physicalizations produced with today's most common 3D printers are fundamentally limited for data exploration and querying tasks that require dynamic input (e.g., touch sensing) and output (e.g., animation), functions that are easily handled with digital displays. We introduce a novel style of hybrid virtual + physical visualization designed specifically to support interactive data exploration tasks. Working toward a "best of both worlds" solution, our approach fuses immersive AR, physical 3D data printouts, and touch sensing through the physicalization. We demonstrate that this solution can support three of the most common spatial data querying interactions used in scientific visualization (streamline seeding, dynamic cutting places, and world-in-miniature visualization). Finally, we present quantitative performance data and describe a first application to exploratory visualization of an actively studied supercomputer climate simulation data with feedback from domain scientists. 
    more » « less
  4. ; ; ; ; ; (, Journal of Medical Internet Research)
    Background: Personal health technologies, including wearable tracking devices and mobile apps, have great potential to equip the general population with the ability to monitor and manage their health. However, being designed for sighted people, much of their functionality is largely inaccessible to the blind and low-vision (BLV) population, threatening the equitable access to personal health data (PHD) and health care services. Objective: This study aims to understand why and how BLV people collect and use their PHD and the obstacles they face in doing so. Such knowledge can inform accessibility researchers and technology companies of the unique self-tracking needs and accessibility challenges that BLV people experience. Methods: We conducted a web-based and phone survey with 156 BLV people. We reported on quantitative and qualitative findings regarding their PHD tracking practices, needs, accessibility barriers, and work-arounds. Results: BLV respondents had strong desires and needs to track PHD, and many of them were already tracking their data despite many hurdles. Popular tracking items (ie, exercise, weight, sleep, and food) and the reasons for tracking were similar to those of sighted people. BLV people, however, face many accessibility challenges throughout all phases of self-tracking, from identifying tracking tools to reviewing data. The main barriers our respondents experienced included suboptimal tracking experiences and insufficient benefits against the extended burden for BLV people. Conclusions: We reported the findings that contribute to an in-depth understanding of BLV people’s motivations for PHD tracking, tracking practices, challenges, and work-arounds. Our findings suggest that various accessibility challenges hinder BLV individuals from effectively gaining the benefits of self-tracking technologies. On the basis of the findings, we discussed design opportunities and research areas to focus on making PHD tracking technologies accessible for all, including BLV people. 
    more » « less
  5. ; ; ; (, JMIR Mental Health)
    Background Supporting mental health and wellness is of increasing interest due to a growing recognition of the prevalence and burden of mental health issues. Mood is a central aspect of mental health, and several technologies, especially mobile apps, have helped people track and understand it. However, despite formative work on and dissemination of mood-tracking apps, it is not well understood how mood-tracking apps used in real-world contexts might benefit people and what people hope to gain from them. Objective To address this gap, the purpose of this study was to understand motivations for and experiences in using mood-tracking apps from people who used them in real-world contexts. Methods We interviewed 22 participants who had used mood-tracking apps using a semistructured interview and card sorting task. The interview focused on their experiences using a mood-tracking app. We then conducted a card sorting task using screenshots of various data entry and data review features from mood-tracking apps. We used thematic analysis to identify themes around why people use mood-tracking apps, what they found useful about them, and where people felt these apps fell short. Results Users of mood-tracking apps were primarily motivated by negative life events or shifts in their own mental health that prompted them to engage in tracking and improve their situation. In general, participants felt that using a mood-tracking app facilitated self-awareness and helped them to look back on a previous emotion or mood experience to understand what was happening. Interestingly, some users reported less inclination to document their negative mood states and preferred to document their positive moods. There was a range of preferences for personalization and simplicity of tracking. Overall, users also liked features in which their previous tracked emotions and moods were visualized in figures or calendar form to understand trends. One gap in available mood-tracking apps was the lack of app-facilitated recommendations or suggestions for how to interpret their own data or improve their mood. Conclusions Although people find various features of mood-tracking apps helpful, the way people use mood-tracking apps, such as avoiding entering negative moods, tracking infrequently, or wanting support to understand or change their moods, demonstrate opportunities for improvement. Understanding why and how people are using current technologies can provide insights to guide future designs and implementations. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email