An official website of the United States government
Speech as a natural and low-burden input modality has great potential to support personal data capture. However, little is known about how people use speech input, together with traditional touch input, to capture different types of data in self-tracking contexts. In this work, we designed and developed NoteWordy, a multimodal self-tracking application integrating touch and speech input, and deployed it in the context of productivity tracking for two weeks (N = 17). Our participants used the two input modalities differently, depending on the data type as well as personal preferences, error tolerance for speech recognition issues, and social surroundings. Additionally, we found speech input reduced participants' diary entry time and enhanced the data richness of the free-form text. Drawing from the findings, we discuss opportunities for supporting efficient personal data capture with multimodal input and implications for improving the user experience with natural language input to capture various self-tracking data.
more »
« less
FoodScrap: Promoting Rich Data Capture and Reflective Food Journaling Through Speech Input
The factors influencing people’s food decisions, such as one’s mood and eating environment, are important information to foster self-reflection and to develop personalized healthy diet. But, it is difficult to consistently collect them due to the heavy data capture burden. In this work, we examine how speech input supports capturing everyday food practice through a week-long data collection study (N = 11). We deployed FoodScrap, a speech-based food journaling app that allows people to capture food components, preparation methods, and food decisions. Using speech input, participants detailed their meal ingredients and elaborated their food decisions by describing the eating moments, explaining their eating strategy, and assessing their food practice. Participants recognized that speech input facilitated self-reflection, but expressed concerns around re-recording, mental load, social constraints, and privacy. We discuss how speech input can support low-burden and reflective food journaling and opportunities for effectively processing and presenting large amounts of speech data.
more »
« less
- Award ID(s):
- 1753452
- PAR ID:
- 10315477
- Date Published:
- Journal Name:
- DIS '21: Designing Interactive Systems Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Prior work on novice programmers' self-regulation have shown it to be inconsistent and shallow, but trainable through direct instruction. However, prior work has primarily studied self-regulation retrospectively, which relies on students to remember how they regulated their process, or in laboratory settings, limiting the ecological validity of findings. To address these limitations, we investigated 31 novice programmers' self-regulation in situ over 10 weeks. We had them to keep journals about their work and later had them to reflect on their journaling. Through a series of qualitative analyses of journals and survey responses, we found that all participants monitored their process and evaluated their work, that few interpreted the problems they were solving or adapted prior solutions. We also found that some students self-regulated their programming in many ways, while others in almost none. Students reported many difficulties integrating reflection into their work; some were completely unaware of their process, some struggled to integrate reflection into their process, and others found reflection conflicted with their work. These results suggest that self-regulation during programming is highly variable in practice, and that teaching self-regulation skills to improve programming outcomes may require differentiated instruction based on students self-awareness and existing programming practices.more » « less
-
The respective benefits and drawbacks of manual food journaling and automated dietary monitoring (ADM) suggest the value of semi-automated journaling systems combining the approaches. However, the current understanding of how people anticipate strategies for implementing semi-automated food journaling systems is limited. We therefore conduct a speculative survey study with 600 responses, examining how people anticipate approaches to automatic capture and prompting for details. Participants feel the location and detection capability of ADM sensors influences anticipated physical, social, and privacy burdens. People more positively anticipate prompts which contain information relevant to their journaling goals, help them recall what they ate, and are quick to respond to. Our work suggests a tradeoff between ADM systems' detection performance and anticipated acceptability, with sensors on facial areas having higher performance but lower acceptability than sensors in other areas and more usable prompting methods like those containing specific foods being more challenging to produce than manual reminders. We suggest opportunities to improve higher-acceptability, lower-accuracy ADM sensors, select approaches based on individual and practitioner journaling needs, and better describe capabilities to potential users.more » « less
-
Studies of personal informatics systems primarily examine people's use or non-use, but people often leverage other technology towards their long-term behavior change processes such as social platforms. We explore how tracking technologies and social platforms together help people build healthy eating behaviors by interviewing 18 people who use Chinese food journaling apps. We contribute a Model of Socially Sustained Self-Tracking in personal informatics, building on the past model of Personal Informatics and the learning components of Social Cognitive Theory. The model illustrates how people get advice from social platforms on when and how to track, transfer data to and apply knowledge from social platforms, evolve to use social platforms after tracking, and occasionally resume using tracking tools. Observational learning and enactive learning are central to these processes, with social technologies helping people to gain deeper and more reliable domain knowledge. We discuss how lapsing and abandoning of tracking can be viewed as evolving to social platforms, offering recommendations for how technology can better facilitate this evolution.more » « less
-
This Innovative Practice work-in-progress describes the use of collaborative autoethnography (CAE) as a methodology to explore the centrality of Whiteness (as an ideology) in engineering and how it informs the culture, climate, and discourse of engineering education. We began the first year of our project by conducting a CAE on our own experiences in engineering spaces. CAE takes a collaborative approach to the process of critical self-reflection and can be conducted in many forms, such as collecting personal memory data (e.g., journaling), interviewing each other, facilitating intentional dialogue, or observing each other (e.g., in the classroom). Our team’s diverse viewpoints facilitated rich conversations that let us interrogate the ways in which Whiteness reveals its form differently depending on one’s positionality and ontology. In this paper, we describe our approach, experiences, and challenges with using CAE to explore our engineering journeys. Future steps include using our findings to create a faculty development program to help engineering faculty further their development of critical consciousness. For movement towards racial equity in engineering to be effective and sustainable, we believe change must begin with action in the classroom, where engineering faculty have direct interactions and influence over students’ beliefs, attitudes, and value systems.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3461778.3462074
