An official website of the United States government
Advances in speech technologies and generative AI (Gen AI) have enabled the possibility of generating conversational cues to improve engagement and creativity during small-group discussions. Such cues could contextually adapt and guide a live conversation, or conversely, serve as a distraction. How do conversational cues impact ideation and social interaction? How does the meeting modality impact the effectiveness of cues? We built a system, CueTip, to generate and deliver real-time contextual conversational cues using the GPT 4o-mini model. In a 2x2 study, N=172 participants in dyads completed a brainstorming task where they received cues or not during either a virtual or in-person discussion. Cued participants’ ideas and discussions were more topically diverse than Non-Cued participants. In-person groups noticed slightly more cues on average than virtual groups. We discuss implications for designing effective conversational cues.
more »
« less
‘More like a person than reading text in a machine’: Characterizing User Choice of Embodied Agents vs. Conventional GUIs on Smartphones
Embodied conversational agents (ECAs) provide an interface modality on smartphones that may be particularly effective for tasks with significant social, affective, reflective, and narrative aspects, such as health education and behavior change counseling. However, the conversational medium is significantly slower than conventional graphical user interfaces (GUIs) for brief, time-sensitive tasks. We conducted a randomized experiment to determine user preferences in performing two kinds of health-related tasks—one affective and narrative in nature and one transactional—and gave participants a choice of a conventional GUI or a functionally equivalent ECA on a smartphone to complete the task. We found significant main effects of task type and user preference on user choice of modality, with participants choosing the conventional GUI more often for transactional and time-sensitive tasks.
more »
« less
- Award ID(s):
- 1831755
- PAR ID:
- 10295857
- Date Published:
- Journal Name:
- CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
- Page Range / eLocation ID:
- 1 to 6
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)A major problem in task-oriented conversational agents is the lack of support for the repair of conversational breakdowns. Prior studies have shown that current repair strategies for these kinds of errors are often ineffective due to: (1) the lack of transparency about the state of the system's understanding of the user's utterance; and (2) the system's limited capabilities to understand the user's verbal attempts to repair natural language understanding errors. This paper introduces SOVITE, a new multi-modal speech plus direct manipulation interface that helps users discover, identify the causes of, and recover from conversational breakdowns using the resources of existing mobile app GUIs for grounding. SOVITE displays the system's understanding of user intents using GUI screenshots, allows users to refer to third-party apps and their GUI screens in conversations as inputs for intent disambiguation, and enables users to repair breakdowns using direct manipulation on these screenshots. The results from a remote user study with 10 users using SOVITE in 7 scenarios suggested that SOVITE's approach is usable and effective.more » « less
-
In affect-aware task adaptation, users' psychological states are recognized with diverse measurements and used to adapt computer-based tasks. User experience with such adaptation improves as the accuracy of psychological state recognition and task adaptation increases. However, it is unclear how user experience is influenced by algorithmic transparency: the degree to which users understand the computer's decision-making process. We thus created an affect-aware task adaptation system with 4 algorithmic transparency levels (none/low/medium/high) and conducted a study where 93 participants first experienced adaptation with no transparency for 16 minutes, then with one of the other 3 levels for 16 minutes. User experience questionnaires and physiological measurements (respiration, skin conductance, heart rate) were analyzed with mixed 2×3 analyses of variance (time × transparency group). Self-reported interest/enjoyment and competence were lower with low transparency than with medium/high transparency, but did not differ between medium and high transparency. The transparency level may also influence participants' respiratory responses to adaptation errors, but this finding is based on ad-hoc t-tests and should be considered preliminary. Overall, results show that the degree of algorithmic transparency does influence self-reported user experience. Since transparency information is relatively easy to provide, it may represent a worthwhile design element in affective computing.more » « less
-
In affective computing, classification algorithms are used to recognize users’ psychological states and adapt tasks to optimize user experience. However, classification is never perfect, and the relationship between adaptation accuracy and user experience remains understudied. It is also unclear whether the adaptation magnitude (‘size’ of action taken to influence user states) influences effects of adaptation accuracy. To evaluate impacts of adaptation accuracy (appropriate vs. inappropriate actions) and magnitude on user experience, we conducted a ‘Wizard of Oz’ study where 112 participants interacted with the Multi-Attribute Task Battery over three 11-minute intervals. An adaptation accuracy (50 % to 80 %) was preassigned for the first 11-minute interval, and accuracy increased by 10 % in each subsequent interval. Task difficulty changed every minute, and participant preferences for difficulty changes were assessed at the same time. Adaptation accuracy was artificially induced by fixing the percentage of times the difficulty changes matched participant preferences. Participants were also randomized to two magnitude conditions, with difficulty modified by 1 (low) or 3 (high) levels each minute. User experience metrics were assessed after each interval. Analysis with latent growth models offered support for linear increases in user experience across increasing levels of adaptation accuracy. For each 10 % gain in accuracy, results indicate a 1.3 (95 % CI [.35, 2.20]) point increase in NASA Task Load Index scores (range 6–60), a 0.40 (95 % CI [.18, 0.57]) increase in effort/importance (range 2–14), and 0.48 (95 % CI [.24, 0.72]) increase in perceived competence (range 2–14). Furthermore, the effect of accuracy on Task Load Index scores was modulated by adaptation magnitude. No effects were observed for interest/enjoyment or pressure/tension. By providing quantitative estimates of effects of adaptation accuracy on user experience, the study provides guidelines for researchers and developers of affect-aware technologies. Furthermore, our methods could be adapted for use in other affective computing scenarios.more » « less
-
null (Ed.)Representing the semantics of GUI screens and components is crucial to data-driven computational methods for modeling user-GUI interactions and mining GUI designs. Existing GUI semantic representations are limited to encoding either the textual content, the visual design and layout patterns, or the app contexts. Many representation techniques also require significant manual data annotation efforts. This paper presents Screen2Vec, a new self-supervised technique for generating representations in embedding vectors of GUI screens and components that encode all of the above GUI features without requiring manual annotation using the context of user interaction traces. Screen2Vec is inspired by the word embedding method Word2Vec, but uses a new two-layer pipeline informed by the structure of GUIs and interaction traces and incorporates screen- and app-specific metadata. Through several sample downstream tasks, we demonstrate Screen2Vec’s key useful properties: representing between-screen similarity through nearest neighbors, composability, and capability to represent user tasks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3411763.3451664
