An official website of the United States government
In affective computing, classification algorithms are used to recognize users’ psychological states and adapt tasks to optimize user experience. However, classification is never perfect, and the relationship between adaptation accuracy and user experience remains understudied. It is also unclear whether the adaptation magnitude (‘size’ of action taken to influence user states) influences effects of adaptation accuracy. To evaluate impacts of adaptation accuracy (appropriate vs. inappropriate actions) and magnitude on user experience, we conducted a ‘Wizard of Oz’ study where 112 participants interacted with the Multi-Attribute Task Battery over three 11-minute intervals. An adaptation accuracy (50 % to 80 %) was preassigned for the first 11-minute interval, and accuracy increased by 10 % in each subsequent interval. Task difficulty changed every minute, and participant preferences for difficulty changes were assessed at the same time. Adaptation accuracy was artificially induced by fixing the percentage of times the difficulty changes matched participant preferences. Participants were also randomized to two magnitude conditions, with difficulty modified by 1 (low) or 3 (high) levels each minute. User experience metrics were assessed after each interval. Analysis with latent growth models offered support for linear increases in user experience across increasing levels of adaptation accuracy. For each 10 % gain in accuracy, results indicate a 1.3 (95 % CI [.35, 2.20]) point increase in NASA Task Load Index scores (range 6–60), a 0.40 (95 % CI [.18, 0.57]) increase in effort/importance (range 2–14), and 0.48 (95 % CI [.24, 0.72]) increase in perceived competence (range 2–14). Furthermore, the effect of accuracy on Task Load Index scores was modulated by adaptation magnitude. No effects were observed for interest/enjoyment or pressure/tension. By providing quantitative estimates of effects of adaptation accuracy on user experience, the study provides guidelines for researchers and developers of affect-aware technologies. Furthermore, our methods could be adapted for use in other affective computing scenarios.
more »
« less
This content will become publicly available on January 1, 2026
Effects of Algorithmic Transparency on User Experience and Physiological Responses in Affect-Aware Task Adaptation
In affect-aware task adaptation, users' psychological states are recognized with diverse measurements and used to adapt computer-based tasks. User experience with such adaptation improves as the accuracy of psychological state recognition and task adaptation increases. However, it is unclear how user experience is influenced by algorithmic transparency: the degree to which users understand the computer's decision-making process. We thus created an affect-aware task adaptation system with 4 algorithmic transparency levels (none/low/medium/high) and conducted a study where 93 participants first experienced adaptation with no transparency for 16 minutes, then with one of the other 3 levels for 16 minutes. User experience questionnaires and physiological measurements (respiration, skin conductance, heart rate) were analyzed with mixed 2×3 analyses of variance (time × transparency group). Self-reported interest/enjoyment and competence were lower with low transparency than with medium/high transparency, but did not differ between medium and high transparency. The transparency level may also influence participants' respiratory responses to adaptation errors, but this finding is based on ad-hoc t-tests and should be considered preliminary. Overall, results show that the degree of algorithmic transparency does influence self-reported user experience. Since transparency information is relatively easy to provide, it may represent a worthwhile design element in affective computing.
more »
« less
- Award ID(s):
- 2151464
- PAR ID:
- 10579587
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- IEEE Transactions on Affective Computing
- ISSN:
- 2371-9850
- Page Range / eLocation ID:
- 1 to 8
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Though recommender systems are defined by personalization, recent work has shown the importance of additional, beyond-accuracy objectives, such as fairness. Because users often expect their recommendations to be purely personalized, these new algorithmic objectives must be communicated transparently in a fairness-aware recommender system. While explanation has a long history in recommender systems research, there has been little work that attempts to explain systems that use a fairness objective. Even though the previous work in other branches of AI has explored the use of explanations as a tool to increase fairness, this work has not been focused on recommendation. Here, we consider user perspectives of fairness-aware recommender systems and techniques for enhancing their transparency. We describe the results of an exploratory interview study that investigates user perceptions of fairness, recommender systems, and fairness-aware objectives. We propose three features – informed by the needs of our participants – that could improve user understanding of and trust in fairness-aware recommender systems.more » « less
-
To address privacy concerns with the Internet of Things (IoT) devices, researchers have proposed enhancements in data collection transparency and user control. However, managing privacy preferences for shared devices with multiple stakeholders remains challenging. We introduced ThingPoll, a system that helps users negotiate privacy configurations for IoT devices in shared settings. We designed ThingPoll by observing twelve participants verbally negotiating privacy preferences, from which we identified potentially successful and inefficient negotiation patterns. ThingPoll bootstraps a preference model from a custom crowdsourced privacy preferences dataset. During negotiations, ThingPoll strategically scaffolds the process by eliciting users’ privacy preferences, providing helpful contexts, and suggesting feasible configuration options. We evaluated ThingPoll with 30 participants negotiating the privacy settings of 4 devices. Using ThingPoll, participants reached an agreement in 97.5% of scenarios within an average of 3.27 minutes. Participants reported high overall satisfaction of 83.3% with ThingPoll as compared to baseline approaches.more » « less
-
VR sickness is a major concern for many users as VR continues its expansion towards widespread everyday use. VR sickness is thought to arise, at least in part, due to the user’s intolerance of conflict between the visually simulated self-motion and actual physical movement. Many mitigation strategies involve consistently modifying the visual stimulus to reduce its impact on the user, but this individualized approach can have drawbacks in terms of complexity of implementation and non-uniformity of user experience. This study presents a novel alternative approach that involves training the user to better tolerate the adverse stimulus by tapping into natural adaptive perceptual mechanisms. In this study, we recruited users with limited VR experience that reported susceptibility to VR sickness. Baseline sickness was measured as participants navigated a rich and naturalistic visual environment. Then, on successive days, participants were exposed to optic flow in a more abstract visual environment, and strength of the optic flow was successively increased by increasing the visual contrast of the scene, because strength of optic flow and the resulting vection are thought to be major causes of VR sickness. Sickness measures decreased on successive days, indicating that adaptation was successful. On the final day, participants were again exposed to the rich and naturalistic visual environment, and the adaptation was maintained, demonstrating that it is possible for adaptation to transfer from more abstract to richer and more naturalistic environments. These results demonstrate that gradual adaptation to increasing optic flow strength in well-controlled, abstract environments allows users to gradually reduce their susceptibility to sickness, thereby increasing VR accessibility for those prone to sickness.more » « less
-
High-performance computing (HPC) resources are used for compute-demanding calculations in various fields of science and engineering. They are large computational facilities utilized by many users simultaneously. High utilization often leads to high waiting times. Simulating users' behavior on such a system can help with future system design, develop user interventions, and ultimately improve the user’s experience and resource utilization. Here, we present HPCMod, an Agent-Based Modeling Framework for Modeling Users on HPC Resources. The key concept of the framework is the representation of the user's computational needs: the user project is represented as a collection of possibly dependent compute tasks. Each task can be executed as a single compute job or a series of jobs, depending on the task size. Some tasks can be too big to be executed in one chunk; such a situation often occurs during molecular dynamics simulation. There are multiple ways in which tasks can be split into jobs, and users will make their decisions based on previous experience, application parallel scalability, and available resources. For example, a user's compute task requires 32 node hours; it can be executed in multiple ways: a single 32-hour job on one node, two sequential 16-hour jobs on one node, one 16-hour job on two nodes, and so on. In the HPCMod, we implemented three models: 1) historical replay of compute jobs, 2) simulation of reconstituted compute tasks using historical job sizes, and 3) adaptive compute tasks splitting where users can modify jobs parameters given available resources till the execution of the next job in line. The framework was tested on a ten-node test system and a larger 1,736-node system modeled after a portion of TACC Stampede-2. The HPC resource model implements a first in first out (FIFO) scheduler with backfill scheduling. The initial results showed that on a tiny system, adaptive task-splitting is beneficial for the user but leads to a larger number of jobs. On a large system, the adaptive task-splitting was also very beneficial, decreasing waiting times for users using this strategy almost two times; however, other users got a 5% increase in their wait time. Further investigation is needed as the current task reconstitution algorithm is deterministic and does not allow quantification of job recombination uncertainties. The Julia-based implementation is fast: five years of historic workflow consisting of a million jobs and a one-hour stepping took around three minutes.more » « less
