An official website of the United States government
With the proliferation of AI, there is a growing concern regarding individuals becoming overly reliant on AI, leading to a decrease in intrinsic skills and autonomy. Assistive AI frameworks, on the other hand, also have the potential to improve human learning and performance by providing personalized learning experiences and real-time feedback. To study these opposing viewpoints on the consequences of AI assistance, we conducted a behavioral experiment using a dynamic decision-making game to assess how AI assistance impacts user performance, skill transfer, and cognitive engagement in task execution. Participants were assigned to one of four conditions that featured AI assistance at different time-points during the task. Our results suggest that AI assistance can improve immediate task performance without inducing human skill degradation or carryover effects in human learning. This observation has important implications for AI assistive frameworks as it suggests that there are classes of tasks in which assistance can be provided without risking the autonomy of the user. We discuss the possible reasons for this set of effects and explore their implications for future research directives.
more »
« less
Measuring Temporal Awareness for Human-Aware AI
This research investigated human performance in response to task demands that may be used to convey information about the human to an artificial agent. We performed an experiment with a dynamic time-sharing task to investigate participants development of temporal awareness of the task event unfolding in time. Temporal awareness as an extension, or a special case, of situation awareness, may provide for useful measures of covert mental models applicable to numerous tasks and for input to human-aware AI agents. Temporal awareness measures may be used to classify human performance into the control modes in the contextual control model (COCOM): scrambled, opportunistic, tactical, and strategic. Twenty-one participants participated in a within subjects experiment with an abstract task of resetting four independent timers within their respective windows of opportunity. The results show that temporal measures of task performance are sensitive to changes in task disruptions and difficulty and therefore have promise for human-aware AI.
more »
« less
- Award ID(s):
- 2125362
- PAR ID:
- 10506881
- Publisher / Repository:
- Sage Journals
- Date Published:
- Journal Name:
- Proceedings of the Human Factors and Ergonomics Society Annual Meeting
- Volume:
- 67
- Issue:
- 1
- ISSN:
- 1071-1813
- Page Range / eLocation ID:
- 1817 to 1823
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
People form perceptions and interpretations of AI through external sources prior to their interaction with new technology. For example, shared anecdotes and media stories influence prior beliefs that may or may not accurately represent the true nature of AI systems. We hypothesize people's prior perceptions and beliefs will affect human-AI interactions and usage behaviors when using new applications. This paper presents a user experiment to explore the interplay between user's pre-existing beliefs about AI technology, individual differences, and previously established sources of cognitive bias from first impressions with an interactive AI application. We employed questionnaire measures as features to categorize users into profiles based on their prior beliefs and attitudes about technology. In addition, participants were assigned to one of two controlled conditions designed to evoke either positive or negative first impressions during an AI-assisted judgment task using an interactive application. The experiment and results provide empirical evidence that profiling users by surveying them on their prior beliefs and differences can be a beneficial approach for bias (and/or unanticipated usage) mitigation instead of seeking one-size-fits-all solutions.more » « less
-
Learning new languages is a complex cognitive task involving both implicit and explicit processes. Batterink, Oudiette, Reber, and Paller (2014) report that participants with vs. without conscious awareness of a hidden semi-artificial language regularity showed no significant differences in behavioral measures of grammar learning, suggesting that implicit/explicit routes may be functionally equivalent. However, their operationalization of learning via median reaction times might not capture underlying differences in cognitive processes. In a conceptual replication, we compared rule-aware (n=14) and rule-unaware (n=21) participants via drift-diffusion modeling, which can quantify distinct subcomponents of evidence-accumulation processes (Ratcliff & Rouder, 1998). For both groups, grammar learning was manifested in non-decision parameters, suggesting anticipation of motor responses. For rule-aware participants only, learning also affected bias in evidence accumulation during word reading. These results suggest that implicit grammar learning may be manifested through low-level mechanisms whereas explicit grammar learning may involve more direct engagement with encoded target meanings.more » « less
-
Combining uncertainty information with AI recommendations supports calibration with domain knowledgeThe use of Artificial Intelligence (AI) decision support is increasing in high-stakes contexts, such as healthcare, defense, and finance. Uncertainty information may help users better leverage AI predictions, especially when combined with their domain knowledge. We conducted a human-subject experiment with an online sample to examine the effects of presenting uncertainty information with AI recommendations. The experimental stimuli and task, which included identifying plant and animal images, are from an existing image recognition deep learning model, a popular approach to AI. The uncertainty information was predicted probabilities for whether each label was the true label. This information was presented numerically and visually. In the study, we tested the effect of AI recommendations in a within-subject comparison and uncertainty information in a between-subject comparison. The results suggest that AI recommendations increased both participants’ accuracy and confidence. Further, providing uncertainty information significantly increased accuracy but not confidence, suggesting that it may be effective for reducing overconfidence. In this task, participants tended to have higher domain knowledge for animals than plants based on a self-reported measure of domain knowledge. Participants with more domain knowledge were appropriately less confident when uncertainty information was provided. This suggests that people use AI and uncertainty information differently, such as an expert versus second opinion, depending on their level of domain knowledge. These results suggest that if presented appropriately, uncertainty information can potentially decrease overconfidence that is induced by using AI recommendations.more » « less
-
Error monitoring is an essential human ability underlying learning and metacognition. In the time domain, humans possess a remarkable ability to learn and adapt to temporal intervals, yet the neural mechanisms underlying this are not well understood. Recently, we demonstrated that humans exhibit improvements in sensorimotor time estimates when given the chance to incorporate feedback from a previous trial (Bader and Wiener, 2021), suggesting that humans are metacognitively aware of their own timing errors. To test the neural basis of this metacognitive ability, human participants of both sexes underwent fMRI while they performed a visual temporal reproduction task with randomized suprasecond intervals (1.5-6s). Crucially, each trial was repeated following feedback, allowing a “re-do” to learn from the successes or errors in the initial trial. Behaviorally, we replicated our previous finding that subjects improve their performance on re-do trials despite the feedback being temporally uninformative (i.e. early or late). For neuroimaging, we observed a dissociation between estimating and reproducing time intervals, with the former more likely to engage regions associated with the default mode network (DMN), including the superior frontal gyri, precuneus, and posterior cingulate, whereas the latter activated regions associated traditionally with the “Timing Network” (TN), including the supplementary motor area (SMA), precentral gyrus, and right supramarginal gyrus. Notably, greater DMN involvement was observed in Re-do trials. Further, the extent of the DMN was greater on re-do trials, whereas for the TN it was more constrained. Finally, Task-based connectivity between these networks demonstrated higher inter-network correlation on initial trials, but primarily when estimating trials, whereas on re-do trials communication was higher during reproduction. Overall, these results suggest the DMN and TN work in concert to mediate subjective awareness of one’s sense of time for the purpose of improving timing performance. Significance StatementA finely tuned sense of time perception is imperative for everyday motor actions (e.g., hitting a baseball). Timing self-regulation requires correct assessment and updating duration estimates if necessary. Using a modified version of a classical task of time measurement, we explored the neural regions involved in error detection, time awareness, and learning to time. Reinforcing the role of the SMA in measuring temporal information and providing evidence of co-activation with the DMN, this study demonstrates that the brain overlays sensorimotor timing with a metacognitive awareness of its passage.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1177/21695067231192635
