An official website of the United States government
Improving our understanding of how humans perceive AI teammates is an important foundation for our general understanding of human-AI teams. Extending relevant work from cognitive science, we propose a framework based on item response theory for modeling these perceptions. We apply this framework to real-world experiments, in which each participant works alongside another person or an AI agent in a question-answering setting, repeatedly assessing their teammate’s performance. Using this experimental data, we demonstrate the use of our framework for testing research questions about people’s perceptions of both AI agents and other people. We contrast mental models of AI teammates with those of human teammates as we characterize the dimensionality of these mental models, their development over time, and the influence of the participants’ own self-perception. Our results indicate that people expect AI agents’ performance to be significantly better on average than the performance of other humans, with less variation across different types of problems. We conclude with a discussion of the implications of these findings for human-AI interaction.
more »
« less
Utilizing Current Technologies to Foster Augmented On-line Learning
psychology, and cognition has progressed sufficiently that the technology exists to develop a mutually beneficial exchange of information between a human and an AI. Dubbed “AI Symbiosis,” this process enables positive feedback between humans and adaptive computer algorithms in which both human and AI would “learn” how to perform tasks more efficiently than either could alone. Several new technologies and inventions al-low a vast array of augmented input and/or output between humans and AI, in-cluding mental activity wirelessly operating computers, manipulation of targeted neurons with or without implants, non-invasive, surface-level implants the size of a coin transmitting real-time neural activity of senses, real-time video feed of human mental images, and estimation of thoughts and emotions. A research pro-ject is planned to study students’ divided attention when they are learning content in on-line environments. The research will target eye-tracking, click timing, and task performance data to determine the levels of impact divided attention has on student learning. We believe that this line of research will also inform best prac-tices in on-line instructional settings.
more »
« less
- Award ID(s):
- 1662487
- PAR ID:
- 10312069
- Date Published:
- Journal Name:
- Augmented Cognition
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Many real-life scenarios require humans to make difficult trade-offs: do we always follow all the traffic rules or do we violate the speed limit in an emergency? These scenarios force us to evaluate the trade-off between collective norms and our own personal objectives. To create effective AI-human teams, we must equip AI agents with a model of how humans make trade-offs in complex, constrained environments. These agents will be able to mirror human behavior or to draw human attention to situations where decision making could be improved. To this end, we propose a novel inverse reinforcement learning (IRL) method for learning implicit hard and soft constraints from demonstrations, enabling agents to quickly adapt to new settings. In addition, learning soft constraints over states, actions, and state features allows agents to transfer this knowledge to new domains that share similar aspects.more » « less
-
While drones have exhibited considerable potential to revolutionize the construction industry, previous studies across domains have proposed that human-drone interaction could cause adverse impacts on humans (e.g., collision and discomfort). Given that construction has been recognized as a hazardous and high-stress workplace, it deserves deep exploration regarding how newly introduced drones will influence worker well-being during the interaction. However, there is a paucity of research on worker stress when communicating with drones in construction. Successful human-drone interaction must necessitate seamless and comfortable communication between workers and drones. Therefore, this study investigates the impact of physically demanding response levels on construction workers’ mental and physical well-being throughout the communication cycle. Three levels of physical responses (low, medium, and high) required for drone communication were simulated in an extended reality roofing experiment. During the communication process, real-time stress levels were assessed through participants’ electrodermal activity. The results indicated that a higher physical level of communication significantly increased workers’ higher stress levels in both the response and decoding phases. Additionally, providing drones’ feedback in verbal human-drone communication is especially important to reduce workers’ confusion and mental stress. This study highlights the critical need for worker-centric design and communication strategies in drone integration within construction.more » « less
-
Abstract Recent work has explored how complementary strengths of humans and artificial intelligence (AI) systems might be productively combined. However, successful forms of human–AI partnership have rarely been demonstrated in real‐world settings. We present the iterative design and evaluation of Lumilo, smart glasses that help teachers help their students in AI‐supported classrooms by presenting real‐time analytics about students’ learning, metacognition, and behavior. Results from a field study conducted in K‐12 classrooms indicate that students learn more when teachers and AI tutors work together during class. We discuss implications of this research for the design of human–AI partnerships. We argue for more participatory approaches to research and design in this area, in which practitioners and other stakeholders are deeply, meaningfully involved throughout the process. Furthermore, we advocate for theory‐building and for principled approaches to the study of human–AI decision‐making in real‐world contexts.more » « less
-
Dialog systems (e.g., chatbots) have been widely studied, yet related research that leverages artificial intelligence (AI) and natural language processing (NLP) is constantly evolving. These systems have typically been developed to interact with humans in the form of speech, visual, or text conversation. As humans continue to adopt dialog systems for various objectives, there is a need to involve humans in every facet of the dialog development life cycle for synergistic augmentation of both the humans and the dialog system actors in real-world settings. We provide a holistic literature survey on the recent advancements inhuman-centered dialog systems(HCDS). Specifically, we provide background context surrounding the recent advancements in machine learning-based dialog systems and human-centered AI. We then bridge the gap between the two AI sub-fields and organize the research works on HCDS under three major categories (i.e., Human-Chatbot Collaboration, Human-Chatbot Alignment, Human-Centered Chatbot Design & Governance). In addition, we discuss the applicability and accessibility of the HCDS implementations through benchmark datasets, application scenarios, and downstream NLP tasks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1007/978-3-030-78114-9_27
