Motivational agents are virtual agents that seek to motivate users by providing feedback and guidance. Prior work has shown how certain factors of an agent, such as the type of feedback given or the agent’s appearance, can influence user motivation when completing tasks. However, it is not known how nonverbal mirroring affects an agent’s ability to motivate users. Specifically, would an agent that mirrors be more motivating than an agent that does not? Would an agent trained on real human behaviors be better? We conducted a within-subjects study asking 30 participants to play a “find-the-hidden-object” game while interacting with a motivational agent that would provide hints and feedback on the user’s performance. We created three agents: a Control agent that did not respond to the user’s movements, a simple Mimic agent that mirrored the user’s movements on a delay, and a Complex agent that used a machine-learned behavior model. We asked participants to complete a questionnaire asking them to rate their levels of motivation and perceptions of the agent and its feedback. Our results showed that the Mimic agent was more motivating than the Control agent and more helpful than the Complex agent. We also found that when participants became aware of the mimicking behavior, it can feel weird or creepy; therefore, it is important to consider the detection of mimicry when designing virtual agents.
more »
« less
Visual Hide and Seek
We train embodied agents to play Visual Hide and Seek to study the relationship between agent behaviors and environmental complexity. In Visual Hide and Seek, a prey must navigate in a simulated environment in order to avoid capture from a predator, only relying on first-person visual observations. By probing different environmental factors, agents exhibit diverse hiding strategies and even the knowledge of its own visibility to other agents in the scene. Furthermore, we quantitatively analyze how agent weaknesses, such as slower speed, affect the learned policy. Our results suggest that, although agent weakness makes the learning problem more challenging, they also cause more useful features to be learned.
more »
« less
- Award ID(s):
- 1925157
- PAR ID:
- 10198588
- Date Published:
- Journal Name:
- The 2020 Conference on Artificial Life
- Page Range / eLocation ID:
- 645 to 655
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Regardless of how much data artificial intelligence agents have available, agents will inevitably encounter previously unseen situations in real-world deployments. Reacting to novel situations by acquiring new information from other people—socially situated learning—is a core faculty of human development. Unfortunately, socially situated learning remains an open challenge for artificial intelligence agents because they must learn how to interact with people to seek out the information that they lack. In this article, we formalize the task of socially situated artificial intelligence—agents that seek out new information through social interactions with people—as a reinforcement learning problem where the agent learns to identify meaningful and informative questions via rewards observed through social interaction. We manifest our framework as an interactive agent that learns how to ask natural language questions about photos as it broadens its visual intelligence on a large photo-sharing social network. Unlike active-learning methods, which implicitly assume that humans are oracles willing to answer any question, our agent adapts its behavior based on observed norms of which questions people are or are not interested to answer. Through an 8-mo deployment where our agent interacted with 236,000 social media users, our agent improved its performance at recognizing new visual information by 112%. A controlled field experiment confirmed that our agent outperformed an active-learning baseline by 25.6%. This work advances opportunities for continuously improving artificial intelligence (AI) agents that better respect norms in open social environments.more » « less
-
We study a model-free federated linear quadratic regulator (LQR) problem where M agents with unknown, distinct yet similar dynamics collaboratively learn an optimal policy to minimize an average quadratic cost while keeping their data private. To exploit the similarity of the agents' dynamics, we propose to use federated learning (FL) to allow the agents to periodically communicate with a central server to train policies by leveraging a larger dataset from all the agents. With this setup, we seek to understand the following questions: (i) Is the learned common policy stabilizing for all agents? (ii) How close is the learned common policy to each agent's own optimal policy? (iii) Can each agent learn its own optimal policy faster by leveraging data from all agents? To answer these questions, we propose a federated and model-free algorithm named FedLQR. Our analysis overcomes numerous technical challenges, such as heterogeneity in the agents' dynamics, multiple local updates, and stability concerns. We show that FedLQR produces a common policy that, at each iteration, is stabilizing for all agents. We provide bounds on the distance between the common policy and each agent's local optimal policy. Furthermore, we prove that when learning each agent's optimal policy, FedLQR achieves a sample complexity reduction proportional to the number of agents M in a low-heterogeneity regime, compared to the single-agent setting.more » « less
-
We evaluate the benefits of intention perception, the ability of an agent to perceive the intentions and plans of others, in improving a software agent's survival likelihood in a simulated virtual environment. To model intention perception, we set up a multi-agent predator and prey model, where the prey agents search for food and the predator agents seek to eat the prey. We then analyze the difference in average survival rates between prey with intention perception-knowledge of which predators are targeting them-and those without. We find that intention perception provides significant survival advantages in almost all cases tested, agreeing with other recent studies investigating intention perception in adversarial situations and environmental danger assessment.more » « less
-
Observational learning models seek to understand how distributed agents learn from observing the actions of others. In the basic model, agents seek to choose between two alternatives, where the underlying value of each alternative is the same for each agent. Agents do not know this value but only observe a noisy signal of the value and make their decision based on this signal and observations of other agents’ actions. Here, instead we consider a scenario in which the choices faced by an agent exhibit a negative externality so that value of a choice may decrease depending on the history of other agents selecting that choice. We study the learning behavior of Bayesian agents with such an externality and show that this can lead to very different outcomes compared to models without such an externality.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1162/isal_a_00269
