skip to main content

Title: I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
We propose a novel task, G4C (Goal-driven Guidance Generation in Grounded Communication), for studying goal-driven and grounded natural language interactions. Specifically, we choose Dungeons and Dragons (D&D) -- a role-playing game consisting of multiple player characters and a Dungeon Master (DM) who collaborate to achieve a set of goals that are beneficial to the players -- as a testbed for this task. Here, each of the player characters is a student, with their own personas and abilities, and the DM is the teacher, an arbitrator of the rules of the world and responsible for assisting and guiding the students towards a global goal. We propose a theory-of-mind-inspired methodology for training such a DM with reinforcement learning (RL), where a DM: (1) learns to predict how the players will react to its utterances using a dataset of D&D dialogue transcripts; and (2) uses this prediction as a reward function providing feedback on how effective these utterances are at guiding the players towards a goal. Human and automated evaluations show that a DM trained with RL to generate guidance by incorporating a theory-of-mind of the players significantly improves the players' ability to achieve goals grounded in their shared world.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ; ; ;
Date Published:
Journal Name:
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics
Page Range / eLocation ID:
11136 to 11155
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Devising models that reliably recognize player goals is a key challenge in creating player-adaptive games. Player goal recognition is the task of automatically recognizing the intent of a player from a sequence of observed player actions in a game environment. In open-world digital games, players often undertake suboptimal and varied sequences of actions to achieve goals, and the high degree of freedom afforded to players makes it challenging to identify sequential patterns that lead toward specific goals. To address these issues, we present a player goal recognition framework that utilizes a fine-tuned T5 language model, which incorporates our novel attention mechanism called Temporal Contrary Attention (TCA). The T5 language model enables the framework to exploit correlations between observations through non-sequential self-attention within input sequences, while TCA enables the framework to learn to eliminate goal hypotheses by considering counterevidence within a temporal window. We evaluate our approach using game trace data collected from 144 players' interactions with an open-world educational game. Specifically, we investigate the predictive capacity of our approach to recognize player goals as well as player plans represented as abstract actions. Results show that our approach outperforms non-linguistic machine learning approaches as well as T5 without TCA. We discuss the implications of these findings for the design and development of player goal recognition models to create player-adaptive games.

    more » « less
  2. null (Ed.)
    Human communication involves far more than words; speak- ers’ utterances are often accompanied by various kinds of emo- tional expressions. How do listeners represent and integrate these distinct sources of information to make communicative inferences? We first show that people, as listeners, integrate both verbal and emotional information when inferring true states of the world and others’ communicative goals, and then present computational models that formalize these inferences by considering different ways in which these signals might be generated. Results suggest that while listeners understand that utterances and emotional expressions are generated by a bal- ance of speakers’ informational and social goals, they addi- tionally consider the possibility that emotional expressions are noncommunicative signals that directly reflect the speaker’s in- ternal states. These results are consistent with the predictions of a probabilistic model that integrates goal inferences with linguistic and emotional signals, moving us towards a more complete formal theory of human communicative reasoning. 
    more » « less
  3. null (Ed.)
    Choice poetics is a formalist framework that seeks to concretely describe the impacts choices have on player experiences within narrative games. Developed in part to support algorithmic generation of narrative choices, the theory includes a detailed analytical framework for understanding the impressions choice structures make by analyzing the relationships among options, outcomes, and player goals. The theory also emphasizes the need to account for players’ various modes of engagement, which vary both during play and between players. In this work, we illustrate the non-computational application of choice poetics to the analysis of two different games to further develop the theory and make it more accessible to others. We focus first on using choice poetics to examine the central repeated choice in “Undertale,” and show how it can be used to contrast two different player types that will approach a choice differently. Finally, we give an example of fine-grained analysis using a choice from the game “Papers, Please,” which breaks down options and their outcomes to illustrate exactly how the choice pushes players towards complicity via the introduction of uncertainty. Through all of these examples, we hope to show the usefulness of choice poetics as a framework for understanding narrative choices, and to demonstrate concretely how one could productively apply it to choices “in the wild.” 
    more » « less
  4. Teamwork is a set of interrelated reasoning, actions and behaviors of team members that facilitate common objectives. Teamwork theory and experiments have resulted in a set of states and processes for team effectiveness in both human-human and agent-agent teams. However, human-agent teaming is less well studied because it is so new and involves asymmetry in policy and intent not present in human teams. To optimize team performance in human-agent teaming, it is critical that agents infer human intent and adapt their polices for smooth coordination. Most literature in human-agent teaming builds agents referencing a learned human model. Though these agents are guaranteed to perform well with the learned model, they lay heavy assumptions on human policy such as optimality and consistency, which is unlikely in many real-world scenarios. In this paper, we propose a novel adaptive agent architecture in human-model-free setting on a two-player cooperative game, namely Team Space Fortress (TSF). Previous human-human team research have shown complementary policies in TSF game and diversity in human players’ skill, which encourages us to relax the assumptions on human policy. Therefore, we discard learning human models from human data, and instead use an adaptation strategy on a pre-trained library of exemplar policies composed of RL algorithms or rule-based methods with minimal assumptions of human behavior. The adaptation strategy relies on a novel similarity metric to infer human policy and then selects the most complementary policy in our library to maximize the team performance. The adaptive agent architecture can be deployed in real-time and generalize to any off-the-shelf static agents. We conducted human-agent experiments to evaluate the proposed adaptive agent framework, and demonstrated the suboptimality, diversity, and adaptability of human policies in human-agent teams. 
    more » « less
  5. null (Ed.)
    his panel paper presents research on connecting theory to practice and the lessons learned in a change project, with a focus on team formation during the early stages of change making. An important yet often overlooked step in any change project is pulling together individuals to form a competent and efficient team. A functional change-making team requires a variety of complementary skill sets, which may come from different disciplinary backgrounds and/or different prior experiences. Kotter (1996) uses the term “guiding coalition” to refer to an effective change-making team. He identifies four key characteristics of guiding coalitions: position power, expertise, credibility, leadership. Kotter also goes on to examine the importance of trust and a common goal. In a review of the literature on guiding coalitions, Have, Have, Huijsmans, and Otto (2017) found that though the concept of a guiding coalition is widely advocated in the literature, only one study showed a moderate correlation between the existence of a guiding coalition and the success of a change process (Abraham, Griffin, & Crawford, 1999). Have et al. (2017) conclude that while the literature provides little evidence to the value of a guiding coalition, it does provide evidence that Kotter’s characteristics of a guiding coalition (position power, expertise, credibility, leadership skills, trust in leadership, and setting common goals) individually have positive effects on the outcomes of a change project. However, we don’t know how these characteristics interact. This analysis of team building and complementary skill sets emerges from our participatory action research with the NSF REvolutionizing engineering and computer science Departments (RED) teams to investigate the change process within STEM higher education. The research-to-practice cycle is integral to our project; data gathered through working with the RED teams provides insights that are then translated into applied, hands-on practices. We utilize an abductive analysis approach, a qualitative methodology that moves recursively between the data and theory-building to remain open to new or contradictory findings, keeping existing theory in mind while not developing formal hypotheses (Timmermans & Tavory, 2012). We find that many of the teams have learned lessons in the early stages of the change process around the guiding coalition characteristics, and our analysis builds on the literature by examining how these characteristics interact. For example, the expertise of the social scientists and education researchers help discern which change strategies have supporting evidence and fit the context, in addition to what is reasonable for planning, implementation, and evaluation. The results presented in this paper connect theory to practice, clarifying practices for building effective change-making teams within higher education. 
    more » « less