skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Narrative Planning in Large Domains through State Abstraction and Option Discovery
Low-level game environments and other simulations present a difficulty of scale for an expensive AI technique like narrative planning, which is normally constrained to environments with small state spaces. Due to this limitation, the intentional and cooperative behavior of agents guided by this technology cannot be deployed for different systems without significant additional authoring effort. I propose a process for automatically creating models for larger-scale domains such that a narrative planner can be employed in these settings. By generating an abstract domain of an environment while retaining the information needed to produce behavior appropriate to the abstract actions, agents are able to reason in a lower-complexity space and act in the higher-complexity one. This abstraction is accomplished by the development of extended-duration actions and the identification of their preconditions and effects. Together these components may be combined to form a narrative planning domain, and plans from this domain can be executed within the low-level environment.  more » « less
Award ID(s):
1911053
PAR ID:
10375803
Author(s) / Creator(s):
Date Published:
Journal Name:
Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
Volume:
18
Issue:
1
ISSN:
2326-909X
Page Range / eLocation ID:
299 to 302
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, COGSCI)
    Successfully navigating the social world requires reasoning about both high-level strategic goals, such as whether to cooperate or compete, as well as the low-level actions needed to achieve those goals. While previous work in experimental game theory has examined the former and work on multi-agent systems has examined the later, there has been little work investigating behavior in environments that require simultaneous planning and inference across both levels. We develop a hierarchical model of social agency that infers the intentions of other agents, strategically decides whether to cooperate or compete with them, and then executes either a cooperative or competitive planning program. Learning occurs across both high-level strategic decisions and low-level actions leading to the emergence of social norms. We test predictions of this model in multi-agent behavioral experiments using rich video-game like environments. By grounding strategic behavior in a formal model of planning, we develop abstract notions of both cooperation and competition and shed light on the computational nature of joint intentionality. 
    more » « less
  2. ; (, Reinforcement Learning Journal)
    General-purpose agents require fine-grained controls and rich sensory inputs to perform a wide range of tasks. However, this complexity often leads to intractable decision-making. Traditionally, agents are provided with task-specific action and observation spaces to mitigate this challenge, but this reduces autonomy. Instead, agents must be capable of building state-action spaces at the correct abstraction level from their sensorimotor experiences. We leverage the structure of a given set of temporally-extended actions to learn abstract Markov decision processes (MDPs) that operate at a higher level of temporal and state granularity. We characterize state abstractions necessary to ensure that planning with these skills, by simulating trajectories in the abstract MDP, results in policies with bounded value loss in the original MDP. We evaluate our approach in goal-based navigation environments that require continuous abstract states to plan successfully and show that abstract model learning improves the sample efficiency of planning and learning. 
    more » « less
  3. ; ; ; ; (, Proceedings of the AAAI Conference on Artificial Intelligence)
    Pedagogical planners can provide adaptive support to students in narrative-centered learning environments by dynamically scaffolding student learning and tailoring problem scenarios. Reinforcement learning (RL) is frequently used for pedagogical planning in narrative-centered learning environments. However, RL-based pedagogical planning raises significant challenges due to the scarcity of data for training RL policies. Most prior work has relied on limited-size datasets and offline RL techniques for policy learning. Unfortunately, offline RL techniques do not support on-demand exploration and evaluation, which can adversely impact the quality of induced policies. To address the limitation of data scarcity and offline RL, we propose INSIGHT, an online RL framework for training data-driven pedagogical policies that optimize student learning in narrative-centered learning environments. The INSIGHT framework consists of three components: a narrative-centered learning environment simulator, a simulated student agent, and an RL-based pedagogical planner agent, which uses a reward metric that is associated with effective student learning processes. The framework enables the generation of synthetic data for on-demand exploration and evaluation of RL-based pedagogical planning. We have implemented INSIGHT with OpenAI Gym for a narrative-centered learning environment testbed with rule-based simulated student agents and a deep Q-learning-based pedagogical planner. Our results show that online deep RL algorithms can induce near-optimal pedagogical policies in the INSIGHT framework, while offline deep RL algorithms only find suboptimal policies even with large amounts of data. 
    more » « less
  4. ; ; ; (, IROS)
    We propose a learning‐from‐demonstration approach for grounding actions from expert data and an algorithm for using these actions to perform a task in new environments. Our approach is based on an application of sampling‐based motion planning to search through the tree of discrete, high‐level actions constructed from a symbolic representation of a task. Recursive sampling‐based planning is used to explore the space of possible continuous‐space instantiations of these actions. We demonstrate the utility of our approach with a magnetic structure assembly task, showing that the robot can intelligently select a sequence of actions in different parts of the workspace and in the presence of obstacles. This approach can better adapt to new environments by selecting the correct high‐level actions for the particular environment while taking human preferences into account. 
    more » « less
  5. ; (, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment)
    Contemporary automated planning research emphasizes the use of domain knowledge abstractions like heuristics to improve search efficiency. Transformative automated abstraction techniques which decompose or otherwise reformulate the problem have a limited presence, owing to poor performance in key metrics like plan length and time efficiency. In this paper, we argue for a reexamination of these transformative techniques in the context of narrative planning, where classical metrics are less appropriate. We propose a model for automating abstraction by decomposing a planning problem into subproblems which serve as abstract features of the problem. We demonstrate the application of this approach on a low-level problem and discuss key features of the resulting abstract problem. Plans in the abstract problem are shorter, representing summaries of low-level plans, but can be directly translated into low-level plans for the original problem. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email