skip to main content

Search for: All records

Creators/Authors contains: "Tucker, G."

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. In offline reinforcement learning (RL), a learner leverages prior logged data to learn a good policy without interacting with the environment. A major challenge in applying such methods in practice is the lack of both theoretically principled and practical tools for model selection and evaluation. To address this, we study the problem of model selection in offline RL with value function approximation. The learner is given a nested sequence of model classes to minimize squared Bellman error and must select among these to achieve a balance between approximation and estimation error of the classes. We propose the first model selection algorithm for offline RL that achieves minimax rate-optimal oracle inequalities up to logarithmic factors. The algorithm, MODBE, takes as input a collection of candidate model classes and a generic base offline RL algorithm. By successively eliminating model classes using a novel one-sided generalization test, MODBE returns a policy with regret scaling with the complexity of the minimally complete model class. In addition to its theoretical guarantees, it is conceptually simple and computationally efficient, amounting to solving a series of square loss regression problems and then comparing relative square loss between classes. We conclude with several numerical simulations showing it ismore »capable of reliably selecting a good model class.« less
    Free, publicly-accessible full text available January 1, 2023
  2. Numerical simulation of the form and characteristics of Earth’s surface provides insight into its evolution. Landlab is an Open Source Python package that contains modularized elements of numerical models for Earth’s surface, thus reducing time required for researchers to create new or reimplement existing models. Landlab contains a gridding engine which represents the model domain as a dual graph of structured quadrilaterals (e.g., raster) or irregular Voronoi polygon-Delaunay triangle mesh (e.g., regular hexagons, radially symmetric meshes, fully irregular meshes). Landlab also contains components— modular implementations of single physical processes—and a suite of utilities which support numerical methods, input/output, and visualization. This contribution describes package development since version 1.0 and backward-compatibility breaking changes which necessitates the new major release, version 2.0. Substantial changes include refactoring the grid, improving the component standard interface, dropping Python 2 support, and creating 30 new components—for a total of 57 components in the Landlab package. We describe reasons why many changes were made in order to provide insight to designers of future packages. We conclude by discussing lessons about the dynamics of scientific software development gained from the experience of using, developing, maintaining, and teaching with Landlab.