Search for: All records

In offline reinforcement learning (RL), a learner leverages prior logged data to learn a good policy without interacting with the environment. A major challenge in applying such methods in practice is the lack of both theoretically principled and practical tools for model selection and evaluation. To address this, we study the problem of model selection in offline RL with value function approximation. The learner is given a nested sequence of model classes to minimize squared Bellman error and must select among these to achieve a balance between approximation and estimation error of the classes. We propose the first model selection algorithm for offline RL that achieves minimax rate-optimal oracle inequalities up to logarithmic factors. The algorithm, MODBE, takes as input a collection of candidate model classes and a generic base offline RL algorithm. By successively eliminating model classes using a novel one-sided generalization test, MODBE returns a policy with regret scaling with the complexity of the minimally complete model class. In addition to its theoretical guarantees, it is conceptually simple and computationally efficient, amounting to solving a series of square loss regression problems and then comparing relative square loss between classes. We conclude with several numerical simulations showing it is capable of reliably selecting a good model class. 

Assessing rivers' and hillslopes' sensitivity to external forcing is paramount to understand landscape evolution, in particular as a response to Quaternary climate changes. River networks are usually considered to be the main conveyors of environmental signals, such as changes in precipitation, temperature, or baselevel. Yet because hillslopes provide the source of sediment for river networks, their response to environmental change also modulates landscape dynamics. In order to characterize such behavior, we analyze the response times of a transport‐limited hillslope. We use simple numerical models of denudation to study hillslope responses to oscillatory forcing and understand their filtering effects on environmental signals. Modifications in the frequency of climate oscillation, such as the change that occurred at the Mid‐Pleistocene Transition, can significantly modulate hillslope sediment‐flux response. We infer a wide range of hillslope responses, ranging from negligible change over the full range of climate‐forcing frequencies, to a significant filtering of long‐period signals.

 

Numerical simulation of the form and characteristics of Earth’s surface provides insight into its evolution. Landlab is an Open Source Python package that contains modularized elements of numerical models for Earth’s surface, thus reducing time required for researchers to create new or reimplement existing models. Landlab contains a gridding engine which represents the model domain as a dual graph of structured quadrilaterals (e.g., raster) or irregular Voronoi polygon-Delaunay triangle mesh (e.g., regular hexagons, radially symmetric meshes, fully irregular meshes). Landlab also contains components— modular implementations of single physical processes—and a suite of utilities which support numerical methods, input/output, and visualization. This contribution describes package development since version 1.0 and backward-compatibility breaking changes which necessitates the new major release, version 2.0. Substantial changes include refactoring the grid, improving the component standard interface, dropping Python 2 support, and creating 30 new components—for a total of 57 components in the Landlab package. We describe reasons why many changes were made in order to provide insight to designers of future packages. We conclude by discussing lessons about the dynamics of scientific software development gained from the experience of using, developing, maintaining, and teaching with Landlab. 

Transmission spectroscopy^1–3of exoplanets has revealed signatures of water vapour, aerosols and alkali metals in a few dozen exoplanet atmospheres^4,5. However, these previous inferences with the Hubble and Spitzer Space Telescopes were hindered by the observations’ relatively narrow wavelength range and spectral resolving power, which precluded the unambiguous identification of other chemical species—in particular the primary carbon-bearing molecules^6,7. Here we report a broad-wavelength 0.5–5.5 µm atmospheric transmission spectrum of WASP-39b⁸, a 1,200 K, roughly Saturn-mass, Jupiter-radius exoplanet, measured with the JWST NIRSpec’s PRISM mode⁹as part of the JWST Transiting Exoplanet Community Early Release Science Team Program^10–12. We robustly detect several chemical species at high significance, including Na (19σ), H₂O (33σ), CO₂(28σ) and CO (7σ). The non-detection of CH₄, combined with a strong CO₂feature, favours atmospheric models with a super-solar atmospheric metallicity. An unanticipated absorption feature at 4 µm is best explained by SO₂(2.7σ), which could be a tracer of atmospheric photochemistry. These observations demonstrate JWST’s sensitivity to a rich diversity of exoplanet compositions and chemical processes.