Search for: All records

We obtain global, non-asymptotic convergence guarantees for independent learning algorithms in competitive reinforcement learning settings with two agents (i.e., zero-sum stochastic games). We consider an episodic setting where in each episode, each player independently selects a policy and observes only their own actions and rewards, along with the state. We show that if both players run policy gradient methods in tandem, their policies will converge to a min-max equilibrium of the game, as long as their learning rates follow a two-timescale rule (which is necessary). To the best of our knowledge, this constitutes the first finite-sample convergence result for independent policy gradient methods in competitive RL; prior work has largely focused on centralized, coordinated procedures for equilibrium computation. 

Bedload transport is an important mechanism for sediment flux in the nearshore. Yet few studies examine the relationship between bedform evolution and net sediment transport. Our work contributes concurrent observations of bedform mobility and bedload transport in response to wave dominant, current dominant, and combined wave‐current flows in the nearshore. Bedload sediment flux from migrating bedforms during combined wave‐current conditions accounted for at least 20% more bedload transport when compared with wave dominant flows and at least 80% more than current‐dominant flows. Bedforms were observed to transport the most sediment during periods with strong currents, with high‐energy skewed waves, and while bedform orientation and transport direction were aligned. Regardless of flow type, bedform migration rates were directly proportional to the total kinetic energy contained in the flow field. Eleven bedload transport models formulated to be used in combined flows (both shear and energetics based) were compared with sediment flux estimated from measured bedform migration. An energetics based sediment transport model was most representative for our data.

 

We developed 3‐D isotropic crustal seismic velocity models of central Idaho and eastern Oregon from the IDOR (western IDaho and eastern ORegon) Passive seismic data. Ambient noise tomography yielded crustal velocity structure from vertical component Rayleigh wave group and phase velocity measurements. Results include a strong shear wave velocity contrast—faster in accreted Blue Mountains terranes west of the western Idaho shear zone (WISZ), slower in the Idaho batholith, emplaced within the Archean Grouse Creek block east of the WISZ—restricted to the upper‐to‐middle crust. In deeper crust not affected by mafic underplating during Columbia River Flood Basalt magmatism, the shear wave velocity of the Mesozoic Olds Ferry continental arc terrane is indistinguishable from that of the Archean Grouse Creek block basement. Crustal columns of the Olds Ferry terrane and the Permian‐Jurassic Wallowa intraoceanic arc terrane are characterized by low seismic velocities, consistent with felsic lithologies down to ∼20 km. West of the WISZ, the Bourne and Greenhorn subterranes of the Baker terrane, an accretionary complex between the arc terranes, have distinct shallow crustal seismic velocities. The Greenhorn subterrane to midcrustal depths is in an overthrust geometry relative to the Bourne subterrane. Lack of mafic lower crust in our results of the Wallowa or Olds Ferry arcs may be due to imbrication of upper crustal felsic plutonic complexes of these arcs. Shortening and thickening of the Blue Mountains arc terranes crust to >30 km, and subduction or delamination of their mafic lower crustal sections is a viable mechanism for growth of a felsic continental crust.