Search for: All records

Abstract Seismicity at active volcanoes provides crucial constraints on the dynamics of magma systems and complex fault activation processes preceding and during an eruption. We characterize time‐dependent spectral features of volcanic earthquakes at Axial Seamount with unsupervised machine learning (ML) methods, revealing mixed frequency signals that rapidly increase in number about 15 hr before eruption onset. The events migrate along pre‐existing fissures, suggesting that they represent brittle crack opening driven by influx of magma or volatiles. These results demonstrate the power of unsupervised ML algorithms to characterize subtle changes in magmatic processes associated with eruption preparation, offering new possibilities for forecasting Axial's anticipated next eruption. This analysis is generalizable and can be employed to identify similar precursory signals at other active volcanoes. 

In this paper, we prove that Distributional Re- inforcement Learning (DistRL), which learns the return distribution, can obtain second-order bounds in both online and offline RL in general settings with function approximation. Second- order bounds are instance-dependent bounds that scale with the variance of return, which we prove are tighter than the previously known small-loss bounds of distributional RL. To the best of our knowledge, our results are the first second-order bounds for low-rank MDPs and for offline RL. When specializing to contextual bandits (one-step RL problem), we show that a distributional learn- ing based optimism algorithm achieves a second- order worst-case regret bound, and a second-order gap dependent bound, simultaneously. We also empirically demonstrate the benefit of DistRL in contextual bandits on real-world datasets. We highlight that our analysis with DistRL is rela- tively simple, follows the general framework of optimism in the face of uncertainty and does not require weighted regression. Our results suggest that DistRL is a promising framework for obtain- ing second-order bounds in general RL settings, thus further reinforcing the benefits of DistRL. 

In this paper, we prove that Distributional Reinforcement Learning (DistRL), which learns the return distribution, can obtain second-order bounds in both online and offline RL in general settings with function approximation. Second-order bounds are instance-dependent bounds that scale with the variance of return, which we prove are tighter than the previously known small-loss bounds of distributional RL. To the best of our knowledge, our results are the first second-order bounds for low-rank MDPs and for offline RL. When specializing to contextual bandits (one-step RL problem), we show that a distributional learning based optimism algorithm achieves a second-order worst-case regret bound, and a second-order gap dependent bound, simultaneously. We also empirically demonstrate the benefit of DistRL in contextual bandits on real-world datasets. We highlight that our analysis with DistRL is relatively simple, follows the general framework of optimism in the face of uncertainty and does not require weighted regression. Our results suggest that DistRL is a promising framework for obtaining second-order bounds in general RL settings, thus further reinforcing the benefits of DistRL. 

Abstract Axial Seamount, an extensively instrumented submarine volcano, lies at the intersection of the Cobb–Eickelberg hot spot and the Juan de Fuca ridge. Since late 2014, the Ocean Observatories Initiative (OOI) has operated a seven-station cabled ocean bottom seismometer (OBS) array that captured Axial’s last eruption in April 2015. This network streams data in real-time, facilitating seismic monitoring and analysis for volcanic unrest detection and eruption forecasting. In this study, we introduce a machine learning (ML)-based real-time seismic monitoring framework for Axial Seamount. Combining both supervised and unsupervised ML and double-difference techniques, we constructed a comprehensive, high-resolution earthquake catalog while effectively discriminating between various seismic and acoustic events. These events include earthquakes generated by different physical processes, acoustic signals of lava–water interaction, and oceanic sources such as whale calls. We first built a labeled ML-based earthquake catalog that extends from November 2014 to the end of 2021 and then implemented real-time monitoring and seismic analysis starting in 2022. With the rapid determination of high-resolution earthquake locations and the capability to track potential precursory signals and coeruption indicators of magma outflow, this system may improve eruption forecasting by providing short-term constraints on Axial’s next eruption. Furthermore, our work demonstrates an effective application that integrates unsupervised learning for signal discrimination in real-time operation, which could be adapted to other regions for volcanic unrest detection and enhanced eruption forecasting. 

While distributional reinforcement learning (DistRL) has been empirically effective, the question of when and why it is better than vanilla, non-distributional RL has remained unanswered. This paper explains the benefits of DistRL through the lens of small-loss bounds, which are instance-dependent bounds that scale with optimal achievable cost. Particularly, our bounds converge much faster than those from non-distributional approaches if the optimal cost is small. As warmup, we propose a distributional contextual bandit (DistCB) algorithm, which we show enjoys small-loss regret bounds and empirically outperforms the state-of-the-art on three real-world tasks. In online RL, we propose a DistRL algorithm that constructs confidence sets using maximum likelihood estimation. We prove that our algorithm enjoys novel small-loss PAC bounds in low-rank MDPs. As part of our analysis, we introduce the l1 distributional eluder dimension which may be of independent interest. Then, in offline RL, we show that pessimistic DistRL enjoys small-loss PAC bounds that are novel to the offline setting and are more robust to bad single-policy coverage.