More Like this

1. High-quantile regression for tail-dependent time series

   https://doi.org/10.1093/biomet/asaa046  

   Zhang, Ting (July 2020, Biometrika)

   null (Ed.)

   Summary Quantile regression is a popular and powerful method for studying the effect of regressors on quantiles of a response distribution. However, existing results on quantile regression were mainly developed for cases in which the quantile level is fixed, and the data are often assumed to be independent. Motivated by recent applications, we consider the situation where (i) the quantile level is not fixed and can grow with the sample size to capture the tail phenomena, and (ii) the data are no longer independent, but collected as a time series that can exhibit serial dependence in both tail and non-tail regions. To study the asymptotic theory for high-quantile regression estimators in the time series setting, we introduce a tail adversarial stability condition, which had not previously been described, and show that it leads to an interpretable and convenient framework for obtaining limit theorems for time series that exhibit serial dependence in the tail region, but are not necessarily strongly mixing. Numerical experiments are conducted to illustrate the effect of tail dependence on high-quantile regression estimators, for which simply ignoring the tail dependence may yield misleading $$p$$-values. 

   more » « less
2. Transformed‐Linear Models for Time Series Extremes

   https://doi.org/10.1111/jtsa.12732  

   Mhatre, Nehali; Cooley, Daniel (February 2024, Journal of Time Series Analysis)

   To capture the dependence in the upper tail of a time series, we develop non‐negative regularly varying time series models that are constructed similarly to classical non‐extreme ARMA models. Rather than fully characterizing tail dependence of the time series, we define the concept of weak tail stationarity which allows us to describe a regularly varying time series via a measure of pairwise extremal dependencies, the tail pairwise dependence function (TPDF). We state consistency requirements among the finite‐dimensional collections of the elements of a regularly varying time series and show that the TPDF's value does not depend on the dimension of the random vector being considered. So that our models take non‐negative values, we use transformed‐linear operations. We show existence and stationarity of these models, and develop their properties such as the model TPDFs. We fit models to hourly windspeed and daily fire weather index data, and we find that the fitted transformed‐linear models produce better estimates of upper tail quantities than a traditional ARMA model, classical linear regularly varying models, a max‐ARMA model, and a Markov model. 

   more » « less
3. Semiparametric Efficient Inference in Adaptive Experiments

   Cook, Thomas; Mishler, Alan; Ramdas, Aaditya (April 2024, Proceedings of Machine Learning Research)

   We consider the problem of efficient inference of the Average Treatment Effect in a sequential experiment where the policy governing the assignment of subjects to treatment or control can change over time. We first provide a central limit theorem for the Adaptive Augmented Inverse-Probability Weighted estimator, which is semiparametric efficient, under weaker assumptions than those previously made in the literature. This central limit theorem enables efficient inference at fixed sample sizes. We then consider a sequential inference setting, deriving both asymptotic and nonasymptotic confidence sequences that are considerably tighter than previous methods. These anytime-valid methods enable inference under data-dependent stopping times (sample sizes). Additionally, we use propensity score truncation techniques from the recent off-policy estimation literature to reduce the finite sample variance of our estimator without affecting the asymptotic variance. Empirical results demonstrate that our methods yield narrower confidence sequences than those previously developed in the literature while maintaining time-uniform error control. 

   more » « less
4. A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks

   Simsekli, Umut; Sagun, Levent; Gurbuzbalaban, Mert (June 2019, Proceedings of Machine Learning Research)

   The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the classical central limit theorem (CLT) kicks in. This assumption is often made for mathematical convenience, since it enables SGD to be analyzed as a stochastic differential equation (SDE) driven by a Brownian motion. We argue that the Gaussianity assumption might fail to hold in deep learning settings and hence render the Brownian motion-based analyses inappropriate. Inspired by non-Gaussian natural phenomena, we consider the GN in a more general context and invoke the generalized CLT (GCLT), which suggests that the GN converges to a heavy-tailed -stable random variable. Accordingly, we propose to analyze SGD as an SDE driven by a Lévy motion. Such SDEs can incur ‘jumps’, which force the SDE transition from narrow minima to wider minima, as proven by existing metastability theory. To validate the -stable assumption, we conduct extensive experiments on common deep learning architectures and show that in all settings, the GN is highly non-Gaussian and admits heavy-tails. We further investigate the tail behavior in varying network architectures and sizes, loss functions, and datasets. Our results open up a different perspective and shed more light on the belief that SGD prefers wide minima. 

   more » « less
5. Asymptotics of sample tail autocorrelations for tail-dependent time series: phase transition and visualization

   https://doi.org/10.1093/biomet/asab038  

   Zhang, Ting (July 2021, Biometrika)

   Summary In this article we develop an asymptotic theory for sample tail autocorrelations of time series data that can exhibit serial dependence in both tail and non-tail regions. Unlike with the traditional autocorrelation function, the study of tail autocorrelations requires a double asymptotic scheme to capture the tail phenomena, and our results do not impose any restrictions on the dependence structure in non-tail regions and allow processes that are not necessarily strongly mixing. The newly developed asymptotic theory reveals a previously undiscovered phase transition phenomenon, where the asymptotic behaviour of sample tail autocorrelations, including their convergence rate, can transition from one phase to another as the lag index moves past the point beyond which serial tail dependence vanishes. The phase transition discovery fills a gap in existing research on tail autocorrelations and can be used to construct the lines of significance, in analogy to the traditional autocorrelation plot, when visualizing sample tail autocorrelations to assess the existence of serial tail dependence or to identify the maximal lag of tail dependence. 

   more » « less