More Like this

1. A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression

   https://doi.org/10.1145/3744970.3727269  

   Zhang, Yixuan; Huo, Dongyan Lucy; Chen, Yudong; Xie, Qiaomin (June 2025, ACM SIGMETRICS Performance Evaluation Review)

   Motivated by robust and quantile regression problems, we investigate the stochastic gradient descent (SGD) algorithm for minimizing an objective functionfthat is locally strongly convex with a sub--quadratic tail. This setting covers many widely used online statistical methods. We introduce a novel piecewise Lyapunov function that enables us to handle functionsfwith only first-order differentiability, which includes a wide range of popular loss functions such as Huber loss. Leveraging our proposed Lyapunov function, we derive finite-time moment bounds under general diminishing stepsizes, as well as constant stepsizes. We further establish the weak convergence, central limit theorem and bias characterization under constant stepsize, providing the first geometrical convergence result for sub--quadratic SGD. Our results have wide applications, especially in online statistical methods. In particular, we discuss two applications of our results. 1) Online robust regression: We consider a corrupted linear model with sub--exponential covariates and heavy--tailed noise. Our analysis provides convergence rates comparable to those for corrupted models with Gaussian covariates and noise. 2) Online quantile regression: Importantly, our results relax the common assumption in prior work that the conditional density is continuous and provide a more fine-grained analysis for the moment bounds. 

   more » « less
2. On the Quenched CLT for Stationary Markov Chains

   https://doi.org/10.1007/s10959-023-01241-0  

   Peligrad, Magda (May 2023, Journal of Theoretical Probability)

   Fill, James Allen (Ed.)

   In this paper, we give sufficient conditions for the almost sure central limit theorem started at a point, known under the name of quenched central limit theorem. This is achieved by using a new idea of conditioning with respect to both the past and the future of the Markov chain. As applications, we provide a new sufficient projective condition for the quenched CLT. 

   more » « less
3. Local limit theorems for densities in Orlicz spaces

   Bobkov, S. G. (October 2019, Journal of mathematical sciences)

   Necessary and sufficient conditions for the validity of the central limit theorem for densities are considered with respect to the norms in Orlicz spaces. The obtained characterization unites several results due to Gnedenko and Kolmogorov (uniform local limit theorem), Prokhorov (convergence in total variation) and Barron (entropic central limit theorem). 

   more » « less
4. Semiparametric Efficient Inference in Adaptive Experiments

   Cook, Thomas; Mishler, Alan; Ramdas, Aaditya (April 2024, Proceedings of Machine Learning Research)

   We consider the problem of efficient inference of the Average Treatment Effect in a sequential experiment where the policy governing the assignment of subjects to treatment or control can change over time. We first provide a central limit theorem for the Adaptive Augmented Inverse-Probability Weighted estimator, which is semiparametric efficient, under weaker assumptions than those previously made in the literature. This central limit theorem enables efficient inference at fixed sample sizes. We then consider a sequential inference setting, deriving both asymptotic and nonasymptotic confidence sequences that are considerably tighter than previous methods. These anytime-valid methods enable inference under data-dependent stopping times (sample sizes). Additionally, we use propensity score truncation techniques from the recent off-policy estimation literature to reduce the finite sample variance of our estimator without affecting the asymptotic variance. Empirical results demonstrate that our methods yield narrower confidence sequences than those previously developed in the literature while maintaining time-uniform error control. 

   more » « less
5. The local limit theorem on nilpotent Lie groups

   https://doi.org/10.1007/s00440-018-0864-7  

   Hough, Robert (July 2018, Probability Theory and Related Fields)

   A local limit theorem is proven on connected, simply connected nilpotent Lie groups, for a class of generating measures satisfying a moment condition and a condition on the characteristic function of the abelianization. The result extends an earlier local limit theorem of Alexopoulos which treated absolutely continuous measures with a continuous density of compact support, and also extends local limit theorems of Breuillard and Diaconis–Hough which treated general measures on the Heisenberg group. 

   more » « less