skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


This content will become publicly available on June 16, 2026

Title: A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression
Motivated by robust and quantile regression problems, we investigate the stochastic gradient descent (SGD) algorithm for minimizing an objective functionfthat is locally strongly convex with a sub--quadratic tail. This setting covers many widely used online statistical methods. We introduce a novel piecewise Lyapunov function that enables us to handle functionsfwith only first-order differentiability, which includes a wide range of popular loss functions such as Huber loss. Leveraging our proposed Lyapunov function, we derive finite-time moment bounds under general diminishing stepsizes, as well as constant stepsizes. We further establish the weak convergence, central limit theorem and bias characterization under constant stepsize, providing the first geometrical convergence result for sub--quadratic SGD. Our results have wide applications, especially in online statistical methods. In particular, we discuss two applications of our results. 1) Online robust regression: We consider a corrupted linear model with sub--exponential covariates and heavy--tailed noise. Our analysis provides convergence rates comparable to those for corrupted models with Gaussian covariates and noise. 2) Online quantile regression: Importantly, our results relax the common assumption in prior work that the conditional density is continuous and provide a more fine-grained analysis for the moment bounds.  more » « less
Award ID(s):
2432546 2339794 1955997
PAR ID:
10610275
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
ACM SIGMETRICS Performance Evaluation Review
Date Published:
Journal Name:
ACM SIGMETRICS Performance Evaluation Review
Volume:
53
Issue:
1
ISSN:
0163-5999
Page Range / eLocation ID:
85 to 87
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This Letter presents a novel, to the best of our knowledge, method to calibrate multi-focus microscopic structured-light three-dimensional (3D) imaging systems with an electrically adjustable camera focal length. We first leverage the conventional method to calibrate the system with a reference focal lengthf0. Then we calibrate the system with other discrete focal lengthsfiby determining virtual features on a reconstructed white plane usingf0. Finally, we fit the polynomial function model using the discrete calibration results forfi. Experimental results demonstrate that our proposed method can calibrate the system consistently and accurately. 
    more » « less
  2. Stochastic Gradient Descent (SGD) has played a central role in machine learning. However, it requires a carefully hand-picked stepsize for fast convergence, which is notoriously tedious and time-consuming to tune. Over the last several years, a plethora of adaptive gradient-based algorithms have emerged to ameliorate this problem. In this paper, we propose new surrogate losses to cast the problem of learning the optimal stepsizes for the stochastic optimization of a non-convex smooth objective function onto an online convex optimization problem. This allows the use of no-regret online algorithms to compute optimal stepsizes on the fly. In turn, this results in a SGD algorithm with self-tuned stepsizes that guarantees convergence rates that are automatically adaptive to the level of noise. 
    more » « less
  3. We present a convergence rate analysis for biased stochastic gradient descent (SGD), where individual gradient updates are corrupted by computation errors. We develop stochastic quadratic constraints to formulate a small linear matrix inequality (LMI) whose feasible points lead to convergence bounds of biased SGD. Based on this LMI condition, we develop a sequential minimization approach to analyze the intricate trade-offs that couple stepsize selection, convergence rate, optimization accuracy, and robustness to gradient inaccuracy. We also provide feasible points for this LMI and obtain theoretical formulas that quantify the convergence properties of biased SGD under various assumptions on the loss functions. 
    more » « less
  4. Abstract Expected shortfall (ES), also known as superquantile or conditional value-at-risk, is an important measure in risk analysis and stochastic optimisation and has applications beyond these fields. In finance, it refers to the conditional expected return of an asset given that the return is below some quantile of its distribution. In this paper, we consider a joint regression framework recently proposed to model the quantile and ES of a response variable simultaneously, given a set of covariates. The current state-of-the-art approach to this problem involves minimising a non-differentiable and non-convex joint loss function, which poses numerical challenges and limits its applicability to large-scale data. Motivated by the idea of using Neyman-orthogonal scores to reduce sensitivity to nuisance parameters, we propose a statistically robust and computationally efficient two-step procedure for fitting joint quantile and ES regression models that can handle highly skewed and heavy-tailed data. We establish explicit non-asymptotic bounds on estimation and Gaussian approximation errors that lay the foundation for statistical inference, even with increasing covariate dimensions. Finally, through numerical experiments and two data applications, we demonstrate that our approach well balances robustness, statistical, and numerical efficiencies for expected shortfall regression. 
    more » « less
  5. We describe the confining instabilities of a proposed quantum spin liquid underlying the pseudogap metal state of the hole-doped cuprates. The spin liquid can be described by a SU(2) gauge theory ofNf= 2 massless Dirac fermions carrying fundamental gauge charges—this is the low-energy theory of a mean-field state of fermionic spinons moving on the square lattice withπ-flux per plaquette in the ℤ2center of SU(2). This theory has an emergent SO(5)fglobal symmetry and is presumed to confine at low energies to the Néel state. At nonzero doping (or smaller Hubbard repulsionUat half-filling), we argue that confinement occurs via the Higgs condensation of bosonic chargons carrying fundamental SU(2) gauge charges also moving inπℤ2-flux. At half-filling, the low-energy theory of the Higgs sector hasNb= 2 relativistic bosons with a possible emergent SO(5)bglobal symmetry describing rotations between ad-wave superconductor, period-2 charge stripes, and the time-reversal breaking “d-density wave” state. We propose a conformal SU(2) gauge theory withNf= 2 fundamental fermions,Nb= 2 fundamental bosons, and a SO(5)f×SO(5)bglobal symmetry, which describes a deconfined quantum critical point between a confining state which breaks SO(5)fand a confining state which breaks SO(5)b. The pattern of symmetry breaking within both SO(5)s is determined by terms likely irrelevant at the critical point, which can be chosen to obtain a transition between Néel order andd-wave superconductivity. A similar theory applies at nonzero doping and largeU, with longer-range couplings of the chargons leading to charge order with longer periods. 
    more » « less