Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to nonfederal websites. Their policies may differ from this site.

We study the problem of solving strongly convex and smooth unconstrained optimization problems using stochastic firstorder algorithms. We devise a novel algorithm, referred to as \emph{Recursive OneOverT SGD} (\ROOTSGD), based on an easily implementable, recursive averaging of past stochastic gradients. We prove that it simultaneously achieves stateoftheart performance in both a finitesample, nonasymptotic sense and an asymptotic sense. On the nonasymptotic side, we prove risk bounds on the last iterate of \ROOTSGD with leadingorder terms that match the optimal statistical risk with a unity prefactor, along with a higherorder term that scales at the sharp rate of O(n−3/2) under the Lipschitz condition on the Hessian matrix. On the asymptotic side, we show that when a mild, onepoint Hessian continuity condition is imposed, the rescaled last iterate of (multiepoch) \ROOTSGD converges asymptotically to a Gaussian limit with the Cram\'{e}rRao optimal asymptotic covariance, for a broad range of stepsize choices.Free, publiclyaccessible full text available July 1, 2023

Loh, P ; Raginsky, M. (Ed.)We study the problem of solving strongly convex and smooth unconstrained optimization problems using stochastic firstorder algorithms. We devise a novel algorithm, referred to as \emph{Recursive OneOverT SGD} (\ROOTSGD), based on an easily implementable, recursive averaging of past stochastic gradients. We prove that it simultaneously achieves stateoftheart performance in both a finitesample, nonasymptotic sense and an asymptotic sense. On the nonasymptotic side, we prove risk bounds on the last iterate of \ROOTSGD with leadingorder terms that match the optimal statistical risk with a unity prefactor, along with a higherorder term that scales at the sharp rate of $O(n^{3/2})$ under the Lipschitz condition on the Hessian matrix. On the asymptotic side, we show that when a mild, onepoint Hessian continuity condition is imposed, the rescaled last iterate of (multiepoch) \ROOTSGD converges asymptotically to a Gaussian limit with the Cram\'{e}rRao optimal asymptotic covariance, for a broad range of stepsize choices.Free, publiclyaccessible full text available July 1, 2023

Free, publiclyaccessible full text available January 1, 2023

Free, publiclyaccessible full text available January 1, 2023