More Like this

1. Quasi-Newton Stochastic Optimization Algorithm for Parameter Estimation of a Stochastic Model of the Budding Yeast Cell Cycle

   https://doi.org/10.1109/TCBB.2017.2773083  

   Chen, Minghan ; Amos, Brandon D. ; Watson, Layne T. ; Tyson, John J. ; Cao, Young ; Shaffer, Clifford A. ; Trosset, Michael W. ; Oguz, Cihan ; Kakoti, Gisella ( January 2019 , IEEE/ACM Transactions on Computational Biology and Bioinformatics)
2. Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning: Examining Distributed and Centralized Stochastic Gradient Descent

   https://doi.org/10.1109/MSP.2020.2975212  

   Pu, Shi ; Olshevsky, Alex ; Paschalidis, Ioannis Ch. ( May 2020 , IEEE Signal Processing Magazine)
3. High-Dimensional Nonconvex Stochastic Optimization by Doubly Stochastic Successive Convex Approximation

   https://doi.org/10.1109/TSP.2020.3033354  

   Mokhtari, Aryan ; Koppel, Alec ( January 2020 , IEEE Transactions on Signal Processing)

   null (Ed.)
4. Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Nonconvex Stochastic Optimization: Nonasymptotic Performance Bounds and Momentum-Based Acceleration

   https://doi.org/10.1287/opre.2021.2162  

   Gao, Xuefeng ; Gürbüzbalaban, Mert ; Zhu, Lingjiong ( January 2021 , Operations Research)

   Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant of stochastic gradients with momentum where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates toward a global minimum. Many works report its empirical success in practice for solving stochastic nonconvex optimization problems; in particular, it has been observed to outperform overdamped Langevin Monte Carlo–based methods, such as stochastic gradient Langevin dynamics (SGLD), in many applications. Although the asymptotic global convergence properties of SGHMC are well known, its finite-time performance is not well understood. In this work, we study two variants of SGHMC based on two alternative discretizations of the underdamped Langevin diffusion. We provide finite-time performance bounds for the global convergence of both SGHMC variants for solving stochastic nonconvex optimization problems with explicit constants. Our results lead to nonasymptotic guarantees for both population and empirical risk minimization problems. For a fixed target accuracy level on a class of nonconvex problems, we obtain complexity bounds for SGHMC that can be tighter than those available for SGLD. 

   more » « less
5. Stochastic Gradient Descent on a Tree: an Adaptive and Robust Approach to Stochastic Convex Optimization

   https://doi.org/10.1109/ALLERTON.2019.8919740  

   Vakili, Sattar ; Salgia, Sudeep ; Zhao, Qing ( September 2019 , 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton))