More Like this

1. Distributed block-diagonal approximation methods for regularized empirical risk minimization

   https://doi.org/10.1007/s10994-019-05859-2  

   Lee, Ching-pei; Chang, Kai-Wei (January 2019, Machine Learning)

   In recent years, there is a growing need to train machine learning models on a huge volume of data. Therefore, designing efficient distributed optimization algorithms for empirical risk minimization (ERM) has become an active and challenging research topic. In this paper, we propose a flexible framework for distributed ERM training through solving the dual problem, which provides a unified description and comparison of existing methods. Our approach requires only approximate solutions of the sub-problems involved in the optimization process, and is versatile to be applied on many large-scale machine learning problems including classification, regression, and structured prediction. We show that our framework enjoys global linear convergence for a broad class of non-strongly-convex problems, and some specific choices of the sub-problems can even achieve much faster convergence than existing approaches by a refined analysis. This improved convergence rate is also reflected in the superior empirical performance of our method. 

   more » « less
2. On the numerical solution of nonlinear eigenvalue problems for the Monge-Ampère operator

   https://doi.org/10.1051/cocv/2020072  

   Glowinski, Roland; Leung, Shingyu; Liu, Hao; Qian, Jianliang (January 2020, ESAIM: Control, Optimisation and Calculus of Variations)

   Buttazzo, G.; Casas, E.; de Teresa, L.; Glowinsk, R.; Leugering, G.; Trélat, E.; Zhang, X. (Ed.)

   In this article, we report the results we obtained when investigating the numerical solution of some nonlinear eigenvalue problems for the Monge-Ampère operator v → det D 2 v . The methodology we employ relies on the following ingredients: (i) a divergence formulation of the eigenvalue problems under consideration. (ii) The time discretization by operator-splitting of an initial value problem (a kind of gradient flow) associated with each eigenvalue problem. (iii) A finite element approximation relying on spaces of continuous piecewise affine functions. To validate the above methodology, we applied it to the solution of problems with known exact solutions: The results we obtained suggest convergence to the exact solution when the space discretization step h → 0. We considered also test problems with no known exact solutions. 

   more » « less
3. An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis

   https://doi.org/10.1287/ijoo.2019.0032  

   Chen, Shixiang; Ma, Shiqian; Xue, Lingzhou; Zou, Hui (January 2020, INFORMS Journal on Optimization)

   Sparse principal component analysis and sparse canonical correlation analysis are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data. Both problems can be formulated as an optimization problem with nonsmooth objective and nonconvex constraints. Because nonsmoothness and nonconvexity bring numerical difficulties, most algorithms suggested in the literature either solve some relaxations of them or are heuristic and lack convergence guarantees. In this paper, we propose a new alternating manifold proximal gradient method to solve these two high-dimensional problems and provide a unified convergence analysis. Numerical experimental results are reported to demonstrate the advantages of our algorithm. 

   more » « less
4. Newton-Anderson at Singular Points

   https://doi.org/10.4208/ijnam2023-1029  

   Dallas, Matt; Pollock, Sara (October 2023, International Journal of Numerical Analysis and Modeling)

   In this paper we develop convergence and acceleration theory for Anderson acceleration applied to Newton’s method for nonlinear systems in which the Jacobian is singular at a solution. For these problems, the standard Newton algorithm converges linearly in a region about the solution; and, it has been previously observed that Anderson acceleration can substantially improve convergence without additional a priori knowledge, and with little additional computation cost. We present an analysis of the Newton-Anderson algorithm in this context, and introduce a novel and theoretically supported safeguarding strategy. The convergence results are demonstrated with the Chandrasekhar H-equation and a variety of benchmark examples. 

   more » « less
5. A note on local convergence of iterative processes for pipe network analysis

   https://doi.org/10.1007/s40314-025-03158-2  

   Luu, Huong; Chrobak, Marek (April 2025, Computational and Applied Mathematics)

   Abstract Analysis of pipe networks involves computing flow rates and pressure differences on pipe segments in the network, given the external inflow/outflow values. This analysis can be conducted using iterative methods, among which the algorithms of Hardy Cross and Newton-Raphson have historically been applied in practice. In this note, we address the mathematical analysis of the local convergence of these algorithms. The loop-based Newton–Raphson algorithm converges quadratically fast, and we provide estimates for its convergence radius to correct some estimates in the previous literature. In contrast, we show that the convergence of the Hardy Cross algorithm is only linear. This provides theoretical confirmation of experimental observations reported earlier in the literature. 

   more » « less