Abstract We studyinexactfixed-point proximity algorithms for solving a class of sparse regularization problems involving the$$\ell _0$$ norm. Specifically, the$$\ell _0$$ model has an objective function that is the sum of a convex fidelity term and a Moreau envelope of the$$\ell _0$$ norm regularization term. Such an$$\ell _0$$ model is non-convex. Existing exact algorithms for solving the problems require the availability of closed-form formulas for the proximity operator of convex functions involved in the objective function. When such formulas are not available, numerical computation of the proximity operator becomes inevitable. This leads to inexact iteration algorithms. We investigate in this paper how the numerical error for every step of the iteration should be controlled to ensure global convergence of the inexact algorithms. We establish a theoretical result that guarantees the sequence generated by the proposed inexact algorithm converges to a local minimizer of the optimization problem. We implement the proposed algorithms for three applications of practical importance in machine learning and image science, which include regression, classification, and image deblurring. The numerical results demonstrate the convergence of the proposed algorithm and confirm that local minimizers of the$$\ell _0$$ models found by the proposed inexact algorithm outperform global minimizers of the corresponding$$\ell _1$$ models, in terms of approximation accuracy and sparsity of the solutions.
more »
« less
Comparing solution paths of sparse quadratic minimization with a Stieltjes matrix
Abstract This paper studies several solution paths of sparse quadratic minimization problems as a function of the weighing parameter of the bi-objective of estimation loss versus solution sparsity. Three such paths are considered: the “$$\ell _0$$ -path” where the discontinuous$$\ell _0$$ -function provides the exact sparsity count; the “$$\ell _1$$ -path” where the$$\ell _1$$ -function provides a convex surrogate of sparsity count; and the “capped$$\ell _1$$ -path” where the nonconvex nondifferentiable capped$$\ell _1$$ -function aims to enhance the$$\ell _1$$ -approximation. Serving different purposes, each of these three formulations is different from each other, both analytically and computationally. Our results deepen the understanding of (old and new) properties of the associated paths, highlight the pros, cons, and tradeoffs of these sparse optimization models, and provide numerical evidence to support the practical superiority of the capped$$\ell _1$$ -path. Our study of the capped$$\ell _1$$ -path is interesting in its own right as the path pertains to computable directionally stationary (= strongly locally minimizing in this context, as opposed to globally optimal) solutions of a parametric nonconvex nondifferentiable optimization problem. Motivated by classical parametric quadratic programming theory and reinforced by modern statistical learning studies, both casting an exponential perspective in fully describing such solution paths, we also aim to address the question of whether some of them can be fully traced in strongly polynomial time in the problem dimensions. A major conclusion of this paper is that a path of directional stationary solutions of the capped$$\ell _1$$ -regularized problem offers interesting theoretical properties and practical compromise between the$$\ell _0$$ -path and the$$\ell _1$$ -path. Indeed, while the$$\ell _0$$ -path is computationally prohibitive and greatly handicapped by the repeated solution of mixed-integer nonlinear programs, the quality of$$\ell _1$$ -path, in terms of the two criteria—loss and sparsity—in the estimation objective, is inferior to the capped$$\ell _1$$ -path; the latter can be obtained efficiently by a combination of a parametric pivoting-like scheme supplemented by an algorithm that takes advantage of the Z-matrix structure of the loss function.
more »
« less
- Award ID(s):
- 2006762
- PAR ID:
- 10411443
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- Mathematical Programming
- Volume:
- 204
- Issue:
- 1-2
- ISSN:
- 0025-5610
- Format(s):
- Medium: X Size: p. 517-566
- Size(s):
- p. 517-566
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract This paper introduces a nonconvex approach for sparse signal recovery, proposing a novel model termed the$$\tau _2$$ -model, which utilizes the squared$$\ell _1/\ell _2$$ norms for this purpose. Our model offers an advancement over the$$\ell _0$$ norm, which is often computationally intractable and less effective in practical scenarios. Grounded in the concept of effective sparsity, our approach robustly measures the number of significant coordinates in a signal, making it a powerful alternative for sparse signal estimation. The$$\tau _2$$ -model is particularly advantageous due to its computational efficiency and practical applicability. We detail two accompanying algorithms based on Dinkelbach’s procedure and a difference of convex functions strategy. The first algorithm views the model as a linear-constrained quadratic programming problem in noiseless scenarios and as a quadratic-constrained quadratic programming problem in noisy scenarios. The second algorithm, capable of handling both noiseless and noisy cases, is based on the alternating direction linearized proximal method of multipliers. We also explore the model’s properties, including the existence of solutions under certain conditions, and discuss the convergence properties of the algorithms. Numerical experiments with various sensing matrices validate the effectiveness of our proposed model.more » « less
-
Abstract We study the sparsity of the solutions to systems of linear Diophantine equations with and without non-negativity constraints. The sparsity of a solution vector is the number of its nonzero entries, which is referred to as the$$\ell _0$$ -norm of the vector. Our main results are new improved bounds on the minimal$$\ell _0$$ -norm of solutions to systems$$A\varvec{x}=\varvec{b}$$ , where$$A\in \mathbb {Z}^{m\times n}$$ ,$${\varvec{b}}\in \mathbb {Z}^m$$ and$$\varvec{x}$$ is either a general integer vector (lattice case) or a non-negative integer vector (semigroup case). In certain cases, we give polynomial time algorithms for computing solutions with$$\ell _0$$ -norm satisfying the obtained bounds. We show that our bounds are tight. Our bounds can be seen as functions naturally generalizing the rank of a matrix over$$\mathbb {R}$$ , to other subdomains such as$$\mathbb {Z}$$ . We show that these new rank-like functions are all NP-hard to compute in general, but polynomial-time computable for fixed number of variables.more » « less
-
Abstract We study holomorphic mapsFfrom a smooth Levi non-degenerate real hypersurface$$ M_{\ell }\subset {\mathbb {C}}^n $$ into a hyperquadric$$ {\mathbb {H}}_{\ell '}^N $$ with signatures$$ \ell \le (n-1)/2 $$ and$$ \ell '\le (N-1)/2,$$ respectively. Assuming that$$ N - n < n - 1,$$ we prove that if$$ \ell = \ell ',$$ thenFis either CR transversal to$$ {\mathbb {H}}_{\ell }^N $$ at every point of$$ M_{\ell },$$ or it maps a neighborhood of$$ M_{\ell } $$ in$$ {\mathbb {C}}^n $$ into$$ {\mathbb {H}}_{\ell }^N.$$ Furthermore, in the case where$$ \ell ' > \ell ,$$ we show that ifFis not CR transversal at$$0\in M_\ell ,$$ then it must be transversally flat. The latter is best possible.more » « less
-
Abstract LetXbe a compact normal complex space of dimensionnandLbe a holomorphic line bundle onX. Suppose that$$\Sigma =(\Sigma _1,\ldots ,\Sigma _\ell )$$ is an$$\ell $$ -tuple of distinct irreducible proper analytic subsets ofX,$$\tau =(\tau _1,\ldots ,\tau _\ell )$$ is an$$\ell $$ -tuple of positive real numbers, and let$$H^0_0(X,L^p)$$ be the space of holomorphic sections of$$L^p:=L^{\otimes p}$$ that vanish to order at least$$\tau _jp$$ along$$\Sigma _j$$ ,$$1\le j\le \ell $$ . If$$Y\subset X$$ is an irreducible analytic subset of dimensionm, we consider the space$$H^0_0 (X|Y, L^p)$$ of holomorphic sections of$$L^p|_Y$$ that extend to global holomorphic sections in$$H^0_0(X,L^p)$$ . Assuming that the triplet$$(L,\Sigma ,\tau )$$ is big in the sense that$$\dim H^0_0(X,L^p)\sim p^n$$ , we give a general condition onYto ensure that$$\dim H^0_0(X|Y,L^p)\sim p^m$$ . WhenLis endowed with a continuous Hermitian metric, we show that the Fubini-Study currents of the spaces$$H^0_0(X|Y,L^p)$$ converge to a certain equilibrium current onY. We apply this to the study of the equidistribution of zeros inYof random holomorphic sections in$$H^0_0(X|Y,L^p)$$ as$$p\rightarrow \infty $$ .more » « less
An official website of the United States government
