An official website of the United States government
Abstract We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $$L^1+\text{(nonlocal)}\operatorname{TV}$$, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense) and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.
more »
« less
Stable Conical Regularization by Constructible Dilating Cones with an Application to $L^{p}$-constrained Optimization Problems
We study a convex constrained optimization problem that suffers from the lack of Slater-type constraint qualification. By employing a constructible representation of the constraint cone, we devise a new family of dilating cones and use it to introduce a family of regularized problems. We establish novel stability estimates for the regularized problems in terms of the regularization parameter. To show the feasibility and efficiency of the proposed framework, we present applications to some Lp-constrained least-squares problems.
more »
« less
- Award ID(s):
- 1720067
- PAR ID:
- 10096221
- Date Published:
- Journal Name:
- Taiwanese Journal of Mathematics
- ISSN:
- 1027-5487
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract We present a new feasible proximal gradient method for constrained optimization where both the objective and constraint functions are given by summation of a smooth, possibly nonconvex function and a convex simple function. The algorithm converts the original problem into a sequence of convex subproblems. Formulating those subproblems requires the evaluation of at most one gradient-value of the original objective and constraint functions. Either exact or approximate subproblems solutions can be computed efficiently in many cases. An important feature of the algorithm is the constraint level parameter. By carefully increasing this level for each subproblem, we provide a simple solution to overcome the challenge of bounding the Lagrangian multipliers and show that the algorithm follows a strictly feasible solution path till convergence to the stationary point. We develop a simple, proximal gradient descent type analysis, showing that the complexity bound of this new algorithm is comparable to gradient descent for the unconstrained setting which is new in the literature. Exploiting this new design and analysis technique, we extend our algorithms to some more challenging constrained optimization problems where (1) the objective is a stochastic or finite-sum function, and (2) structured nonsmooth functions replace smooth components of both objective and constraint functions. Complexity results for these problems also seem to be new in the literature. Finally, our method can also be applied to convex function constrained problems where we show complexities similar to the proximal gradient method.more » « less
-
We consider various versions of the obstacle and thin-obstacle problems, we interpret them as variational inequalities, with non-smooth constraint, and prove that they satisfy a new constrained Łojasiewicz inequality. The difficulty lies in the fact that, since the constraint is non-analytic, the pioneering method of L. Simon ([]) does not apply and we have to exploit a better understanding on the constraint itself. We then apply this inequality to two associated problems. First we combine it with an abstract result on parabolic variational inequalities, to prove the convergence at infinity of the strong global solutions to the parabolic obstacle and thin-obstacle problems to a unique stationary solution with a rate. Secondly, we give an abstract proof, based on a parabolic approach, of the epiperimetric inequality, which we then apply to the singular points of the obstacle and thin-obstacle problems.more » « less
-
The problem of sampling constrained continuous distributions has frequently appeared in many machine/statistical learning models. Many Markov Chain Monte Carlo (MCMC) sampling methods have been adapted to handle different types of constraints on random variables. Among these methods, Hamilton Monte Carlo (HMC) and the related approaches have shown significant advantages in terms of computational efficiency compared with other counterparts. In this article, we first review HMC and some extended sampling methods, and then we concretely explain three constrained HMC-based sampling methods, reflection, reformulation, and spherical HMC. For illustration, we apply these methods to solve three well-known constrained sampling problems, truncated multivariate normal distributions, Bayesian regularized regression, and nonparametric density estimation. In this review, we also connect constrained sampling with another similar problem in the statistical design of experiments with constrained design space.more » « less
-
Summary A method is developed to numerically solve chance constrained optimal control problems. The chance constraints are reformulated as nonlinear constraints that retain the probability properties of the original constraint. The reformulation transforms the chance constrained optimal control problem into a deterministic optimal control problem that can be solved numerically. The new method developed in this paper approximates the chance constraints using Markov Chain Monte Carlo sampling and kernel density estimators whose kernels have integral functions that bound the indicator function. The nonlinear constraints resulting from the application of kernel density estimators are designed with bounds that do not violate the bounds of the original chance constraint. The method is tested on a nontrivial chance constrained modification of a soft lunar landing optimal control problem and the results are compared with results obtained using a conservative deterministic formulation of the optimal control problem. Additionally, the method is tested on a complex chance constrained unmanned aerial vehicle problem. The results show that this new method can be used to reliably solve chance constrained optimal control problems.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.11650/tjm/181103
