More Like this

1. Radial partitioning with spectral penalty parameter selection in distributed optimization for power systems

   https://doi.org/10.1016/j.segan.2024.101613  

   Karimi, Mehdi (March 2025, Sustainable Energy, Grids and Networks)

   This paper introduces a novel concept of intelligent partitioning for group-based distributed optimization (DO) algorithms applied to optimal power flow (OPF) problems. Radial partitioning of the graph of a network is introduced as a systematic way to split a large-scale problem into more tractable sub-problems, which can potentially be solved efficiently with methods such as convex relaxations. Spectral parameter selection is introduced for group-based DO as a crucial hyper-parameter selection step in DO. A software package DiCARP is created, which is implemented in Python using the Pyomo optimization package. Through several numerical examples, we compare the proposed group-based algorithm to component-based approaches, evaluate our radial partitioning method against other partitioning strategies, and assess adaptive parameter selection in comparison to non-adaptive methods. The results highlight the critical role of effective partitioning and parameter selection in solving large-scale network optimization problems. 

   more » « less
2. Distributing Frank–Wolfe via Map-Reduce

   https://doi.org/10.1007/s10115-018-1294-7  

   Moharrer, Armin; Ioannidis, Stratis (December 2018, Knowledge and Information Systems)

   Large-scale optimization problems abound in data mining and machine learning applications, and the computational challenges they pose are often addressed through parallelization. We identify structural properties under which a convex optimization problem can be massively parallelized via map-reduce operations using the Frank–Wolfe (FW) algorithm. The class of problems that can be tackled this way is quite broad and includes experimental design, AdaBoost, and projection to a convex hull. Implementing FW via map-reduce eases parallelization and deployment via commercial distributed computing frameworks. We demonstrate this by implementing FW over Spark, an engine for parallel data processing, and establish that parallelization through map-reduce yields significant performance improvements: We solve problems with 20 million variables using 350 cores in 79 min; the same operation takes 48 h when executed serially. 

   more » « less
3. HyperPlan: A Framework for Motion Planning Algorithm Selection and Parameter Optimization

   https://doi.org/10.1109/IROS51168.2021.9636651  

   Moll, Mark; Chamzas, Constantinos; Kingston, Zachary; Kavraki, Lydia E. (September 2021, IEEE/RSJ International Conference on Intelligent Robots and Systems)

   Over the years, many motion planning algorithms have been proposed. It is often unclear which algorithm might be best suited for a particular class of problems. The problem is compounded by the fact that algorithm performance can be highly dependent on parameter settings. This paper shows that hyperparameter optimization is an effective tool in both algorithm selection and parameter tuning over a given set of motion planning problems. We present different loss functions for optimization that capture different notions of optimality. The approach is evaluated on a broad range of scenes using two different manipulators, a Fetch and a Baxter. We show that optimized planning algorithm performance significantly improves upon baseline performance and generalizes broadly in the sense that performance improvements carry over to problems that are very different from the ones considered during optimization. 

   more » « less
4. Equivalence of ML decoding to a continuous optimization problem

   S. Sundararajan, S N. (January 2020, IEEE International Symposium on Information Theory (ISIT))

   Maximum likelihood (ML) and symbolwise maximum aposteriori (MAP) estimation for discrete input sequences play a central role in a number of applications that arise in communications, information and coding theory. Many instances of these problems are proven to be intractable, for example through reduction to NP-complete integer optimization problems. In this work, we prove that the ML estimation of a discrete input sequence (with no assumptions on the encoder/channel used) is equivalent to the solution of a continuous non-convex optimization problem, and that this formulation is closely related to the computation of symbolwise MAP estimates. This equivalence is particularly useful in situations where a function we term the expected likelihood is efficiently computable. In such situations, we give a ML heuristic and show numerics for sequence estimation over the deletion channel. 

   more » « less
5. Hyperparameter Estimation in Bayesian MAP Estimation: Parameterizations and Consistency

   https://doi.org/10.5802/smai-jcm.62  

   Dunlop, Matthew M.; Helin, Tapio; Stuart, Andrew M. (April 2020, The SMAI journal of computational mathematics)

   null (Ed.)

   The Bayesian formulation of inverse problems is attractive for three primary reasons: it provides a clear modelling framework; it allows for principled learning of hyperparameters; and it can provide uncertainty quantification. The posterior distribution may in principle be sampled by means of MCMC or SMC methods, but for many problems it is computationally infeasible to do so. In this situation maximum a posteriori (MAP) estimators are often sought. Whilst these are relatively cheap to compute, and have an attractive variational formulation, a key drawback is their lack of invariance under change of parameterization; it is important to study MAP estimators, however, because they provide a link with classical optimization approaches to inverse problems and the Bayesian link may be used to improve upon classical optimization approaches. The lack of invariance of MAP estimators under change of parameterization is a particularly significant issue when hierarchical priors are employed to learn hyperparameters. In this paper we study the effect of the choice of parameterization on MAP estimators when a conditionally Gaussian hierarchical prior distribution is employed. Specifically we consider the centred parameterization, the natural parameterization in which the unknown state is solved for directly, and the noncentred parameterization, which works with a whitened Gaussian as the unknown state variable, and arises naturally when considering dimension-robust MCMC algorithms; MAP estimation is well-defined in the nonparametric setting only for the noncentred parameterization. However, we show that MAP estimates based on the noncentred parameterization are not consistent as estimators of hyperparameters; conversely, we show that limits of finite-dimensional centred MAP estimators are consistent as the dimension tends to infinity. We also consider empirical Bayesian hyperparameter estimation, show consistency of these estimates, and demonstrate that they are more robust with respect to noise than centred MAP estimates. An underpinning concept throughout is that hyperparameters may only be recovered up to measure equivalence, a well-known phenomenon in the context of the Ornstein–Uhlenbeck process. The applicability of the results is demonstrated concretely with the study of hierarchical Whittle–Matérn and ARD priors. 

   more » « less