More Like this

1. Computing Optimal Upper Bounds on the H2-norm of ODE-PDE Systems using Linear Partial Inequalities

   https://doi.org/10.1016/j.ifacol.2023.10.538  

   Braghini, Danilo; Peet, Matthew M. (January 2023, IFAC-PapersOnLine)

   Recently, a broad class of linear delayed and ODE-PDEs systems was shown to have an equivalent representation using Partial Integral Equations (PIEs). In this paper, we use this PIE representation, combined with algorithms for convex optimization of Partial Integral (PI) operators to bound the H2-norm for input-output systems of this class. Specifically, the methods proposed here apply to delayed and ODE-PDE systems (including delayed PDE systems) in one or two spatial variables where the disturbance does not enter through the boundary. For such systems, we define a notion of H2-norm using an initial state-to-output framework and show that this notion reduces to more traditional concepts under the assumption of existence of a strongly continuous semigroup. Next, we consider input-output systems for which there exists a PIE representation and for such systems show that computing a minimal upper bound on the H2-norm of delayed and PDE systems can be equivalently formulated as a convex optimization problem subject to linear PI operator inequalities (LPIs). We convert, then, these optimization problems to Semi-Definite Programming (SDP) problems using the PIETOOLS toolbox. Finally, we apply the results to several numerical examples – focusing on time-delay systems (TDS) for which comparable H2 approximation results are available in the literature. The numerical results demonstrate the accuracy of the computed upper bound on the H2-norm. 

   more » « less
2. Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs.

   Ergen, T.; Pilanci, M. (January 2021, International Conference on Machine Learning (ICML) 2021)

   Understanding the fundamental mechanism behind the success of deep neural networks is one of the key challenges in the modern machine learning literature. Despite numerous attempts, a solid theoretical analysis is yet to be developed. In this paper, we develop a novel unified framework to reveal a hidden regularization mechanism through the lens of convex optimization. We first show that the training of multiple threelayer ReLU sub-networks with weight decay regularization can be equivalently cast as a convex optimization problem in a higher dimensional space, where sparsity is enforced via a group `1- norm regularization. Consequently, ReLU networks can be interpreted as high dimensional feature selection methods. More importantly, we then prove that the equivalent convex problem can be globally optimized by a standard convex optimization solver with a polynomial-time complexity with respect to the number of samples and data dimension when the width of the network is fixed. Finally, we numerically validate our theoretical results via experiments involving both synthetic and real datasets. 

   more » « less
3. Using SOS and Sublevel Set Volume Minimization for Estimation of Forward Reachable Sets

   Jones, M.; Peet, M. (January 2019, IFAC proceedings series)

   In this paper we propose a convex Sum-of-Squares optimization problem for finding outer approximations of forward reachable sets for nonlinear uncertain Ordinary Differential Equations (ODE’s) with either (or both) L2 or point-wise bounded input disturbances. To make our approximations tight we seek to minimize the volume of our approximation set. Our approach to volume minimization is based on the use of a convex determinant-like objective function.We provide several numerical examples including the Lorenz system and the Van der Pol oscillator. 

   more » « less
4. Integrative Generalized Convex Clustering Optimization and Feature Selection for Mixed Multi-View Data

   Wang, Minjie; Allen, Genevera I. (January 2021, Journal of Machine Learning Research)

   In mixed multi-view data, multiple sets of diverse features are measured on the same set of samples. By integrating all available data sources, we seek to discover common group structure among the samples that may be hidden in individualistic cluster analyses of a single data view. While several techniques for such integrative clustering have been explored, we propose and develop a convex formalization that enjoys strong empirical performance and inherits the mathematical properties of increasingly popular convex clustering methods. Specifically, our Integrative Generalized Convex Clustering Optimization (iGecco) method employs different convex distances, losses, or divergences for each of the different data views with a joint convex fusion penalty that leads to common groups. Additionally, integrating mixed multi-view data is often challenging when each data source is high-dimensional. To perform feature selection in such scenarios, we develop an adaptive shifted group-lasso penalty that selects features by shrinking them towards their loss-specific centers. Our so-called iGecco+ approach selects features from each data view that are best for determining the groups, often leading to improved integrative clustering. To solve our problem, we develop a new type of generalized multi-block ADMM algorithm using sub-problem approximations that more efficiently fits our model for big data sets. Through a series of numerical experiments and real data examples on text mining and genomics, we show that iGecco+ achieves superior empirical performance for high-dimensional mixed multi-view data. 

   more » « less
5. Edge Selections in Bilinear Dynamic Networks

   https://doi.org/10.1109/TAC.2023.3269323  

   de Oliveira, Arthur Castello; Siami, Milad; Sontag, Eduardo D. (April 2023, IEEE Transactions on Automatic Control)

   We develop some basic principles for the design and robustness analysis of a continuous-time bilinear dynamical network, where an attacker can manipulate the strength of the interconnections/edges between some of the agents/nodes. We formulate the edge protection optimization problem of picking a limited number of attack-free edges and minimizing the impact of the attack over the bilinear dynamical network. In particular, the H2-norm of bilinear systems is known to capture robustness and performance properties analogous to its linear counterpart and provides valuable insights for identifying which edges are most sensitive to attacks. The exact optimization problem is combinatorial in the number of edges, and brute-force approaches show poor scalability. However, we show that the H2-norm as a cost function is supermodular and, therefore, allows for efficient greedy approximations of the optimal solution. We illustrate and compare the effectiveness of our theoretical findings via numerical simulation 

   more » « less