%APilanci, Mert%AErgen, Tolga%Anull Ed.%BJournal Name: International Conference on Machine Learning
%D2020%I
%JJournal Name: International Conference on Machine Learning
%K
%MOSTI ID: 10206903
%PMedium: X
%TNeural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-Layer Networks
%XWe develop exact representations of training twolayer neural networks with rectified linear units (ReLUs) in terms of a single convex program with number of variables polynomial in the number of training samples and the number of hidden neurons. Our theory utilizes semi-infinite duality and minimum norm regularization. We show that ReLU networks trained with standard weight decay are equivalent to block `1 penalized convex models. Moreover, we show that certain standard convolutional linear networks are equivalent semidefinite programs which can be simplified to `1 regularized linear models in a polynomial sized discrete Fourier feature space.
%0Journal Article
Country unknown/Code not availableOSTI-MSA