We introduce a new technique for reducing the dimension of the ambient space of lowdegree polynomials in the Gaussian space while preserving their relative correlation structure. As an application, we obtain an explicit upper bound on the dimension of an epsilonoptimal noisestable Gaussian partition. In fact, we address the more general problem of upper bounding the number of samples needed to epsilonapproximate any joint distribution that can be noninteractively simulated from a correlated Gaussian source. Our results significantly improve (from Ackermannlike to "merely" exponential) the upper bounds recently proved on the above problems by De, Mossel & Neeman [CCC 2017, SODA 2018 resp.] and imply decidability of the larger alphabet case of the gap noninteractive simulation problem posed by Ghazi, Kamath & Sudan [FOCS 2016]. Our technique of dimension reduction for lowdegree polynomials is simple and can be seen as a generalization of the JohnsonLindenstrauss lemma and could be of independent interest.
Monotone circuit lower bounds from resolution
We introduce a new technique for reducing the dimension of the ambient space of lowdegree polynomials in the Gaussian space while preserving their relative correlation structure. As an application, we obtain an explicit upper bound on the dimension of an epsilonoptimal noisestable Gaussian partition. In fact, we address the more general problem of upper bounding the number of samples needed to epsilonapproximate any joint distribution that can be noninteractively simulated from a correlated Gaussian source. Our results significantly improve (from Ackermannlike to "merely" exponential) the upper bounds recently proved on the above problems by De, Mossel & Neeman [CCC 2017, SODA 2018 resp.] and imply decidability of the larger alphabet case of the gap noninteractive simulation problem posed by Ghazi, Kamath & Sudan [FOCS 2016]. Our technique of dimension reduction for lowdegree polynomials is simple and can be seen as a generalization of the JohnsonLindenstrauss lemma and could be of independent interest.
 Publication Date:
 NSFPAR ID:
 10075657
 Journal Name:
 Proceedings of the annual ACM Symposium on Theory of Computing
 ISSN:
 07378017
 Sponsoring Org:
 National Science Foundation
More Like this


We introduce a new technique for reducing the dimension of the ambient space of lowdegree polynomials in the Gaussian space while preserving their relative correlation structure. As an application, we obtain an explicit upper bound on the dimension of an epsilonoptimal noisestable Gaussian partition. In fact, we address the more general problem of upper bounding the number of samples needed to epsilonapproximate any joint distribution that can be noninteractively simulated from a correlated Gaussian source. Our results significantly improve (from Ackermannlike to "merely" exponential) the upper bounds recently proved on the above problems by De, Mossel & Neeman [CCC 2017, SODA 2018 resp.] and imply decidability of the larger alphabet case of the gap noninteractive simulation problem posed by Ghazi, Kamath & Sudan [FOCS 2016]. Our technique of dimension reduction for lowdegree polynomials is simple and can be seen as a generalization of the JohnsonLindenstrauss lemma and could be of independent interest.

Raz, Ran (Ed.)We give upper and lower bounds on the power of subsystems of the Ideal Proof System (IPS), the algebraic proof system recently proposed by Grochow and Pitassi, where the circuits comprising the proof come from various restricted algebraic circuit classes. This mimics an established research direction in the boolean setting for subsystems of Extended Frege proofs whose lines are circuits from restricted boolean circuit classes. Essentially all of the subsystems considered in this paper can simulate the wellstudied Nullstellensatz proof system, and prior to this work there were no known lower bounds when measuring proof size by the algebraic complexity of the polynomials (except with respect to degree, or to sparsity). Our main contributions are two general methods of converting certain algebraic lower bounds into proof complexity ones. Both require stronger arithmetic lower bounds than common, which should hold not for a specific polynomial but for a whole family defined by it. These may be likened to some of the methods by which Boolean circuit lower bounds are turned into related proofcomplexity ones, especially the "feasible interpolation" technique. We establish algebraic lower bounds of these forms for several explicit polynomials, against a variety of classes, and infer the relevant proofmore »

We prove two new results about the inability of lowdegree polynomials to uniformly approximate constantdepth circuits, even to slightlybetterthantrivial error. First, we prove a tight Omega~(n^{1/2}) lower bound on the threshold degree of the SURJECTIVITY function on n variables. This matches the best known threshold degree bound for any AC^0 function, previously exhibited by a much more complicated circuit of larger depth (Sherstov, FOCS 2015). Our result also extends to a 2^{Omega~(n^{1/2})} lower bound on the signrank of an AC^0 function, improving on the previous best bound of 2^{Omega(n^{2/5})} (Bun and Thaler, ICALP 2016). Second, for any delta>0, we exhibit a function f : {1,1}^n > {1,1} that is computed by a circuit of depth O(1/delta) and is hard to approximate by polynomials in the following sense: f cannot be uniformly approximated to error epsilon=12^{Omega(n^{1delta})}, even by polynomials of degree n^{1delta}. Our recent prior work (Bun and Thaler, FOCS 2017) proved a similar lower bound, but which held only for error epsilon=1/3. Our result implies 2^{Omega(n^{1delta})} lower bounds on the complexity of AC^0 under a variety of basic measures such as discrepancy, margin complexity, and threshold weight. This nearly matches the trivial upper bound of 2^{O(n)} that holds for everymore »

We study deep neural networks with polynomial activations, particularly their expressive power. For a fixed architecture and activation degree, a polynomial neural network defines an algebraic map from weights to polynomials. The image of this map is the functional space associated to the network, and it is an irreducible algebraic variety upon taking closure. This paper proposes the dimension of this variety as a precise measure of the expressive power of polynomial neural networks. We obtain several theoretical results regarding this dimension as a function of architecture, including an exact formula for high activation degrees, as well as upper and lower bounds on layer widths in order for deep polynomials networks to fill the ambient functional space. We also present computational evidence that it is profitable in terms of expressiveness for layer widths to increase monotonically and then decrease monotonically. Finally, we link our study to favorable optimization properties when training weights, and we draw intriguing connections with tensor and polynomial decompositions.