Set representation has become ubiquitous in deep learning for modeling the inductive bias of neural networks that are insensitive to the input order. DeepSets is the most widely used neural network architecture for set representation. It involves embedding each set element into a latent space with dimension L, followed by a sum pooling to obtain a whole-set embedding, and finally mapping the whole-set embedding to the output. In this work, we investigate the impact of the dimension L on the expressive power of DeepSets. Previous analyses either oversimplified high-dimensional features to be one-dimensional features or were limited to analytic activations, thereby diverging from practical use or resulting in L that grows exponentially with the set size N and feature dimension D. To investigate the minimal value of L that achieves sufficient expressive power, we present two set-element embedding layers: (a) linear + power activation (LP) and (b) linear + exponential activations (LE). We demonstrate that L being poly(N,D) is sufficient for set representation using both embedding layers. We also provide a lower bound of L for the LP embedding layer. Furthermore, we extend our results to permutation-equivariant set functions and the complex field.
more »
« less
A modulation invariant Carleson embedding theorem outside local L^2
The article arXiv:1309.0945 by Do and Thiele develops a theory of Carleson embeddings in outer Lp spaces for the wave packet transform of functions in L^p, in the 2≤p≤∞ range referred to as local L^2. In this article, we formulate a suitable extension of this theory to exponents 1<2, answering a question posed in arXiv:1309.0945. The proof of our main embedding theorem involves a refined multi-frequency Calder\'on-Zygmund decomposition. We apply our embedding theorem to recover the full known range of Lp estimates for the bilinear Hilbert transforms without reducing to discrete model sums or appealing to generalized restricted weak-type interpolation.
more »
« less
- Award ID(s):
- 1500449
- PAR ID:
- 10017331
- Date Published:
- Journal Name:
- Journal d’analyse mathématique
- ISSN:
- 1565-8538
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Nicola Arcozzi, Pavel Mozolyako, Karl-Mikael Perfekt, and Giulia Sarfatti recently gave the proof of a bi-parameter Carleson embedding theorem. Their proof uses heavily the notion of capacity on the bi-tree. In this note we give another proof of a bi-parameter Carleson embedding theorem that avoids the use of bi-tree capacity. Unlike the proof on a simple tree in a previous paper of the authors (Arcozzi et al. in Bellman function sitting on a tree, arXiv:1809.03397, 2018), which used the Bellman function technique, the proof here is based on some rather subtle comparisons of energies of measures on the bi-tree.more » « less
-
We show that the Priess-Crampe & Ribenboim fixed point theorem is provable in R C A 0 . Furthermore, we show that Caristi’s fixed point theorem for both Baire and Borel functions is equivalent to the transfinite leftmost path principle, which falls strictly between A T R 0 and Π 1 1 - C A 0 . We also exhibit several weakenings of Caristi’s theorem that are equivalent to W K L 0 and to A C A 0 . This article is part of the theme issue ‘Modern perspectives in Proof Theory’.more » « less
-
Abstract This paper continues the study initiated in Davey (Arch Ration Mech Anal 228:159–196, 2018), where a high-dimensional limiting technique was developed and used to prove certain parabolic theorems from their elliptic counterparts. In this article, we extend these ideas to the variable-coefficient setting. This generalized technique is demonstrated through new proofs of three important theorems for variable-coefficient heat operators, one of which establishes a result that is, to the best of our knowledge, also new. Specifically, we give new proofs of$$L^2 \rightarrow L^2$$ Carleman estimates and the monotonicity of Almgren-type frequency functions, and we prove a new monotonicity of Alt–Caffarelli–Friedman-type functions. The proofs in this article rely only on their related elliptic theorems and a limiting argument. That is, each parabolic theorem is proved by taking a high-dimensional limit of a related elliptic result.more » « less
-
Recently, there has been much progress in understanding stationary measures for colored (also called multi-species or multi-type) interacting particle systems, motivated by asymptotic phenomena and rich underlying algebraic and combinatorial structures (such as nonsymmetric Macdonald polynomials). In this paper, we present a unified approach to constructing stationary measures for most of the known colored particle systems on the ring and the line, including (1) the Asymmetric Simple Exclusion Process (multispecies ASEP, or mASEP); (2) the q-deformed Totally Asymmetric Zero Range Process (TAZRP) also known as the q-Boson particle system; (3) the q-deformed Pushing Totally Asymmetric Simple Exclusion Process (q-PushTASEP). Our method is based on integrable stochastic vertex models and the Yang-Baxter equation. We express the stationary measures as partition functions of new "queue vertex models" on the cylinder. The stationarity property is a direct consequence of the Yang-Baxter equation. For the mASEP on the ring, a particular case of our vertex model is equivalent to the multiline queues of Martin (arXiv:1810.10650). For the colored q-Boson process and the q-PushTASEP on the ring, we recover and generalize known stationary measures constructed using multiline queues or other methods by Ayyer-Mandelshtam-Martin (arXiv:2011.06117, arXiv:2209.09859), and Bukh-Cox (arXiv:1912.03510). Our proofs of stationarity use the Yang-Baxter equation and bypass the Matrix Product Ansatz used for the mASEP by Prolhac-Evans-Mallick (arXiv:0812.3293). On the line and in a quadrant, we use the Yang-Baxter equation to establish a general colored Burke's theorem, which implies that suitable specializations of our queue vertex models produce stationary measures for particle systems on the line. We also compute the colored particle currents in stationarity.more » « less
An official website of the United States government

