More Like this

1. Rich representations and superrigidity

   https://doi.org/10.1017/etds.2024.136  

   BALDI, GREGORIO; MILLER, NICHOLAS; STOVER, MATTHEW; ULLMO, EMMANUEL (August 2025, Ergodic Theory and Dynamical Systems)

   Abstract We investigate and compare applications of the Zilber–Pink conjecture and dynamical methods to rigidity problems for arithmetic real and complex hyperbolic lattices. Along the way, we obtain new general results about reconstructing a variation of Hodge structure from its typical Hodge locus that may be of independent interest. Applications to Siu’s immersion problem are also discussed, the most general of which only requires the hypothesis that infinitely many closed geodesics map to proper totally geodesic subvarieties under the immersion. 

   more » « less
2. Kernel and Rich Regimes in Overparametrized Models

   Woodworth, B; Gunasekar, S; Lee, J; Moroshko, E; Savarese, P; Golan, I; Soudry, D; Srebro, N (October 2020, Conference on Learning Theory)

   null (Ed.)
3. Kernel and Rich Regimes in Overparametrized Models

   Woodworth, Blake; Gunasekar, Suriya; Lee, Jason D.; Moroshko, Edward; Savarese, Pedro; Golan, Itay; Soudry, Daniel; Srebro, Nati (January 2020, COLT 2020)
4. Kernel and Rich Regimes in Overparametrized Models

   Woodworth, Blake; Gunasekar, Suriya; Lee, Jason D; Moroshko, Edward; Savarese, Pedro; Golan, Itay; Soudry, Daniel; Srebro, Nathan (February 2020, Conference on Learning Theory (COLT))

   A recent line of work studies overparametrized neural networks in the “kernel regime,” i.e. when during training the network behaves as a kernelized linear predictor, and thus, training with gradient descent has the effect of finding the corresponding minimum RKHS norm solution. This stands in contrast to other studies which demonstrate how gradient descent on overparametrized networks can induce rich implicit biases that are not RKHS norms. Building on an observation by \citet{chizat2018note}, we show how the \textbf{\textit{scale of the initialization}} controls the transition between the “kernel” (aka lazy) and “rich” (aka active) regimes and affects generalization properties in multilayer homogeneous models. We provide a complete and detailed analysis for a family of simple depth-D linear networks that exhibit an interesting and meaningful transition between the kernel and rich regimes, and highlight an interesting role for the \emph{width} of the models. We further demonstrate this transition empirically for matrix factorization and multilayer non-linear networks. 

   more » « less
5. Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks

   https://doi.org/10.1145/3442381.3450114  

   Zhang, Xinyang; Zhang, Chenwei; Dong, Xin Luna; Shang, Jingbo; Han, Jiawei (April 2021, WWW '21: The Web Conference 2021)

   null (Ed.)

   Text categorization is an essential task in Web content analysis. Considering the ever-evolving Web data and new emerging categories, instead of the laborious supervised setting, in this paper, we focus on the minimally-supervised setting that aims to categorize documents effectively, with a couple of seed documents annotated per category. We recognize that texts collected from the Web are often structure-rich, i.e., accompanied by various metadata. One can easily organize the corpus into a text-rich network, joining raw text documents with document attributes, high-quality phrases, label surface names as nodes, and their associations as edges. Such a network provides a holistic view of the corpus’ heterogeneous data sources and enables a joint optimization for network-based analysis and deep textual model training. We therefore propose a novel framework for minimally supervised categorization by learning from the text-rich network. Specifically, we jointly train two modules with different inductive biases – a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning. Each module generates pseudo training labels from the unlabeled document set, and both modules mutually enhance each other by co-training using pooled pseudo labels. We test our model on two real-world datasets. On the challenging e-commerce product categorization dataset with 683 categories, our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92%, significantly outperforming all compared methods; our accuracy is only less than 2% away from the supervised BERT model trained on about 50K labeled documents. 

   more » « less