NSF PAR Search | NSF Public Access Repository

Neural Scaling Laws From Large-N Field Theory: Solvable Model Beyond the Ridgeless Limit

https://doi.org/10.1088/2632-2153/adc872

Zhang, Zhengkang (April 2025, Machine Learning: Science and Technology)

Abstract Many machine learning models based on neural networks exhibit scaling laws: their performance scales as power laws with respect to the sizes of the model and training data set. We use large-N field theory methods to solve a model recently proposed by Maloney, Roberts and Sully which provides a simplified setting to study neural scaling laws. Our solution extends the result in this latter paper to general nonzero values of the ridge parameter, which are essential to regularize the behavior of the model. In addition to obtaining new and more precise scaling laws, we also uncover a duality transformation at the diagrams level which explains the symmetry between model and training data set sizes. The same duality underlies recent efforts to design neural networks to simulate quantum field theories.

We elaborate on a recently proposed geometric framework for scalar effective field theories. Starting from the action, a metric can be identified that enables the construction of geometric quantities on the associated functional manifold. These objects transform covariantly under general field redefinitions that relate different operator bases, including those involving derivatives. We present a novel geometric formula for the amplitudes of the theory, where the vertices in Feynman diagrams are replaced by their geometrized counterparts. This makes the on-shell covariance of amplitudes manifest, providing the link between functional geometry and effective field theories. Published by the American Physical Society2025

Search for: All records