Search for: All records

We study how neural networks trained by gradient descent extrapolate, i.e., what they learn outside the support of the training distribution. Previous works report mixed empirical results when extrapolating with neural networks: while feedforward neural networks, a.k.a. multilayer perceptrons (MLPs), do not extrapolate well in certain simple tasks, Graph Neural Networks (GNNs) – structured networks with MLP modules – have shown some success in more complex tasks. Working towards a theoretical explanation, we identify conditions under which MLPs and GNNs extrapolate well. First, we quantify the observation that ReLU MLPs quickly converge to linear functions along any direction from the origin, which implies that ReLU MLPs do not extrapolate most nonlinear functions. But, they can provably learn a linear target function when the training distribution is sufficiently “diverse”. Second, in connection to analyzing the successes and limitations of GNNs, these results suggest a hypothesis for which we provide theoretical and empirical evidence: the success of GNNs in extrapolating algorithmic tasks to new data (e.g., larger graphs or edge weights) relies on encoding task-specific non-linearities in the architecture or features. Our theoretical analysis builds on a connection of over-parameterized networks to the neural tangent kernel. Empirically, our theory holds across different training settings. 

Non-Hermitian optical systems with parity-time (PT) symmetry have recently revealed many intriguing prospects that outperform conservative structures. The previous works are mostly rooted in complex arrangements with controlled gain-loss interplay. Here, we demonstrate anti-PT symmetry inherent in the nonlinear optical interaction based upon forward optical four-wave mixing in a laser-cooled atomic ensemble with negligible linear gain and loss. We observe that the pair of frequency modes undergo a nontrivial anti-PT phase transition between coherent power oscillation and optical parametric amplification in presence of a large phase mismatch. 

The possibility that charged particles are accelerated statistically in a “sea” of small-scale interacting magnetic flux ropes in the supersonic solar wind is gaining credence. In this Letter, we extend the Zank et al. statistical transport theory for a nearly isotopic particle distribution by including an escape term corresponding to particle loss from a finite acceleration region. Steady-state 1D solutions for both the accelerated particle velocity distribution function and differential intensity are derived. We show Ulysses observations of an energetic particle flux enhancement event downstream of a shock near 5 au that is inconsistent with the predictions of classical diffusive shock acceleration (DSA) but may be explained by local acceleration associated with magnetic islands. An automated Grad-Shafranov reconstruction approach is employed to identify small-scale magnetic flux ropes behind the shock. For the first time, the observed energetic particle “time-intensity” profile and spectra are quantitatively compared with theoretical predictions. The results show that stochastic acceleration by interacting magnetic islands accounts successfully for the observed (i) peaking of particle intensities behind the shock instead of at the shock front as standard DSA predicts; (ii) increase in the particle flux amplification factor with increasing particle energy; (ii) increase in distance between the particle intensity peak and the shock front with increasing energy; and (iv) hardening of particle power-law spectra with increasing distance downstream of the shock. 

The possibility that charged particles are accelerated statistically in a “sea” of small-scale interacting magnetic flux ropes in the supersonic solar wind is gaining credence. In this Letter, we extend the Zank et al. statistical transport theory for a nearly isotopic particle distribution by including an escape term corresponding to particle loss from a finite acceleration region. Steady-state 1D solutions for both the accelerated particle velocity distribution function and differential intensity are derived. We show Ulysses observations of an energetic particle flux enhancement event downstream of a shock near 5 au that is inconsistent with the predictions of classical diffusive shock acceleration (DSA) but may be explained by local acceleration associated with magnetic islands. An automated Grad-Shafranov reconstruction approach is employed to identify small-scale magnetic flux ropes behind the shock. For the first time, the observed energetic particle “time-intensity” profile and spectra are quantitatively compared with theoretical predictions. The results show that stochastic acceleration by interacting magnetic islands accounts successfully for the observed (i) peaking of particle intensities behind the shock instead of at the shock front as standard DSA predicts; (ii) increase in the particle flux amplification factor with increasing particle energy; (ii) increase in distance between the particle intensity peak and the shock front with increasing energy; and (iv) hardening of particle power-law spectra with increasing distance downstream of the shock.