An official website of the United States government
We study first passage percolation (FPP) with stationary edge weights on Cayley graphs of finitely generated virtually nilpotent groups. Previous works of Benjamini-Tessera and Cantrell-Furman show that scaling limits of such FPP are given by Carnot-Carathéodory metrics on the associated graded nilpotent Lie group. We show a converse, i.e. that for any Cayley graph of a finitely generated nilpotent group, any Carnot-Carathéodory metric on the associated graded nilpotent Lie group is the scaling limit of some FPP with stationary edge weights on that graph. Moreover, for any Cayley graph of any finitely generated virtually nilpotent group, any conjugation-invariant metric is the scaling limit of some FPP with stationary edge weights on that graph. We also show that the conjugation-invariant condition is also a necessary condition in all cases where scaling limits are known to exist.
more »
« less
On the approximation of derivative values using a WENO algorithm with progressive order of accuracy close to discontinuities
Abstract In this article, we introduce a new WENO algorithm that aims to calculate an approximation to derivative values of a function in a non-regular grid. We adapt the ideas presented in [Amat et al., SIAM J. Numer. Anal. (2020)] to design the nonlinear weights in a manner such that the order of accuracy is maximum in the intervals close to the discontinuities. Some proofs, remarks on the choice of the stencils and explicit formulas for the weights and smoothness indicators are given. We also present some numerical experiments to confirm the theoretical results.
more »
« less
- Award ID(s):
- 2010107
- PAR ID:
- 10420239
- Date Published:
- Journal Name:
- Computational and Applied Mathematics
- Volume:
- 41
- Issue:
- 6
- ISSN:
- 2238-3603
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Why do brains have inhibitory connections? Why do deep networks have negative weights? We propose an answer from the perspective of representation capacity. We believe representing functions is the primary role of both (i) the brain in natural intelligence, and (ii) deep networks in artificial intelligence. Our answer to why there are inhibitory/negative weights is: to learn more functions. We prove that, in the absence of negative weights, neural networks with non-decreasing activation functions are not universal approximators. While this may be an intuitive result to some, to the best of our knowledge, there is no formal theory, in either machine learning or neuroscience, that demonstrates why negative weights are crucial in the context of representation capacity. Further, we provide insights on the geometric properties of the representation space that non-negative deep networks cannot represent. We expect these insights will yield a deeper understanding of more sophisticated inductive priors imposed on the distribution of weights that lead to more efficient biological and machine learning.more » « less
-
null (Ed.)We investigate the extent to which individual attention heads in pretrained transformer language models, such as BERT and RoBERTa, implicitly capture syntactic dependency relations. We employ two methods---taking the maximum attention weight and computing the maximum spanning tree---to extract implicit dependency relations from the attention weights of each layer/head, and compare them to the ground-truth Universal Dependency (UD) trees. We show that, for some UD relation types, there exist heads that can recover the dependency type significantly better than baselines on parsed English text, suggesting that some self-attention heads act as a proxy for syntactic structure. We also analyze BERT fine-tuned on two datasets---the syntax-oriented CoLA and the semantics-oriented MNLI---to investigate whether fine-tuning affects the patterns of their self-attention, but we do not observe substantial differences in the overall dependency relations extracted using our methods. Our results suggest that these models have some specialist attention heads that track individual dependency types, but no generalist head that performs holistic parsing significantly better than a trivial baseline, and that analyzing attention weights directly may not reveal much of the syntactic knowledge that BERT-style models are known to learn.more » « less
-
Abstract We provide a novel characterization of augmented balancing weights, also known as automatic debiased machine learning. These popular doubly robust estimators combine outcome modelling with balancing weights—weights that achieve covariate balance directly instead of estimating and inverting the propensity score. When the outcome and weighting models are both linear in some (possibly infinite) basis, we show that the augmented estimator is equivalent to a single linear model with coefficients that combine those of the original outcome model with those from unpenalized ordinary least-squares (OLS). Under certain choices of regularization parameters, the augmented estimator in fact collapses to the OLS estimator alone. We then extend these results to specific outcome and weighting models. We first show that the augmented estimator that uses (kernel) ridge regression for both outcome and weighting models is equivalent to a single, undersmoothed (kernel) ridge regression—implying a novel analysis of undersmoothing. When the weighting model is instead lasso-penalized, we demonstrate a familiar ‘double selection’ property. Our framework opens the black box on this increasingly popular class of estimators, bridges the gap between existing results on the semiparametric efficiency of undersmoothed and doubly robust estimators, and provides new insights into the performance of augmented balancing weights.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1007/s40314-022-02004-z
