skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, June 12 until 2:00 AM ET on Friday, June 13 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information
This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.  more » « less
Award ID(s):
1750362
PAR ID:
10221391
Author(s) / Creator(s):
Date Published:
Journal Name:
Entropy
Volume:
22
Issue:
11
ISSN:
1099-4300
Page Range / eLocation ID:
1244
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, PLDI '21: 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation)
    null (Ed.)
    For probabilistic programs, it is usually not possible to automatically derive exact information about their properties, such as the distribution of states at a given program point. Instead, one can attempt to derive approximations, such as upper bounds on tail probabilities. Such bounds can be obtained via concentration inequalities, which rely on the moments of a distribution, such as the expectation (the first raw moment) or the variance (the second central moment). Tail bounds obtained using central moments are often tighter than the ones obtained using raw moments, but automatically analyzing central moments is more challenging. This paper presents an analysis for probabilistic programs that automatically derives symbolic upper and lower bounds on variances, as well as higher central moments, of cost accumulators. To overcome the challenges of higher-moment analysis, it generalizes analyses for expectations with an algebraic abstraction that simultaneously analyzes different moments, utilizing relations between them. A key innovation is the notion of moment-polymorphic recursion, and a practical derivation system that handles recursive functions. The analysis has been implemented using a template-based technique that reduces the inference of polynomial bounds to linear programming. Experiments with our prototype central-moment analyzer show that, despite the analyzer’s upper/lower bounds on various quantities, it obtains tighter tail bounds than an existing system that uses only raw moments, such as expectations. 
    more » « less
  2. ; (, Computers and Geotechnics)
    The subject of the influence of the seismic excitation on limit loads of footings is revisited, with emphasis on the moment load. The kinematic approach of limit analysis is employed using two collapse mechanisms allowing footing rotation and one with pure translational kinematics. Two of the mechanisms have novel elements, not presented in earlier literature. The paper is focused on the resistance of the soil weight to activating a mechanism of failure, which can be best cast in terms of the seismic bearing capacity factor Nsγ . Seismic loads from the superstructure are interpreted as those caused by a three-mass model, each mass with its own seismic coefficient. The notion of generalized loads is used to present the yield locus for the footing in terms of the gravity force, horizontal force, and moment. The non-symmetric components of the load are interpreted as seismically activated. The approach yields a strict upper bound to the magnitude of the load vector causing failure. Of the three failure mechanisms considered none yields the best (least) solutions for all combinations of loads. In general, the two mechanisms with footing rotation perform better for large moments, whereas the translational mechanism yields better results when moments are small. However, even in the absence of a moment load, the rotational mechanism can yield better estimates of the limit load when the seismic coefficient is relatively large. 
    more » « less
  3. ; (, Random Structures & Algorithms)
    Abstract We study the universality of superconcentration for the free energy in the Sherrington–Kirkpatrick model. In [10], Chatterjee showed that when the system consists of spins and Gaussian disorders, the variance of this quantity is superconcentrated by establishing an upper bound of order , in contrast to the bound obtained from the Gaussian–Poincaré inequality. In this paper, we show that superconcentration indeed holds for any choice of centered disorders with finite third moment, where the upper bound is expressed in terms of an auxiliary nondecreasing function that arises in the representation of the disorder as for standard normal. Under an additional regularity assumption on , we further show that the variance is of order at most . 
    more » « less
  4. ; ; ; ; (, Journal of physics Conference series)
    We develop implicit-explicit (IMEX) schemes for neutrino transport in a background material in the context of a two-moment model that evolves the angular moments of a neutrino phase-space distribution function. Considering the upper and lower bounds that are introduced by Pauli’s exclusion principle on the moments, an algebraic moment closure based on Fermi-Dirac statistics and a convex-invariant time integrator both are demanded. A finite-volume/first-order discontinuous Galerkin(DG) method is used to illustrate how an algebraic moment closure based on Fermi-Dirac statistics is needed to satisfy the bounds. Several algebraic closures are compared with these bounds in mind, and the Cernohorsky and Bludman closure, which satisfies the bounds, is chosen for our IMEX schemes. For the convex-invariant time integrator, two IMEX schemes named PD-ARS have been proposed. PD-ARS denotes a convex-invariant IMEX Runge-Kutta scheme that is high-order accurate in the streaming limit, and works well in the diffusion limit. Our two PD-ARS schemes use second- and third-order, explicit, strong-stability-preserving Runge-Kutta methods as their explicit part, respectively, and therefore are second- and third-order accurate in the streaming limit, respectively. The accuracy and convex-invariance of our PD-ARS schemes are demonstrated in the numerical tests with a third-order DG method for spatial discretization and a simple Lax-Friedrichs flux. The method has been implemented in our high-order neutrino-radiation hydrodynamics (thornado) toolkit. We show preliminary results employing tabulated neutrino opacities. 
    more » « less
  5. (, IEEE transactions on information theory)
    Upper bounds on absolute moments of the scores are derived for sums of independent random variables in terms of the moments of the scores, as well as in terms of the total variation norm of densities of summands. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email