PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation
- Award ID(s):
- 2047040
- NSF-PAR ID:
- 10403078
- Date Published:
- Journal Name:
- Proceedings of the 39th International Conference on Machine Learning
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
Gradient sampling (GS) methods for the minimization of objective functions that may be nonconvex and/or nonsmooth are proposed, analyzed, and tested. One of the most computationally expensive components of contemporary GS methods is the need to solve a convex quadratic subproblem in each iteration. By contrast, the methods proposed in this paper allow the use of inexact solutions of these subproblems, which, as proved in the paper, can be incorporated without the loss of theoretical convergence guarantees. Numerical experiments show that, by exploiting inexact subproblem solutions, one can consistently reduce the computational effort required by a GS method. Additionally, a strategy is proposed for aggregating gradient information after a subproblem is solved (potentially inexactly) as has been exploited in bundle methods for nonsmooth optimization. It is proved that the aggregation scheme can be introduced without the loss of theoretical convergence guarantees. Numerical experiments show that incorporating this gradient aggregation approach can also reduce the computational effort required by a GS method.more » « less
-
Abstract Future projections of the poleward eddy heat flux by the atmosphere are often regarded as being uncertain because of the competing effect between surface and upper-tropospheric meridional temperature gradients. Previous idealized modeling studies showed that eddy heat flux response is more sensitive to the variability of lower-tropospheric temperature gradient. However, observational evidence is lacking. In this study, observational data analyses are performed to examine the relationships between eddy heat fluxes and temperature gradients during boreal winter by constructing daily indices. On the intraseasonal time scale, the surface temperature gradient is found to be more effective at regulating the synoptic-scale eddy heat flux (SF) than is the upper-tropospheric temperature gradient. Enhancements in surface temperature gradient, however, are subject to an inactive planetary-scale eddy heat flux (PF). The PF in turn is dependent on the zonal gradient in tropical convective heating. Consistent with these interactions, over the past 40 winters, the zonal gradient in tropical heating and PF have been trending upward, while the surface temperature gradient and SF have been trending downward. These results indicate that for a better understanding of eddy heat fluxes, attention should be given to zonal convective heating gradients in the tropics as much as to meridional temperature gradients.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
