More Like this

1. Should We Learn Most Likely Functions or Parameters?

   Qiu, S; Rudner, Tim G; Kapoor, S; Wilson, Andrew G (December 2023, Advances in Neural Information Processing Systems)

   Standard regularized training procedures correspond to maximizing a posterior distribution over parameters, known as maximum a posteriori (MAP) estimation. However, model parameters are of interest only insomuch as they combine with the functional form of a model to provide a function that can make good predictions. Moreover, the most likely parameters under the parameter posterior do not generally correspond to the most likely function induced by the parameter posterior. In fact, we can re-parametrize a model such that any setting of parameters can maximize the parameter posterior. As an alternative, we investigate the benefits and drawbacks of directly estimating the most likely function implied by the model and the data. We show that this procedure leads to pathological solutions when using neural networks and prove conditions under which the procedure is well-behaved, as well as a scalable approximation. Under these conditions, we find that function-space MAP estimation can lead to flatter minima, better generalization, and improved robustness to overfitting. 

   more » « less
2. Should We Learn Most Likely Functions or Parameters?

   Qiu, S; Rudner, T; Kapoor, S; Wilson, AG (December 2023, Advances in Neural Information Processing Systems)

   Standard regularized training procedures correspond to maximizing a posterior distribution over parameters, known as maximum a posteriori (MAP) estimation. However, model parameters are of interest only insomuch as they combine with the functional form of a model to provide a function that can make good predictions. Moreover, the most likely parameters under the parameter posterior do not generally correspond to the most likely function induced by the parameter posterior. In fact, we can re-parametrize a model such that any setting of parameters can maximize the parameter posterior. As an alternative, we investigate the benefits and drawbacks of directly estimating the most likely function implied by the model and the data. We show that this procedure leads to pathological solutions when using neural networks and prove conditions under which the procedure is well-behaved, as well as a scalable approximation. Under these conditions, we find that function-space MAP estimation can lead to flatter minima, better generalization, and improved robustness to overfitting 

   more » « less
3. The small-mass limit for some constrained wave equations with nonlinear conservative noise

   https://doi.org/10.1214/25-EJP1284  

   Cerrai, Sandra; Xie, Mengzi (January 2025, Electronic Journal of Probability)
4. Projecting the Most Likely Annual Urban Heat Extremes in the Central United States

   https://doi.org/10.3390/atmos10120727  

   Jahn, David E.; Gallus, William A.; Nguyen, Phong T.; Pan, Qiyun; Cetin, Kristen; Byon, Eunshin; Manuel, Lance; Zhou, Yuyu; Jahani, Elham (December 2019, Atmosphere)

   Climate studies based on global climate models (GCMs) project a steady increase in annual average temperature and severe heat extremes in central North America during the mid-century and beyond. However, the agreement of observed trends with climate model trends varies substantially across the region. The present study focuses on two different locations: Des Moines, IA and Austin, TX. In Des Moines, annual extreme temperatures have not increased over the past three decades unlike the trend of regionally-downscaled GCM data for the Midwest, likely due to a “warming hole” over the area linked to agricultural factors. This warming hole effect is not evident for Austin over the same time period, where extreme temperatures have been higher than projected by regionally-downscaled climate (RDC) forecasts. In consideration of the deviation of such RDC extreme temperature forecasts from observations, this study statistically analyzes RDC data in conjunction with observational data to define for these two cities a 95% prediction interval of heat extreme values by 2040. The statistical model is constructed using a linear combination of RDC ensemble-member annual extreme temperature forecasts with regression coefficients for individual forecasts estimated by optimizing model results against observations over a 52-year training period. 

   more » « less
5. KPZ equation with a small noise, deep upper tail and limit shape

   https://doi.org/10.1007/s00440-022-01185-2  

   Gaudreau_Lamarre, Pierre Yves; Lin, Yier; Tsai, Li-Cheng (April 2023, Probability Theory and Related Fields)

   In this paper, we consider the KPZ equation under the weak noise scaling. That is, we introduce a small parameter \sqrt{\varepsilon} in front of the noise and let \varepsilon \to 0. We prove that the one-point large deviation rate function has a \frac{3}{2} power law in the deep upper tail. Furthermore, by forcing the value of the KPZ equation at a point to be very large, we prove a limit shape of the KPZ equation as \varepsilon \to 0. This confirms the physics prediction in Kolokolov and Korshunov (2007), Kolokolov and Korshunov (2009), Meerson, Katzav, and Vilenkin (2016), Kamenev, Meerson, and Sasorov (2016), and Le Doussal, Majumdar, Rosso, and Schehr (2016). 

   more » « less