More Like this

1. Inaccuracy of the variance evolution associated with discrete covariance propagation

   https://doi.org/10.1002/qj.5016  

   Gilpin, Shay; Matsuo, Tomoko; Cohn, Stephen (June 2025, Quarterly Journal of the Royal Meteorological Society)

   Abstract Data assimilation methods often also employ the same discrete dynamical model used to evolve the state estimate in time to propagate an approximation of the state estimation error covariance matrix. Four‐dimensional variational methods, for instance, evolve the covariance matrix implicitly via discrete tangent linear dynamics. Ensemble methods, while not forming this matrix explicitly, approximate its evolution at low rank from the evolution of the ensemble members. Such approximate evolution schemes for the covariance matrix imply an approximate evolution of the estimation error variances along its diagonal. For states that satisfy the advection equation, the continuity equation, or related hyperbolic partial differential equations (PDEs), the estimation error variance itself satisfies a known PDE, so the accuracy of the various approximations to the variances implied by the discrete covariance propagation can be determined directly. Experiments conducted by the atmospheric chemical constituent data assimilation community have indicated that such approximate variance evolution can be highly inaccurate. Through careful analysis and simple numerical experiments, we show that such poor accuracy must be expected, due to the inherent nature of discrete covariance propagation, coupled with a special property of the continuum covariance dynamics for states governed by these types of hyperbolic PDE. The intuitive explanation for this inaccuracy is that discrete covariance propagation involves approximating diagonal elements of the covariance matrix with combinations of off‐diagonal elements, even when there is a discontinuity in the continuum covariance dynamics along the diagonal. Our analysis uncovers the resulting error terms that depend on the ratio of the grid spacing to the correlation length, and these terms become very large when correlation lengths begin to approach the grid scale, for instance, as gradients steepen near the diagonal. We show that inaccurate variance evolution can manifest itself as both spurious loss and gain of variance. 

   more » « less
2. GOES-R land surface products at Western Hemisphere eddy covariance tower locations

   https://doi.org/10.1038/s41597-024-03071-z  

   Losos, Danielle; Hoffman, Sophie; Stoy, Paul C. (March 2024, Scientific Data)

   Abstract The terrestrial carbon cycle varies dynamically on hourly to weekly scales, making it difficult to observe. Geostationary (“weather”) satellites like the Geostationary Environmental Operational Satellite - R Series (GOES-R) deliver near-hemispheric imagery at a ten-minute cadence. The Advanced Baseline Imager (ABI) aboard GOES-R measures visible and near-infrared spectral bands that can be used to estimate land surface properties and carbon dioxide flux. However, GOES-R data are designed for real-time dissemination and are difficult to link with eddy covariance time series of land-atmosphere carbon dioxide exchange. We compiled three-year time series of GOES-R land surface attributes including visible and near-infrared reflectances, land surface temperature (LST), and downwelling shortwave radiation (DSR) at 314 ABI fixed grid pixels containing eddy covariance towers. We demonstrate how to best combine satellite andin-situdatasets and show how ABI attributes useful for ecosystem monitoring vary across space and time. By connecting observation networks that infer rapid changes to the carbon cycle, we can gain a richer understanding of the processes that control it. 

   more » « less
3. Covariance loss, Szemeredi regularity, and differential privacy

   https://doi.org/10.1090/proc/17126  

   Boedihardjo, March; Strohmer, Thomas; Vershynin, Roman (February 2025, Proceedings of the American Mathematical Society)

   We show how randomized rounding based on Grothendieck’s identity can be used to prove a nearly tight bound on the covariance loss–the amount of covariance that is lost by taking conditional expectation. This result yields a new type of weak Szemeredi regularity lemma for positive semidefinite matrices and kernels. Moreover, it can be used to construct differentially private synthetic data. 

   more » « less
4. Estimation of the covariance structure of heavy-tailed distributions

   Minsker, Stanislav; Wei, Xiaohan (December 2017, NIPS)

   We propose and analyze a new estimator of the covariance matrix that admits strong theoretical guarantees under weak assumptions on the underlying distribution, such as existence of moments of only low order. While estimation of covariance matrices corresponding to sub-Gaussian distributions is well-understood, much less in known in the case of heavy-tailed data. As K. Balasubramanian and M. Yuan write, "data from real-world experiments oftentimes tend to be corrupted with outliers and/or exhibit heavy tails. In such cases, it is not clear that those covariance matrix estimators .. remain optimal" and "what are the other possible strategies to deal with heavy tailed distributions warrant further studies." We make a step towards answering this question and prove tight deviation inequalities for the proposed estimator that depend only on the parameters controlling the intrinsic dimension'' associated to the covariance matrix (as opposed to the dimension of the ambient space); in particular, our results are applicable in the case of high-dimensional observations. 

   more » « less
5. GridNet: Fast date-driven EM-induced IR drop prediction and localized fixing for on-chip power grid networks

   Zhou, H; Jin, W.; Tan, S. X.-D. (October 2020, Proc. IEEE/ACM International Conf. on Computer-Aided Design (ICCAD’20),)

   Electromigration (EM) is a major failure effect for on-chip power grid networks of deep submicron VLSI circuits. EM degradation of metal grid lines can lead to excessive voltage drops (IR drops) before the target lifetime. In this paper, we propose a fast data-driven EM-induced IR drop analysis framework for power grid networks, named {\it GridNet}, based on the conditional generative adversarial networks (CGAN). It aims to accelerate the incremental full-chip EM-induced IR drop analysis, as well as IR drop violation fixing during the power grid design and optimization. More importantly, {\it GridNet} can naturally leverage the differentiable feature of deep neural networks (DNN) to {\it obtain the sensitivity information of node voltage with respect to the wire resistance (or width) with marginal cost}. {\it GridNet} treats continuous time and the given electrical features as input conditions, and the EM-induced time-varying voltage of power grid networks as the conditional outputs, which are represented as data series images. We show that {\it GridNet} is able to learn the temporal dynamics of the aging process in continuous time domain. Besides, we can take advantage of the sensitivity information provided by {\it GridNet} to perform efficient localized IR drop violation fixing in the late stage design and optimization. Numerical results on 36000 synthesized power grid network samples demonstrate that the new method can lead to $$10^5\times$$ speedup over the recently proposed full-chip coupled EM and IR drop analysis tool. We further show that localized IR drop violation fix for the same set of power grid networks can be performed remarkably efficiently using the cheap sensitivity computation from {\it GridNet}. 

   more » « less