More Like this

1. Radar Clutter Covariance Estimation: A Nonlinear Spectral Shrinkage Approach

   https://doi.org/10.1109/TAES.2023.3292069  

   Jain, Shashwat; Krishnamurthy, Vikram; Rangaswamy, Muralidhar; Kang, Bosung; Gogineni, Sandeep (December 2023, IEEE Transactions on Aerospace and Electronic Systems)

   In this article, we exploit the spiked covariance structure of the clutter plus noise covariance matrix for radar signal processing. Using state-of-the-art techniques high dimensional statistics, we propose a nonlinear shrinkage-based rotation invariant spiked covariance ma- trix estimator. We state the convergence of the estimated spiked eigen- values. We use a dataset generated from the high-fidelity, site-specific physics-based radar simulation software RFView to compare the proposed algorithm against the existing rank constrained maximum likelihood (RCML)-expected likelihood (EL) covariance estimation 

   more » « less
2. Estimation of the covariance structure of heavy-tailed distributions

   Minsker, Stanislav; Wei, Xiaohan (December 2017, NIPS)

   We propose and analyze a new estimator of the covariance matrix that admits strong theoretical guarantees under weak assumptions on the underlying distribution, such as existence of moments of only low order. While estimation of covariance matrices corresponding to sub-Gaussian distributions is well-understood, much less in known in the case of heavy-tailed data. As K. Balasubramanian and M. Yuan write, "data from real-world experiments oftentimes tend to be corrupted with outliers and/or exhibit heavy tails. In such cases, it is not clear that those covariance matrix estimators .. remain optimal" and "what are the other possible strategies to deal with heavy tailed distributions warrant further studies." We make a step towards answering this question and prove tight deviation inequalities for the proposed estimator that depend only on the parameters controlling the intrinsic dimension'' associated to the covariance matrix (as opposed to the dimension of the ambient space); in particular, our results are applicable in the case of high-dimensional observations. 

   more » « less
3. Multi-response Regression for Block-missing Multi-modal Data without Imputation

   https://doi.org/10.5705/ss.202021.0170  

   Wang, Haodong; Li, Quefeng; Liu, Yufeng (January 2024, Statistica Sinica)

   Multi-modal data are prevalent in many scientific fields. In this study, we consider the parameter estimation and variable selection for a multi-response regression using block-missing multi-modal data. Our method allows the dimensions of both the responses and the predictors to be large, and the responses to be incomplete and correlated, a common practical problem in high-dimensional settings. Our proposed method uses two steps to make a prediction from a multi-response linear regression model with block-missing multi-modal predictors. In the first step, without imputing missing data, we use all available data to estimate the covariance matrix of the predictors and the cross-covariance matrix between the predictors and the responses. In the second step, we use these matrices and a penalized method to simultaneously estimate the precision matrix of the response vector, given the predictors, and the sparse regression parameter matrix. Lastly, we demonstrate the effectiveness of the proposed method using theoretical studies, simulated examples, and an analysis of a multi-modal imaging data set from the Alzheimer’s Disease Neuroimaging Initiative. 

   more » « less
4. Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix

   https://doi.org/10.3934/fods.2021030  

   Turčičová, Marie; Mandel, Jan; Eben, Kryštof (January 2021, Foundations of Data Science)

   We present an ensemble filtering method based on a linear model for the precision matrix (the inverse of the covariance) with the parameters determined by Score Matching Estimation. The method provides a rigorous covariance regularization when the underlying random field is Gaussian Markov. The parameters are found by solving a system of linear equations. The analysis step uses the inverse formulation of the Kalman update. Several filter versions, differing in the construction of the analysis ensemble, are proposed, as well as a Score matching version of the Extended Kalman Filter. 

   more » « less
5. Nonlinear spiked covariance matrices and signal propagation in deep neural networks

   Wang, Zhichao; Wu, Denny; Fan, Zhou (June 2024, Proceedings of Machine Learning Research)

   Many recent works have studied the eigenvalue spectrum of the Conjugate Kernel (CK) defined by the nonlinear feature map of a feedforward neural network. However, existing results only establish weak convergence of the empirical eigenvalue distribution, and fall short of providing precise quantitative characterizations of the “spike” eigenvalues and eigenvectors that often capture the low-dimensional signal structure of the learning problem. In this work, we characterize these signal eigenvalues and eigenvectors for a nonlinear version of the spiked covariance model, including the CK as a special case. Using this general result, we give a quantitative description of how spiked eigenstructure in the input data propagates through the hidden layers of a neural network with random weights. As a second application, we study a simple regime of representation learning where the weight matrix develops a rank-one signal component over training and characterize the alignment of the target function with the spike eigenvector of the CK on test data. 

   more » « less