More Like this

1. Longitudinal Deep Kernel Gaussian Process Regression

   Liang, Junjie; Wu, Yanting; Xu, Dongxuan; Honavar, Vasant (January 2021, Proceedings of the AAAI Conference on Artificial Intelligence)

   null (Ed.)

   Gaussian processes offer an attractive framework for predictive modeling from longitudinal data, i.e., irregularly sampled, sparse observations from a set of individuals over time. However, such methods have two key shortcomings: (i) They rely on ad hoc heuristics or expensive trial and error to choose the effective kernels, and (ii) They fail to handle multilevel correlation structure in the data. We introduce Longitudinal deep kernel Gaussian process regression (L-DKGPR) to overcome these limitations by fully automating the discovery of complex multilevel correlation structure from longitudinal data. Specifically, L-DKGPR eliminates the need for ad hoc heuristics or trial and error using a novel adaptation of deep kernel learning that combines the expressive power of deep neural networks with the flexibility of non-parametric kernel methods. L-DKGPR effectively learns the multilevel correlation with a novel additive kernel that simultaneously accommodates both time-varying and the time-invariant effects. We derive an efficient algorithm to train L-DKGPR using latent space inducing points and variational inference. Results of extensive experiments on several benchmark data sets demonstrate that L-DKGPR significantly outperforms the state-of-the-art longitudinal data analysis (LDA) methods. 

   more » « less
2. Conditional Deep Gaussian Processes: Empirical Bayes Hyperdata Learning

   https://doi.org/10.3390/e23111387  

   Lu, Chi-Ken; Shafto, Patrick (November 2021, Entropy)

   null (Ed.)

   It is desirable to combine the expressive power of deep learning with Gaussian Process (GP) in one expressive Bayesian learning model. Deep kernel learning showed success as a deep network used for feature extraction. Then, a GP was used as the function model. Recently, it was suggested that, albeit training with marginal likelihood, the deterministic nature of a feature extractor might lead to overfitting, and replacement with a Bayesian network seemed to cure it. Here, we propose the conditional deep Gaussian process (DGP) in which the intermediate GPs in hierarchical composition are supported by the hyperdata and the exposed GP remains zero mean. Motivated by the inducing points in sparse GP, the hyperdata also play the role of function supports, but are hyperparameters rather than random variables. It follows our previous moment matching approach to approximate the marginal prior for conditional DGP with a GP carrying an effective kernel. Thus, as in empirical Bayes, the hyperdata are learned by optimizing the approximate marginal likelihood which implicitly depends on the hyperdata via the kernel. We show the equivalence with the deep kernel learning in the limit of dense hyperdata in latent space. However, the conditional DGP and the corresponding approximate inference enjoy the benefit of being more Bayesian than deep kernel learning. Preliminary extrapolation results demonstrate expressive power from the depth of hierarchy by exploiting the exact covariance and hyperdata learning, in comparison with GP kernel composition, DGP variational inference and deep kernel learning. We also address the non-Gaussian aspect of our model as well as way of upgrading to a full Bayes inference. 

   more » « less
3. RoeNet: Predicting discontinuity of hyperbolic systems from continuous data

   https://doi.org/10.1002/nme.7406  

   Tong, Yunjin; Xiong, Shiying; He, Xingzhe; Yang, Shuqi; Wang, Zhecheng; Tao, Rui; Liu, Runze; Zhu, Bo (March 2024, International Journal for Numerical Methods in Engineering)

   Abstract Predicting future discontinuous phenomena that are unobservable from training data sets has long been a challenging problem in scientific machine learning. We introduce a novel paradigm to predict the emergence and evolution of various discontinuities of hyperbolic partial differential equations (PDEs) based on given training data over a short window with limited discontinuity information. Our method is inspired by the classical Roe solver [P. L. Roe, J Comput Phys., vol. 43, 1981], a basic tool for simulating various hyperbolic PDEs in computational physics. By carefully designing the computing primitives, the data flow, and the novel pseudoinverse processing module, we enable our data‐driven predictor to satisfy all the essential mathematical criteria of a Roe solver and hence deliver accurate predictions of hyperbolic PDEs. We demonstrate through various examples that our data‐driven Roe predictor outperforms original human‐designed Roe solver and deep neural networks with weak priors in terms of accuracy and robustness. 

   more » « less
4. A Kernel Multiple Change-point Algorithm via Model Selection

   Arlot, Sylvain; Celisse, Alain; Harchaoui, Zaid (January 2019, Journal of machine learning research)

   We consider a general formulation of the multiple change-point problem, in which the data is assumed to belong to a set equipped with a positive semidefinite kernel. We propose a model-selection penalty allowing to select the number of change points in Harchaoui and Cappe's kernel-based change-point detection method. The model-selection penalty generalizes non-asymptotic model-selection penalties for the change-in-mean problem with univariate data. We prove a non-asymptotic oracle inequality for the resulting kernel-based change-point detection method, whatever the unknown number of change points, thanks to a concentration result for Hilbert-space valued random variables which may be of independent interest. Experiments on synthetic and real data illustrate the proposed method, demonstrating its ability to detect subtle changes in the distribution of data. 

   more » « less
5. Correlation-aware Online Change Point Detection

   Deng, Chengyuan; Chen, Zhengzhangn; Zhao, Xujiang; Wang, Haoyu; Wang, Junxiang; Chen, Haifeng; Gao, Jie (November 2025, Proceedings of the 34th ACM Conference on Information and Knowledge Management (CIKM’25))

   Change point detection aims to identify abrupt shifts occurring at multiple points within a data sequence. This task becomes particularly challenging in the online setting, where different types of changes can occur, including shifts in both the marginal and joint distributions of the data. In this paper, we address these challenges by tracking the Riemannian geometry of correlation matrices, allowing Riemannian metrics to compute the geodesic distance as an accurate measure of correlation dynamics. We introduce Rio-CPD, a non-parametric, correlation-aware online change point detection framework that integrates the Riemannian geometry of the manifold of symmetric positive definite matrices with the cumulative sum (CUSUM) statistic for detecting change points. Rio-CPD employs a novel CUSUM design by computing the geodesic distance between current observations and the Fréchet mean of prior observations. With appropriate choices of Riemannian metrics, Rio-CPD offers a simple yet effective and computationally efficient algorithm. Experimental results on both synthetic and real-world datasets demonstrate that Rio-CPD outperforms existing methods on detection accuracy, average detection delay and efficiency. 

   more » « less