More Like this

1. Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit

   Zhu, Shengyu; Chen, Biao; Yang, Pengfei; Chen, Zhitang (April 2019, International Conference on Artificial Intelligence and Statistics)

   We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We show that two classes of Maximum Mean Discrepancy (MMD) based tests attain this optimality on Rd, while the quadratictime Kernel Stein Discrepancy (KSD) based tests achieve the maximum exponential decay rate under a relaxed level constraint. Under the same performance metric, we proceed to show that the quadratic-time MMD based two-sample tests are also optimal for general two-sample problems, provided that kernels are bounded continuous and characteristic. Key to our approach are Sanov’s theorem from large deviation theory and the weak metrizable properties of the MMD and KSD. 

   more » « less
2. Two-sample statistics based on anisotropic kernels

   https://doi.org/10.1093/imaiai/iaz018  

   Cheng, Xiuyuan; Cloninger, Alexander; Coifman, Ronald R (December 2019, Information and Inference: A Journal of the IMA)

   Abstract The paper introduces a new kernel-based Maximum Mean Discrepancy (MMD) statistic for measuring the distance between two distributions given finitely many multivariate samples. When the distributions are locally low-dimensional, the proposed test can be made more powerful to distinguish certain alternatives by incorporating local covariance matrices and constructing an anisotropic kernel. The kernel matrix is asymmetric; it computes the affinity between $$n$$ data points and a set of $$n_R$$ reference points, where $$n_R$$ can be drastically smaller than $$n$$. While the proposed statistic can be viewed as a special class of Reproducing Kernel Hilbert Space MMD, the consistency of the test is proved, under mild assumptions of the kernel, as long as $$\|p-q\| \sqrt{n} \to \infty $$, and a finite-sample lower bound of the testing power is obtained. Applications to flow cytometry and diffusion MRI datasets are demonstrated, which motivate the proposed approach to compare distributions. 

   more » « less
3. Self-Discrepancy Conditional Independence Test

   Lee, Sanghack; Honavar, Vasant G (January 2017, Uncertainty in artificial intelligence)

   Tests of conditional independence (CI) of ran- dom variables play an important role in ma- chine learning and causal inference. Of partic- ular interest are kernel-based CI tests which allow us to test for independence among ran- dom variables with complex distribution func- tions. The efficacy of a CI test is measured in terms of its power and its calibratedness. We show that the Kernel CI Permutation Test (KCIPT) suffers from a loss of calibratedness as its power is increased by increasing the number of bootstraps. To address this limita- tion, we propose a novel CI test, called Self- Discrepancy Conditional Independence Test (SDCIT). SDCIT uses a test statistic that is a modified unbiased estimate of maximum mean discrepancy (MMD), the largest difference in the means of features of the given sample and its permuted counterpart in the kernel-induced Hilbert space. We present results of experi- ments that demonstrate SDCIT is, relative to the other methods: (i) competitive in terms of its power and calibratedness, outperforming other methods when the number of condition- ing variables is large; (ii) more robust with re- spect to the choice of the kernel function; and (iii) competitive in run time. 

   more » « less
4. Pitman efficiency lower bounds for multivariate distribution-free tests based on optimal transport

   https://doi.org/10.1093/jrsssb/qkaf072  

   Deb, Nabarun; Bhattacharya, Bhaswar B; Sen, Bodhisattva (February 2026, Journal of the Royal Statistical Society Series B: Statistical Methodology)

   Abstract The Wilcoxon rank sum test is one of the most popular distribution-free two-sample tests for univariate data. Among the important reasons for their popularity are the striking results of Hodges–Lehmann and Chernoff–Savage, where the authors show that the asymptotic (Pitman) relative efficiency of Wilcoxon’s test compared to Student’s t-test, never falls below 0.864 (with identity score) and 1 (with Gaussian score), respectively. Motivated by these results, we propose and study a large family of exactly distribution-free multivariate rank-based two-sample tests by leveraging the theory of optimal transport. First, we propose distribution-free analogues of the Hotelling T2 test and show that they satisfy Hodges–Lehmann and Chernoff–Savage-type efficiency lower bounds over natural sub-families of multivariate distributions—making them the first multivariate, nonparametric, finite-sample distribution-free tests that provably achieve such efficiency lower bounds. Next, we propose exactly distribution-free versions of the celebrated kernel maximum mean discrepancy test. In addition to being distribution-free in finite-samples, these tests are universally consistent under no moment assumptions and have nontrivial Pitman efficiency. To the best of our knowledge, these are the first class of tests to have this trifecta of properties. 

   more » « less
5. A Regularization-Based Adaptive Test for High-Dimensional Generalized Linear Models

   Wu, C; Xu, G; Shen, X. Pan (June 2020, Journal of machine learning research)

   In spite of its urgent importance in the era of big data, testing high-dimensional parameters in generalized linear models (GLMs) in the presence of high-dimensional nuisance parameters has been largely under-studied, especially with regard to constructing powerful tests for general (and unknown) alternatives. Most existing tests are powerful only against certain alternatives and may yield incorrect Type I error rates under high-dimensional nuisance parameter situations. In this paper, we propose the adaptive interaction sum of powered score (aiSPU) test in the framework of penalized regression with a non-convex penalty, called truncated Lasso penalty (TLP), which can maintain correct Type I error rates while yielding high statistical power across a wide range of alternatives. To calculate its p-values analytically, we derive its asymptotic null distribution. Via simulations, its superior finite-sample performance is demonstrated over several representative existing methods. In addition, we apply it and other representative tests to an Alzheimer’s Disease Neuroimaging Initiative (ADNI) data set, detecting possible gene-gender interactions for Alzheimer’s disease. We also put R package “aispu” implementing the proposed test on GitHub. 

   more » « less