More Like this

1. Interactive rank testing by betting

   Duan, Boyan ; Ramdas, Aaditya ; Wasserman, Larry ( April 2022 , First Conference on Causal Learning and Reasoning, PMLR)

   Scholkopf, Bernhard ; Uhler, Caroline ; Zhang, Kun (Ed.)

   In order to test if a treatment is perceptibly different from a placebo in a randomized experiment with covariates, classical nonparametric tests based on ranks of observations/residuals have been employed (eg: by Rosenbaum), with finite-sample valid inference enabled via permutations. This paper proposes a different principle on which to base inference: if — with access to all covariates and outcomes, but without access to any treatment assignments — one can form a ranking of the subjects that is sufficiently nonrandom (eg: mostly treated followed by mostly control), then we can confidently conclude that there must be a treatment effect. Based on a more nuanced, quantifiable, version of this principle, we design an interactive test called i-bet: the analyst forms a single permutation of the subjects one element at a time, and at each step the analyst bets toy money on whether that subject was actually treated or not, and learns the truth immediately after. The wealth process forms a real-valued measure of evidence against the global causal null, and we may reject the null at level if the wealth ever crosses 1= . Apart from providing a fresh “game-theoretic” principle on which to base the causal conclusion, the i-bet has other statistical and computational benefits, for example (A) allowing a human to adaptively design the test statistic based on increasing amounts of data being revealed (along with any working causal models and prior knowledge), and (B) not requiring permutation resampling, instead noting that under the null, the wealth forms a nonnegative martingale, and the type-1 error control of the aforementioned decision rule follows from a tight inequality by Ville. Further, if the null is not rejected, new subjects can later be added and the test can be simply continued, without any corrections (unlike with permutation p-values). Numerical experiments demonstrate good power under various heterogeneous treatment effects. We first describe i-bet test for two-sample comparisons with unpaired data, and then adapt it to paired data, multi-sample comparison, and sequential settings; these may be viewed as interactive martingale variants of the Wilcoxon, Kruskal-Wallis, and Friedman tests. 

   more » « less
2. Testing Mutual Independence in High Dimension via Distance Covariance

   https://doi.org/10.1111/rssb.12259  

   Yao, Shun ; Zhang, Xianyang ; Shao, Xiaofeng ( October 2017 , Journal of the Royal Statistical Society Series B: Statistical Methodology)

   We introduce an L2-type test for testing mutual independence and banded dependence structure for high dimensional data. The test is constructed on the basis of the pairwise distance covariance and it accounts for the non-linear and non-monotone dependences among the data, which cannot be fully captured by the existing tests based on either Pearson correlation or rank correlation. Our test can be conveniently implemented in practice as the limiting null distribution of the test statistic is shown to be standard normal. It exhibits excellent finite sample performance in our simulation studies even when the sample size is small albeit the dimension is high and is shown to identify non-linear dependence in empirical data analysis successfully. On the theory side, asymptotic normality of our test statistic is shown under quite mild moment assumptions and with little restriction on the growth rate of the dimension as a function of sample size. As a demonstration of good power properties for our distance-covariance-based test, we further show that an infeasible version of our test statistic has the rate optimality in the class of Gaussian distributions with equal correlation.

    

   more » « less
3. Inference on finite-population treatment effects under limited overlap

   https://doi.org/10.1093/ectj/utz017  

   Hong, Han ; Leung, Michael P. ; Li, Jessie ( September 2019 , The Econometrics Journal)

   This paper studies inference on finite-population average and local average treatment effects under limited overlap, meaning that some strata have a small proportion of treated or untreated units. We model limited overlap in an asymptotic framework, sending the propensity score to zero (or one) with the sample size. We derive the asymptotic distribution of analogue estimators of the treatment effects under two common randomization schemes: conditionally independent and stratified block randomization. Under either scheme, the limit distribution is the same and conventional standard error formulas remain asymptotically valid, but the rate of convergence is slower the faster the propensity score degenerates. The practical import of these results is two-fold. When overlap is limited, standard methods can perform poorly in smaller samples, as asymptotic approximations are inadequate owing to the slower rate of convergence. However, in larger samples, standard methods can work quite well even when the propensity score is small.

    

   more » « less
4. Addressing patient heterogeneity in disease predictive model development

   https://doi.org/10.1111/biom.13514  

   Gao, Xu ; Shen, Weining ; Ning, Jing ; Feng, Ziding ; Hu, Jianhua ( August 2021 , Biometrics)

   This paper addresses patient heterogeneity associated with prediction problems in biomedical applications. We propose a systematic hypothesis testing approach to determine the existence of patient subgroup structure and the number of subgroups in patient population if subgroups exist. A mixture of generalized linear models is considered to model the relationship between the disease outcome and patient characteristics and clinical factors, including targeted biomarker profiles. We construct a test statistic based on expectation maximization (EM) algorithm and derive its asymptotic distribution under the null hypothesis. An important computational advantage of the test is that the involved parameter estimates under the complex alternative hypothesis can be obtained through a small number of EM iterations, rather than optimizing the objective function. We demonstrate the finite sample performance of the proposed test in terms of type‐I error rate and power, using extensive simulation studies. The applicability of the proposed method is illustrated through an application to a multicenter prostate cancer study.

    

   more » « less
5. A Smoothing-Based Goodness-of-Fit Test of Covariance for Functional Data

   https://doi.org/10.1111/biom.13005  

   Chen, Stephanie T. ; Xiao, Luo ; Staicu, Ana-Maria ( November 2018 , Biometrics)

   Functional data methods are often applied to longitudinal data as they provide a more flexible way to capture dependence across repeated observations. However, there is no formal testing procedure to determine if functional methods are actually necessary. We propose a goodness-of-fit test for comparing parametric covariance functions against general nonparametric alternatives for both irregularly observed longitudinal data and densely observed functional data. We consider a smoothing-based test statistic and approximate its null distribution using a bootstrap procedure. We focus on testing a quadratic polynomial covariance induced by a linear mixed effects model and the method can be used to test any smooth parametric covariance function. Performance and versatility of the proposed test is illustrated through a simulation study and three data applications.

    

   more » « less