More Like this

1. On relationships between Chatterjee's and Spearman's correlation coefficients

   Qingyang Zhang (February 2023, arXivorg)

   In his seminal work, Chatterjee (2021) introduced a novel correlation measure which is distribution-free, asymptotically normal, and consistent against all alternatives. In this paper, we study the probabilistic relationships between Chatterjee's correlation and the widely used Spearman's correlation. We show that, under independence, the two sample-based correlations are asymptotically joint normal and asymptotically independent. Under dependence, the magnitudes of two correlations can be substantially different. We establish some extremal cases featuring large differences between these two correlations. Motivated by these findings, a new independence test is proposed by combining Chatterjee's and Spearman's correlations into a maximal strength measure of variable association. Our simulation study and real data application show the good sensitivity of the new test to different correlation patterns. 

   more » « less
2. On the power of Chatterjee’s rank correlation

   https://doi.org/10.1093/biomet/asab028  

   Shi, H; Drton, M; Han, F (October 2021, Biometrika)

   Summary Chatterjee (2021) introduced a simple new rank correlation coefficient that has attracted much attention recently. The coefficient has the unusual appeal that it not only estimates a population quantity first proposed by Dette et al. (2013) that is zero if and only if the underlying pair of random variables is independent, but also is asymptotically normal under independence. This paper compares Chatterjee’s new correlation coefficient with three established rank correlations that also facilitate consistent tests of independence, namely Hoeffding’s $$D$$, Blum–Kiefer–Rosenblatt’s $$R$$, and Bergsma–Dassios–Yanagimoto’s $$\tau^*$$. We compare the computational efficiency of these rank correlation coefficients in light of recent advances, and investigate their power against local rotation and mixture alternatives. Our main results show that Chatterjee’s coefficient is unfortunately rate-suboptimal compared to $$D$$, $$R$$ and $$\tau^*$$. The situation is more subtle for a related earlier estimator of Dette et al. (2013). These results favour $$D$$, $$R$$ and $$\tau^*$$ over Chatterjee’s new correlation coefficient for the purpose of testing independence. 

   more » « less
3. On the failure of the bootstrap for Chatterjee’s rank correlation

   https://doi.org/10.1093/biomet/asae004  

   Lin, Zhexiao; Han, Fang (February 2024, Biometrika)

   Abstract While researchers commonly use the bootstrap to quantify the uncertainty of an estimator, it has been noticed that the standard bootstrap, in general, does not work for Chatterjee’s rank correlation. In this paper, we provide proof of this issue under an additional independence assumption, and complement our theory with simulation evidence for general settings. Chatterjee’s rank correlation thus falls into a category of statistics that are asymptotically normal, but bootstrap inconsistent. Valid inferential methods in this case are Chatterjee’s original proposal for testing independence and the analytic asymptotic variance estimator of Lin & Han (2022) for more general purposes. [Received on 5 April 2023. Editorial decision on 10 January 2024] 

   more » « less
4. On boosting the power of Chatterjee’s rank correlation

   https://doi.org/10.1093/biomet/asac048  

   Lin, Z.; Han, F. (August 2022, Biometrika)

   Summary The ingenious approach of Chatterjee (2021) to estimate a measure of dependence first proposed by Dette et al. (2013) based on simple rank statistics has quickly caught attention. This measure of dependence has the appealing property of being between 0 and 1, and being 0 or 1 if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely. However, more recent studies (Cao & Bickel 2020; Shi et al. 2022b) showed that independence tests based on Chatterjee’s rank correlation are unfortunately rate inefficient against various local alternatives and they call for variants. We answer this call by proposing an improvement to Chatterjee’s rank correlation that still consistently estimates the same dependence measure, but provably achieves near-parametric efficiency in testing against Gaussian rotation alternatives. This is possible by incorporating many right nearest neighbours in constructing the correlation coefficients. We thus overcome the ‘ only one disadvantage’ of Chatterjee’s rank correlation (Chatterjee, 2021, § 7). 

   more » « less
5. Investigating Sub-soil Moisture Variation of Engineered Turf Cover for Landfills Through Field Instrumentation

   Md. Jobair Bin Alam, Maalvika Aggarwal (March 2023, Atlantis highlights in engineering)

   S. Javankhoshdel and Y. Abolfazlzadeh (Ed.)

   Understanding the moisture distribution pattern and associated suction variability of soil in response to environmental loading (e.g., precipitation, temperature) is important. However, there is a lack of understanding of the spatial variability of moisture and suction in different final cover systems. In this study, the spatial correlations between soil moisture and suction data from field instrumentation are examined using Spearman’s rank correlation test of three different types of landfill final cover systems: evapotranspiration (ET) cover, conventional clay cover, and engineered turf cover, under identical atmospheric conditions. In addition, box and whiskers plots were used to investigate the distribution of the field-measured data under environmental fluctuation. As observed from the box plot, soil moisture displayed maximum spatial heterogeneity in clay cover and very less in the engineered turf cover under identical environmental conditions. The ET cover exhibited a very strong spatial correlation of moisture and suction as indicated by the highly significant Spearman’s rank correlations (rs) ranging from −0.88 to −0.93. The clay cover showed a strong to moderate correlation (−0.51 < rs < −0.74) between the spatial distribution of moisture and suction. On the other hand, the engineered turf cover displayed poor agreement of the spatial moisture-suction distribution implying the soil under the engineered turf is relatively non-responsive under environmental variability compared to clay and ET cover. The preliminary findings from this study showed engineered turf’s capacity to maintain more moisture stability of the turf under the humid subtropical climate than other landfill covers. 

   more » « less