An official website of the United States government
We study robust testing and estimation of discrete distributions in the strong contamination model. Our results cover both centralized setting and distributed setting with general local information constraints including communication and LDP constraints. Our technique relates the strength of manipulation attacks to the earth-mover distance using Hamming distance as the metric between messages (samples) from the users. In the centralized setting, we provide optimal error bounds for both learning and testing. Our lower bounds under local information constraints build on the recent lower bound methods in distributed inference. In the communication constrained setting, we develop novel algorithms based on random hashing and an L1-L1 isometry.
more »
« less
This content will become publicly available on January 1, 2027
Distributed robust estimation and inference with contaminated data
We study the problem of robust estimation and inference for contaminated data in distributed learning systems. Existing methods, such as those designed for Byzantine failures, typically assume that contamination is limited to a small subset of machines experiencing complete failure. In contrast, we address distributed partial contamination, where all machines may handle datasets containing some corrupted observations. By generalizing Huber’s 𝜖-contamination model to distributed settings, we propose a robust framework in which each machine’s dataset may be contaminated by a proportion 𝜖 of observations from an arbitrary distribution. In a linear model setting, we develop a communication-efficient M-estimator that achieves the optimal convergence rate of cen- tralized data. For robust inference, we introduce a distributed multiplier bootstrap method that requires no additional communication post-estimation while maintaining efficiency. To handle high contamination proportions, we present a debiasing procedure to mitigate bias. Extensive simulations demonstrate the robustness and scalability of our methods across diverse contamination scenarios.
more »
« less
- Award ID(s):
- 2113568
- PAR ID:
- 10699869
- Publisher / Repository:
- the Institute of Mathematical Statistics (IMS) and the Bernoulli Society
- Date Published:
- Journal Name:
- Electronic Journal of Statistics
- Volume:
- 20
- Issue:
- 1
- ISSN:
- 1935-7524
- Page Range / eLocation ID:
- 261-323
- Subject(s) / Keyword(s):
- Bootstrap inference communication-efficient estimation distributed learning Huber’s contamination model M-estimator robust regression
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Loh, Po-Ling (Ed.)When data are distributed across multiple sites or machines rather than centralized in one location, researchers face the challenge of extracting meaningful information without directly sharing individual data points. While there are many distributed methods for point estimation using sparse regression, few options are available for estimating uncertainties or conducting hypothesis tests based on the estimated sparsity. In this paper, we introduce a procedure for performing selective inference with distributed data. We consider a scenario where each local machine solves a lasso problem and communicates the selected predictors to a central machine. The central machine then aggregates these selected predictors to form a generalized linear model (GLM). Our goal is to provide valid inference for the selected GLM while reusing data that have been used in the model selection process. Our proposed procedure only requires low-dimensional summary statistics from local machines, thus keeping communication costs low and preserving the privacy of individual data sets. Furthermore, this procedure can be applied in scenarios where model selection is repeatedly conducted on randomly subsampled data sets, addressing the p-value lottery problem linked with model selection. We demonstrate the effectiveness of our approach through simulations and an analysis of a medical data set on ICU admissions.more » « less
-
Abstract: Estimating the conditional average treatment effect (CATE) using data from the National Health and Nutrition Examination Survey (NHANES) provides valuable insights into the heterogeneousimpacts of health interventions across diverse populations, facilitating public health strategies that consider individual differences in health behaviors and conditions. However, estimating CATE with NHANES data face challenges often encountered in observational studies, such as outliers, heavy-tailed error distributions, skewed data, model misspecification, and the curse of dimensionality. To address these challenges, this dissertation presents three consecutive studies that thoroughly explore robust methods for estimating heterogeneous treatment effects. The first study introduces an outlier-resistant estimation method by incorporating M-estimation, replacing the \(L_2\) loss in the traditional inverse propensity weighting (IPW) method with a robust loss function. To assess the robustness of our approach, we investigate its influence function and breakdown point. Additionally, we derive the asymptotic properties of the proposed estimator, enabling valid inference for the proposed outlier-resistant estimator of CATE.The method proposed in the first study relies on a symmetric assumption which is commonly required by standard outlier-resistant methods. To remove this assumption while maintaining unbiasedness, the second study employs the adaptive Huber loss, which dynamically adjusts the robustification parameter based on the sample size to achieve optimal tradeoff between bias and robustness. The robustification parameter is explicitly derived from theoretical results, making it unnecessary to rely on time-consuming data-driven methods for its selection.We also derive concentration and Berry-Esseen inequalities to precisely quantify the convergence rates as well as finite sample performance.In both previous studies, the propensity scores were estimated parametrically, which is sensitive to model misspecification issues. The third study extends the robust estimator from our first project by plugging in a kernel-based nonparametric estimation of the propensity score with sufficient dimension reduction (SDR). Specifically, we adopt a robust minimum average variance estimation (rMAVE) for the central mean space under the potential outcome framework. Together with higher-order kernels, the resulting CATE estimation gains enhanced efficiency.In all three studies, the theoretical results are derived, and confidence intervals are constructed for inference based on these findings. The properties of the proposed estimators are verified through extensive simulations. Additionally, applying these methods to NHANES data validates the estimators' ability to handle diverse and contaminated datasets, further demonstrating their effectiveness in real-world scenarios.more » « less
-
Navigation problems are generally solved applying least-squares (LS) adjustments. Techniques based on LS can be shown to perform optimally when the system noise is Gaussian distributed and the parametric model is accurately known. Unfortunately, real world problems usually contain unexpectedly large errors, so-called outliers, that violate the noise model assumption, leading to a spoiled solution estimation. In this work, the framework of robust statistics is explored to provide robust solutions to the global navigation satellite systems (GNSS) single point positioning (SPP) problem. Considering that GNSS observables may be contaminated by erroneous measurements, we survey the most popular approaches for robust regression (M-, S-, and MM-estimators) and how they can be adapted into a general methodology for robust GNSS positioning. We provide both theoretical insights and validation over experimental datasets, which serves in discussing the robust methods in detail.more » « less
-
ABSTRACT Bounded count data are commonly encountered in environmental studies. This paper examines two environmental applications illustrating their relevance. The first investigates the effect of winter malnutrition on mule deer (Odocoileus hemionus) fawn mortality. The second application analyzes public perceptions of environmental issues using data from the Eurobarometer 95.1 survey (March–April 2021), which includes a question rating the perceived severity of climate change on a scale from 1 to 10. Together, these studies demonstrate the need for flexible bounded count models in environmental research. In this context, the binomial and beta‐binomial (BB) models are widely used for bounded count data, with the BB model offering the advantage of accounting for overdispersion. However, atypical observations in real‐world applications may hinder the performance of the BB model and lead to biased or misleading inferences. To address this limitation, we propose the contaminated beta‐binomial (cBB) distribution (cBB‐D), which introduces an additional BB component to accommodate atypical observations while preserving the mean and variance structure of the BB model. The cBB‐D thus captures both overdispersion and contamination effects in bounded count data. To incorporate explanatory variables, we further develop the contaminated BB regression model (cBB‐RM), in which none, some, or all cBB parameters may depend on covariates. The proposed models are applied to two environmental datasets, complemented by a sensitivity analysis on simulated data to assess the influence of atypical observations on parameter estimation. The methodology is implemented in the open‐sourcecBBpackage forR, available athttps://github.com/arnootto/cBB.more » « less
