More Like this

1. The Maximum Separation Subspace in Sufficient Dimension Reduction with Categorical Response

   Zhang, X.; Mai, Q.; Zou, H. (January 2020, Journal of machine learning research)

   Sufficient dimension reduction (SDR) is a very useful concept for exploratory analysis and data visualization in regression, especially when the number of covariates is large. Many SDR methods have been proposed for regression with a continuous response, where the central subspace (CS) is the target of estimation. Various conditions, such as the linearity condition and the constant covariance condition, are imposed so that these methods can estimate at least a portion of the CS. In this paper we study SDR for regression and discriminant analysis with categorical response. Motivated by the exploratory analysis and data visualization aspects of SDR, we propose a new geometric framework to reformulate the SDR problem in terms of manifold optimization and introduce a new concept called Maximum Separation Subspace (MASES). The MASES naturally preserves the “sufficiency” in SDR without imposing additional conditions on the predictor distribution, and directly inspires a semi-parametric estimator. Numerical studies show MASES exhibits superior performance as compared with competing SDR methods in specific settings. 

   more » « less
2. Supervised Dimension Reduction via Local Gradient Elongation

   https://doi.org/10.1007/978-3-031-87804-6_9  

   Chhoa, Jannatul Ferdous; Huang, Longxiu; Little, Anna; Maurais, Aimee; Morris, Kirsten D; Walt, Maria_D_van der; Verma, Geetika; Wang, Rongrong (January 2025, Springer Nature Switzerland)

   Garcia-Cardona, Cristina; Lee, Harlin (Ed.)

   We define a new local metric for labeled data, or features, which elongates the standard Euclidean distance in the direction of the label gradient, and explore its utility for supervised dimension reduction (SDR). In particular, we create supervised low-dimensional feature representations by embedding a gradient elongated geodesic distance. Experiments indicate that this procedure leads to faithful visualization of the underlying structure of the feature-label relationship, i.e., the active manifold, when other SDR methods fail. Furthermore, we propose a new method for prediction on unlabeled points, which extends Laplacian learning to leverage the gradient elongated geometry. Extensive numerical comparisons with classic and neural network-based methods indicate that such an approach outperforms other prediction methods when the number of labeled points is small. Finally, we validate the proposed methodology on several real-world datasets, including data on acute respiratory syndrome-coronavirus-2 and data on human stem cell differentiation. 

   more » « less
3. Inverse moment methods for sufficient forecasting using high-dimensional predictors

   https://doi.org/10.1093/biomet/asab037  

   Luo, Wei; Xue, Lingzhou; Yao, Jiawei; Yu, Xiufan (June 2021, Biometrika)

   Summary We consider forecasting a single time series using a large number of predictors in the presence of a possible nonlinear forecast function. Assuming that the predictors affect the response through the latent factors, we propose to first conduct factor analysis and then apply sufficient dimension reduction on the estimated factors to derive the reduced data for subsequent forecasting. Using directional regression and the inverse third-moment method in the stage of sufficient dimension reduction, the proposed methods can capture the nonmonotone effect of factors on the response. We also allow a diverging number of factors and only impose general regularity conditions on the distribution of factors, avoiding the undesired time reversibility of the factors by the latter. These make the proposed methods fundamentally more applicable than the sufficient forecasting method of Fan et al. (2017). The proposed methods are demonstrated both in simulation studies and an empirical study of forecasting monthly macroeconomic data from 1959 to 2016. Also, our theory contributes to the literature of sufficient dimension reduction, as it includes an invariance result, a path to perform sufficient dimension reduction under the high-dimensional setting without assuming sparsity, and the corresponding order-determination procedure. 

   more » « less
4. Learning Fair Representations for Kernel Models

   Zilong Tan, Samuel Yeom (January 2020, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics)

   Fair representations are a powerful tool for establishing criteria like statistical parity, proxy non-discrimination, and equality of opportunity in learned models. Existing techniques for learning these representations are typically model-agnostic, as they preprocess the original data such that the output satisfies some fairness criterion, and can be used with arbitrary learning methods. In contrast, we demonstrate the promise of learning a model-aware fair representation, focusing on kernel-based models. We leverage the classical Sufficient Dimension Reduction (SDR) framework to construct representations as subspaces of the reproducing kernel Hilbert space (RKHS), whose member functions are guaranteed to satisfy fairness. Our method supports several fairness criteria, continuous and discrete data, and multiple protected attributes. We further show how to calibrate the accuracy tradeoff by characterizing it in terms of the principal angles between subspaces of the RKHS. Finally, we apply our approach to obtain the first Fair Gaussian Process (FGP) prior for fair Bayesian learning, and show that it is competitive with, and in some cases outperforms, state-of-the-art methods on real data. 

   more » « less
5. ROBUST INFERENCE FOR HETEROGENEOUS TREATMENT EFFECTS WITH APPLICATIONS TO NHANES DATA

   https://doi.org/10.25394/pgs.27910248  

   Mo, Ran (January 2025, Purdue University Graduate School)

   Abstract: Estimating the conditional average treatment effect (CATE) using data from the National Health and Nutrition Examination Survey (NHANES) provides valuable insights into the heterogeneousimpacts of health interventions across diverse populations, facilitating public health strategies that consider individual differences in health behaviors and conditions. However, estimating CATE with NHANES data face challenges often encountered in observational studies, such as outliers, heavy-tailed error distributions, skewed data, model misspecification, and the curse of dimensionality. To address these challenges, this dissertation presents three consecutive studies that thoroughly explore robust methods for estimating heterogeneous treatment effects. The first study introduces an outlier-resistant estimation method by incorporating M-estimation, replacing the \(L_2\) loss in the traditional inverse propensity weighting (IPW) method with a robust loss function. To assess the robustness of our approach, we investigate its influence function and breakdown point. Additionally, we derive the asymptotic properties of the proposed estimator, enabling valid inference for the proposed outlier-resistant estimator of CATE.The method proposed in the first study relies on a symmetric assumption which is commonly required by standard outlier-resistant methods. To remove this assumption while maintaining unbiasedness, the second study employs the adaptive Huber loss, which dynamically adjusts the robustification parameter based on the sample size to achieve optimal tradeoff between bias and robustness. The robustification parameter is explicitly derived from theoretical results, making it unnecessary to rely on time-consuming data-driven methods for its selection.We also derive concentration and Berry-Esseen inequalities to precisely quantify the convergence rates as well as finite sample performance.In both previous studies, the propensity scores were estimated parametrically, which is sensitive to model misspecification issues. The third study extends the robust estimator from our first project by plugging in a kernel-based nonparametric estimation of the propensity score with sufficient dimension reduction (SDR). Specifically, we adopt a robust minimum average variance estimation (rMAVE) for the central mean space under the potential outcome framework. Together with higher-order kernels, the resulting CATE estimation gains enhanced efficiency.In all three studies, the theoretical results are derived, and confidence intervals are constructed for inference based on these findings. The properties of the proposed estimators are verified through extensive simulations. Additionally, applying these methods to NHANES data validates the estimators' ability to handle diverse and contaminated datasets, further demonstrating their effectiveness in real-world scenarios. 

   more » « less