More Like this

1. The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learning

   Wang, Kaiwen; Zhou, Kevin; Wu, Runzhe; Kallus, Nathan; Sun, Wen (December 2023, 37th Conference on Neural Information Processing Systems (NeurIPS 2023))

   While distributional reinforcement learning (DistRL) has been empirically effective, the question of when and why it is better than vanilla, non-distributional RL has remained unanswered. This paper explains the benefits of DistRL through the lens of small-loss bounds, which are instance-dependent bounds that scale with optimal achievable cost. Particularly, our bounds converge much faster than those from non-distributional approaches if the optimal cost is small. As warmup, we propose a distributional contextual bandit (DistCB) algorithm, which we show enjoys small-loss regret bounds and empirically outperforms the state-of-the-art on three real-world tasks. In online RL, we propose a DistRL algorithm that constructs confidence sets using maximum likelihood estimation. We prove that our algorithm enjoys novel small-loss PAC bounds in low-rank MDPs. As part of our analysis, we introduce the l1 distributional eluder dimension which may be of independent interest. Then, in offline RL, we show that pessimistic DistRL enjoys small-loss PAC bounds that are novel to the offline setting and are more robust to bad single-policy coverage. 

   more » « less
2. Superpolynomial Lower Bounds for Learning One-Layer Neural Networks using Gradient Descent

   Goel, Surbhi; Gollakota, Aravind; Jin, Zhihan; Karmalkar, Sushrut; Klivans, Adam (July 2020, International Conference on Machine Learning)

   We prove the first superpolynomial lower bounds for learning one-layer neural networks with respect to the Gaussian distribution using gradient descent. We show that any classifier trained using gradient descent with respect to square-loss will fail to achieve small test error in polynomial time given access to samples labeled by a one-layer neural network. For classification, we give a stronger result, namely that any statistical query (SQ) algorithm (including gradient descent) will fail to achieve small test error in polynomial time. Prior work held only for gradient descent run with small batch sizes, required sharp activations, and applied to specific classes of queries. Our lower bounds hold for broad classes of activations including ReLU and sigmoid. The core of our result relies on a novel construction of a simple family of neural networks that are exactly orthogonal with respect to all spherically symmetric distributions. 

   more » « less
3. CRISP: Consensus Regularized Selection based Prediction

   https://doi.org/10.1145/2983323.2983779  

   Wang, Ping; Padthe, Karthik K.; Vinzamuri, Bhanukiran; Reddy, Chandan K. (January 2016, Proceedings of the 25th ACM International on Conference on Information and Knowledge Management)

   Integrating regularization methods with standard loss functions such as the least squares, hinge loss, etc., within a regression framework has become a popular choice for researchers to learn predictive models with lower variance and better generalization ability. Regularizers also aid in building interpretable models with high-dimensional data which makes them very appealing. It is observed that each regularizer is uniquely formulated in order to capture data-specific properties such as correlation, structured sparsity and temporal smoothness. The problem of obtaining a consensus among such diverse regularizers while learning a predictive model is extremely important in order to determine the optimal regularizer for the problem. The advantage of such an approach is that it preserves the simplicity of the final model learned by selecting a single candidate model which is not the case with ensemble methods as they use multiple candidate models for prediction. This is called the consensus regularization problem which has not received much attention in the literature due to the inherent difficulty associated with learning and selecting a model from an integrated regularization framework. To solve this problem, in this paper, we propose a method to generate a committee of non-convex regularized linear regression models, and use a consensus criterion to determine the optimal model for prediction. Each corresponding non-convex optimization problem in the committee is solved efficiently using the cyclic-coordinate descent algorithm with the generalized thresholding operator. Our Consensus RegularIzation Selection based Prediction (CRISP) model is evaluated on electronic health records (EHRs) obtained from a large hospital for the congestive heart failure readmission prediction problem. We also evaluate our model on high-dimensional synthetic datasets to assess its performance. The results indicate that CRISP outperforms several state-of-the-art methods such as additive, interactions-based and other competing non-convex regularized linear regression methods. 

   more » « less
4. Scanner: Simultaneously temporal trend and spatial cluster detection for spatial‐temporal data

   https://doi.org/10.1002/env.2849  

   Wang, Xin; Zhang, Xin (April 2024, Environmetrics)

   Abstract Identifying the underlying trajectory pattern in the spatial‐temporal data analysis is a fundamental but challenging task. In this paper, we study the problem of simultaneously identifying temporal trends and spatial clusters of spatial‐temporal trajectories. To achieve this goal, we propose a novel method named spatial clustered and sparse nonparametric regression (). Our method leverages the B‐spline model to fit the temporal data and penalty terms on spline coefficients to reveal the underlying spatial‐temporal patterns. In particular, our method estimates the model by solving a doubly‐penalized least square problem, in which we use a group sparse penalty for trend detection and a spanning tree‐based fusion penalty for spatial cluster recovery. We also develop an algorithm based on the alternating direction method of multipliers (ADMM) algorithm to efficiently minimize the penalized least square loss. The statistical consistency properties of estimator are established in our work. In the end, we conduct thorough numerical experiments to verify our theoretical findings and validate that our method outperforms the existing competitive approaches. 

   more » « less
5. Deep Neural Networks for Estimation and Inference

   https://doi.org/10.3982/ECTA16901  

   Farrell, Max H.; Liang, Tengyuan; Misra, Sanjog (January 2021, Econometrica)

   We study deep neural networks and their use in semiparametric inference. We establish novel nonasymptotic high probability bounds for deep feedforward neural nets. These deliver rates of convergence that are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second‐step inference after first‐step estimation with deep learning, a result also new to the literature. Our nonasymptotic high probability bounds, and the subsequent semiparametric inference, treat the current standard architecture: fully connected feedforward neural networks (multilayer perceptrons), with the now‐common rectified linear unit activation function, unbounded weights, and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed‐width, very deep networks. We establish the nonasymptotic bounds for these deep nets for a general class of nonparametric regression‐type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, and demonstrate the effectiveness of deep learning with an empirical application to direct mail marketing. 

   more » « less